Logtail synchronization and visibility¶
Baseline: v4.2.5. TN/TAE provides Logtail so CN can reconstruct timestamp-dependent read state. It is not another name for durable WAL.
Contents and reads¶
Synchronization carries committed inserts/deletes, transaction timestamps, data/tombstone object metadata and checkpoint locations. CN maintains PartitionState, selects snapshot-visible objects and combines memory rows with deletions. Flushing an object does not place every later deletion inside that object.
Initial subscription and increments¶
CN requests a table and [known timestamp, target timestamp]
→ TN collects checkpoint locations covering the range
→ return checkpoint only when it covers the target
→ otherwise collect increments after the covered endpoint
→ CN loads/applies state and advances readable progress
HandleSyncLogTailReq calls CollectCheckpointsInRange and checks its coverage endpoint. Comparing a subscription timestamp with a checkpoint timestamp alone is insufficient. The current table response builder visits catalog.ObjectEntry; historical Segment/Block callback interfaces are not the v4.2.5 API.
Push delivers increments after subscription. System Catalog subscription, ordinary tables and reconnection each establish state. Progress/heartbeat messages advance watermarks even without row changes; implementation/configuration cadence is not a latency SLA.
Visibility invariant¶
A transaction snapshot needs the required synchronized state. Commit time, target snapshot and applied watermark are separate markers. A successful client commit does not prove every CN has applied it. Reads must also incorporate the transaction’s own workspace.
Reconnection and memory¶
A broken stream requires subscription/state recovery; TCP reconnection alone does not establish catch-up. Flush/Merge metadata and deletion state govern removal of old memory entries. File presence or wall-clock age alone cannot safely drive cleanup.
For a write missing on another CN, confirm commit, target table and connection first, then subscription, snapshot, applied watermark/reconnect logs and object read errors. Separate checkpoint loading from increment construction when subscription is slow. Retain table IDs, requested ranges and response size.