Logtail synchronization and visibility

Baseline: v4.2.5. TN/TAE provides Logtail so CN can reconstruct timestamp-dependent read state. It is not another name for durable WAL.

Contents and reads

Synchronization carries committed inserts/deletes, transaction timestamps, data/tombstone object metadata and checkpoint locations. CN maintains PartitionState, selects snapshot-visible objects and combines memory rows with deletions. Flushing an object does not place every later deletion inside that object.

Initial subscription and increments

CN requests a table and [known timestamp, target timestamp]
  → TN collects checkpoint locations covering the range
  → return checkpoint only when it covers the target
  → otherwise collect increments after the covered endpoint
  → CN loads/applies state and advances readable progress

HandleSyncLogTailReq calls CollectCheckpointsInRange and checks its coverage endpoint. Comparing a subscription timestamp with a checkpoint timestamp alone is insufficient. The current table response builder visits catalog.ObjectEntry; historical Segment/Block callback interfaces are not the v4.2.5 API.

Push delivers increments after subscription. System Catalog subscription, ordinary tables and reconnection each establish state. Progress/heartbeat messages advance watermarks even without row changes; implementation/configuration cadence is not a latency SLA.

Visibility invariant

A transaction snapshot needs the required synchronized state. Commit time, target snapshot and applied watermark are separate markers. A successful client commit does not prove every CN has applied it. Reads must also incorporate the transaction’s own workspace.

Reconnection and memory

A broken stream requires subscription/state recovery; TCP reconnection alone does not establish catch-up. Flush/Merge metadata and deletion state govern removal of old memory entries. File presence or wall-clock age alone cannot safely drive cleanup.

For a write missing on another CN, confirm commit, target table and connection first, then subscription, snapshot, applied watermark/reconnect logs and object read errors. Separate checkpoint loading from increment construction when subscription is slow. Retain table IDs, requested ranges and response size.

Source and regression entry points