CDC Backfill Race
A backfill front sweeps forward through history on the lower rail while the live tail creeps on above it, the two fronts converging on the same offset.
A backfill front sweeps forward through history on the lower rail while the live tail creeps on above it, the two fronts converging on the same offset.
Records fed from three partition lanes arrive at the sequencer at different moments, and the output staircase commits strictly left to right in log sequence.
A snapshot band sweeps down a grid of table rows and locks each row it crosses into a consistent cut, while writes keep landing on the log above it.
A write head races ahead of a read cursor on the same log rail, opening a measured lag span that then closes as the consumer catches up.
A malformed record fails the parse gate and is diverted down the siding into the dead-letter tray while the rest of the stream carries on unbroken.
Two records with the same key reach the dedupe gate; the first is admitted to the sink and the second is turned back at the fence and dropped in the tray.
The same keyed record arrives again and again at the target table, and the matching row rewrites its value in place instead of the table growing.
A record arrives behind the cursor, the ordered rail opens a gap at its offset and the late record drops into its correct slot before the rail closes up.
A changelog of repeated keys is compacted: superseded records drop away and the surviving latest-per-key records close up as the rail contracts.
A read cursor advances along a write-ahead log rail and each record it passes brightens and grows as it is emitted, before the cursor re-acquires the head.
The cursor rewinds to a marked offset, the downstream column drops everything past it, and the bars rebuild one by one as the cursor re-reads forward.
Records flowing past the alter marker gain a fourth field, and the column legend below shows the reader's schema before and after the change.
A snapshot block fills to its final offset, the baton flashes at the seam marker and streamed records begin arriving in the slots from that point onward.
A keyed value record travelling the changelog is emptied at the delete gate and carries on downstream as a null-marked tombstone rather than vanishing.
A watermark rule advances across the event-time axis, sealing each window it passes while out-of-order events keep dropping in ahead of it.