Live commentary feed

Business constraint

Sports/live ops commentary: many readers, ordered events, late joiners need catch-up.

Naive v0

Replay entire history to each subscriber from RAM. Block on slow clients.

Failure drill

Memory cliffs. Unfair latency. Reconnect storms after halftime.

Evolution path

Iteration 1

Append-only event log with offsets; live hub for tail (reuse ws-fanout ideas).

Iteration 2

Bounded retention; snapshot + delta for late joiners.

Iteration 3

Shed slow readers; measure catch-up separately from live lag.

Implementation cut

Define ordering and retention as product contracts. Test late-join catch-up size limits.

Numbers

Live lag p99. Catch-up bytes/client. Disconnect rate for slow consumers.

Pattern tags

• append-only-log
• backpressure

Self-check

1. Why offsets matter for late joiners?
2. How do snapshots help?
3. What is unfair about blocking fan-out?
4. How do you bound retention?
5. What metrics split live vs catch-up?

Walkthrough

Recordings will appear here when published. Until then, work the failure drill and lab locally — pause after each iteration and write the metric that moved.

Lab package: see /systems/labs/ws-fanout/.