Scaling WebSockets

Business constraint

Concurrent connections grow past one machine's file descriptors and RAM. Rooms must not split-brain message order casually.

Naive v0

Round-robin L7 without stickiness. Each instance holds partial room state. Broadcast becomes best-effort chaos.

Failure drill

Users miss messages. Reconnect storms on deploy. Memory per connection underestimated (buffers, TLS, hub maps).

Evolution path

Iteration 1

Sticky sessions or consistent hashing by room id to concentrate room state. Drain connections gracefully on deploy.

Iteration 2

Move room truth to a shared layer when stickiness is insufficient; keep local hubs as caches of interest.

Iteration 3

Budget fds, RAM, and CPU per connection with load tests. Cap connections per instance and shed with clear errors.

Implementation cut

Publish connection budgets. Practice deploy drains. Prefer consistent hashing by room when sticky cookies are awkward for native clients.

Numbers

Max connections/instance with headroom. Reconnect spike after deploy < retry budget. Cross-instance fan-out latency SLO.

Pattern tags

• sticky-sessions
• consistent-hashing

Self-check

1. What does sticky routing buy for rooms?
2. Why do deploys cause reconnect storms?
3. How does consistent hashing help?
4. What resources dominate per connection?
5. When must room state leave the instance?

Walkthrough

Recordings will appear here when published. Until then, work the failure drill and lab locally — pause after each iteration and write the metric that moved.

Lab package: see /systems/labs/ws-fanout/.