LSM ground up

Business constraint

Write-heavy workload needs fast durable writes and acceptable read amplification tradeoffs.

Naive v0

B-Tree updates in place for every write under high churn. Or never compact LSM levels.

Failure drill

Write stalls on page splits. Or reads slow as levels pile up. Disk fills with obsolete keys.

Evolution path

Iteration 1

Buffer in memtable; flush to immutable segment (lab lsm-memtable).

Iteration 2

Compact segments to reclaim and keep read paths bounded.

Iteration 3

Tune flush size vs compaction debt; observe read/write/space amplification.

Implementation cut

Teach the control loop: writes are cheap until compaction debt calls due. Measure amplification trio.

Numbers

Flush latency. Compaction debt bytes. Read amp under query mix. Stall count.

Pattern tags

• lsm-compaction
• append-only-log

Self-check

1. What problem does the memtable solve?
2. Why immutable segments?
3. What does compaction reclaim?
4. What are the three amplifications?
5. When do write stalls appear?

Walkthrough

Recordings will appear here when published. Until then, work the failure drill and lab locally — pause after each iteration and write the metric that moved.

Lab package: see /systems/labs/lsm-memtable/.