Dictionary without a DB

Business constraint

You need a durable string→string dictionary with fast gets, simple ops, and crash recovery — without standing up a full database.

Naive v0

Rewrite a whole JSON file on each put. Or keep only an in-memory map.

Failure drill

Crash loses data or corrupts JSON mid-write. Large files make puts O(n). Restarts reload slowly without structure.

Evolution path

Iteration 1

Append records to a log; keep in-memory offset index (lab bitcask-dict). Get = index lookup + positional read.

Iteration 2

Hint files / sparse index compaction to reclaim space from overwritten keys.

Iteration 3

CRC records; discard torn tails on open. Document single-writer assumption.

Implementation cut

Single writer process. Prove crash recovery with tests that truncate mid-record. Compaction as a planned maintenance path.

Numbers

Get p99 from SSD. Restart load time vs key count. Compaction reclaim ratio.

Pattern tags

• append-only-log
• lsm-compaction

Self-check

1. What does the sparse index store?
2. How do you detect torn writes?
3. Why single-writer?
4. What does compaction do?
5. When do you outgrow this design?

Walkthrough

Recordings will appear here when published. Until then, work the failure drill and lab locally — pause after each iteration and write the metric that moved.

Lab package: see /systems/labs/bitcask-dict/.