Photo upload pipeline

Business constraint

Users upload photos. API VMs should not proxy gigabytes. Processing (EXIF strip, variants) is async.

Naive v0

multipart through the API to disk, then DB. Or client uploads without authz checks.

Failure drill

API saturates on bandwidth. Partial uploads corrupt state. Retries create duplicate objects. Processing runs twice with different outputs.

Evolution path

Iteration 1

Presign PUT to object store (lab mental model: mini-object-store). API records intent row first (outbox).

Iteration 2

Completion callback or poll marks object durable; worker claims processing with idempotency key = object id.

Iteration 3

Virus scan / EXIF strip as gated stages before public visibility.

Implementation cut

Never mark public before processing gates. Idempotent variant writes. Expire abandoned intents.

Numbers

API CPU during uploads near-flat. Duplicate variant rate ~0. Abandoned intent cleanup lag SLO.

Pattern tags

• signed-urls
• outbox-idempotency

Self-check

1. Why presign instead of proxy?
2. What does the intent row prevent?
3. How do you idempotent processing?
4. When is an object safe to show?
5. What expires abandoned uploads?

Walkthrough

Recordings will appear here when published. Until then, work the failure drill and lab locally — pause after each iteration and write the metric that moved.

Lab package: see /systems/labs/mini-object-store/.