Connection pools mental models

Business constraint

An API service talks to Postgres and Redis. Traffic is bursty. Leadership wants "just raise max connections" whenever latency spikes.

Naive v0

Open a new connection per request. Or set pool_max to thousands "to be safe". Timeouts are infinite. Retry storms on errors.

Failure drill

Database hits max_connections. App threads block in pool.Acquire. Tail latency jumps while CPU looks idle. Retries multiply acquire pressure. A single slow query holds a connection and starves the fleet.

Evolution path

Iteration 1

Size pools from server capacity: (db_max_conn - admin_headroom) / app_replicas, not from app thread count. Prefer wait timeouts over unbounded queues.

Iteration 2

Separate pools for latency-sensitive vs batch work. Add circuit breakers that fail fast when wait time exceeds SLO, shedding load instead of queueing forever.

Iteration 3

Instrument acquire wait, active, idle, and checkout duration. Practice failure drills: kill idle connections, inject slow queries, watch backpressure signals propagate to HTTP 503s.

Implementation cut

Draw the pool as a bounded semaphore. Document acquire timeout = client budget − query budget. Reject work when the semaphore is exhausted — that is a feature.

Numbers

Acquire wait p99 should stay under 10–20% of request SLO. Idle connections should not exceed steady-state concurrency by more than ~2×.

Pattern tags

• backpressure
• sticky-sessions

Self-check

1. Why can raising pool_max make outages worse?
2. What is the relationship between replicas and db max_connections?
3. How do retries interact with pool exhaustion?
4. What signal should trip shedding?
5. Where does sticky session affinity fit (if at all)?

Walkthrough

Recordings will appear here when published. Until then, work the failure drill and lab locally — pause after each iteration and write the metric that moved.