Don't Panic — Connection Pooling and Concurrency at Scale

Opening a database connection is more work than it looks: a network handshake, usually an encryption negotiation, an authentication round trip, and some memory set aside on the database server. Do all of that once per request and you have spent most of your time getting ready to work rather than working. A connection pool keeps a small set of these connections open and lends them out. A request borrows one, runs a query or a transaction, hands it back, and the connection stays open for the next borrower.

That is the dull half. The interesting half is that a pool is really a bounded queue: it has a maximum size, and when every connection is lent out, the next request waits. If it waits too long, an acquisition timeout ends the wait with a clean error instead of an indefinite hang. So the pool quietly does two jobs at once. It avoids repeated setup cost, and it caps how much work reaches the database.

The part that surprises people is that the maximum is not a user count. It is a limit on concurrent database work, and you size it from the database outward: start with the server's connection limit, subtract what administrators, migrations, monitoring and replication already take, then divide the rest across every application process that can reach the database. Autoscaling makes this arithmetic sharp. A pool of twenty looks modest until fifty replicas each want their twenty, and now you have asked for a thousand connections the database was never going to grant.

The other surprise is where the pool can sit. Inside each application process is the usual choice, but then every replica owns a separate pool with a separate maximum. A shared proxy, such as PgBouncer or a managed database proxy, sits between many clients and the database and enforces one backend ceiling for the whole fleet. Proxies can also loosen the reuse boundary: transaction pooling returns the connection after each commit instead of holding it for a whole client session, which packs far more clients onto the same backends, as long as the code does not depend on session state like temporary tables or prepared statements.

When a pool is exhausted, resist the reflex to enlarge it. A growing queue is evidence, and it usually points at slow queries, long transactions, a leaked connection that was borrowed and never returned, or a database already at its limit. A bigger pool often just relocates the queue from the application into the database, where waiting work is harder to see and harder to shed.

What to read next: the Intro lays out the full request path, the pool's internal states, and its failure behavior. Slides is the quick conceptual map. The Cheatsheet holds the sizing formulas and a symptom-to-cause table for when something is genuinely on fire. Field Notes covers what teams get wrong in production, and the Quiz checks whether the vocabulary has stuck.

Don't Panic — Connection Pooling and Concurrency at Scale

Relevant careers