Distributed Transactions

A local database transaction gives one system a clear boundary. All of its changes commit, or none of them do. That promise becomes harder when one business operation crosses databases, message brokers, or independently deployed services.

A distributed transaction coordinates changes across more than one transactional resource. The central question is not how to send several requests. It is how every participant reaches a safe final outcome when messages, processes, or machines fail.

Consider an order that reserves inventory and records a payment. A crash after only one change can leave the business state inconsistent. Retrying the whole request can create a second payment. Waiting forever can preserve safety while making the order unavailable.

Distributed transaction design is therefore a choice among guarantees, failure behavior, latency, coupling, and operational cost.

The core mental model

Separate the business operation from each local transaction.

The business operation describes the outcome you need, such as placing an order.
A local transaction commits changes inside one resource manager.
A coordination protocol connects those local outcomes.
Recovery logic decides what happens after partial progress or uncertain communication.

The network creates an ambiguity that local code cannot remove. After sending a commit request, a coordinator might lose the reply. The participant may have committed, or the request may never have arrived. A timeout reports uncertainty, not the transaction outcome.

That uncertainty drives the two main families of designs:

Atomic commit protocols delay the final outcome until participants agree on commit or abort.
Sagas commit a sequence of local transactions and use later actions to compensate for completed work when needed.

Neither family is universally better. They make different promises.

Two-phase commit

Two-phase commit, often shortened to 2PC, uses one coordinator and multiple participants.

During the prepare phase, the coordinator asks every participant whether it can commit. A participant that votes yes records enough durable state to honor that promise. It cannot then abort by itself.

During the decision phase, the coordinator chooses commit only if every participant voted yes. Otherwise, it chooses abort. The coordinator records the decision and sends it to every participant.

This protocol provides one atomic outcome across supported resources. A database and message broker can participate when their resource managers and transaction manager share a compatible interface.

The X/Open distributed transaction processing model names three important roles:

The application defines the transaction boundary.
The transaction manager coordinates completion and recovery.
Each resource manager controls a transactional resource, such as a database.

The XA interface standardizes communication between a transaction manager and resource managers. XA is an integration contract, while 2PC is the commit protocol used through that contract.

Why prepared transactions need care

A prepared participant has promised that it can commit later. It must retain transaction state and often keeps locks until it learns the decision.

This state makes recovery possible, but it also creates operational pressure. PostgreSQL warns that prepared transactions left open can retain locks and interfere with routine storage maintenance. Its documentation recommends an external transaction manager and short prepared intervals.

Ordinary 2PC can also block after a failure. If a participant has voted yes but cannot learn the coordinator's decision, it cannot safely choose by itself. Waiting protects atomicity, but reduces availability.

Gray and Lamport describe standard 2PC as a transaction commit protocol that lacks fault tolerance in the coordinator role. Their Paxos Commit protocol replaces the fragile decision path with consensus among replicated coordinators. That improves fault tolerance, but adds protocol and deployment cost.

Sagas and compensation

A saga divides a business operation into local transactions. Each local transaction commits independently. If a later step fails, the saga runs compensating transactions for earlier steps.

Compensation is a new business action, not a database rollback across services. Refunding a payment does not erase the original charge. It creates another durable fact. Other actors may observe the intermediate state before compensation finishes.

A saga can coordinate steps in two common ways:

Choreography lets participants react to events and trigger later steps.
Orchestration gives a coordinator explicit control over the sequence and recovery state.

Choreography can keep a short flow decentralized. As the number of steps grows, the overall path can become difficult to trace. Orchestration makes the path visible in one place, but the orchestrator becomes important infrastructure.

Compensation can fail too. Store saga progress durably, make repeatable steps idempotent, and support resumption. Some business effects cannot be reversed exactly. A shipped parcel may need a return process rather than an undo operation.

The dual-write problem

A common distributed transaction appears when one service updates its database and publishes an event.

Publishing first risks announcing a change that later rolls back. Committing first risks a crash before publication. Two independent writes cannot be made atomic through ordering alone.

The transactional outbox pattern writes the business change and an outgoing message to the same local database transaction. A separate relay later publishes the message. This removes the database and broker from one shared 2PC boundary.

The relay may publish a message more than once. Consumers therefore need idempotency, which means processing the same message again does not create another business effect. A stored message identifier is a common deduplication key.

Choosing a design

Start with the invariant, not the pattern name.

Use an atomic commit protocol when all participants support it and the operation truly needs one commit point. Confirm how prepared state, coordinator recovery, timeouts, and lock retention behave in production.

Use a saga when the work naturally separates into business steps and temporary inconsistency is acceptable. Define compensation before implementation. Identify irreversible pivot points and the recovery path after them.

Use a transactional outbox when one local transaction must reliably cause a message. It solves the database-and-message dual write. It does not, by itself, coordinate an entire multi-service business operation.

Sometimes the best boundary is smaller. Keeping strongly consistent data in one service or one database can remove the need for distributed coordination. A local transaction is easier to reason about, recover, and operate.

Operational questions

Before shipping a design, answer these questions:

What invariant must always hold?
Which intermediate states may users or other services observe?
How does each step respond to a retry?
Where is the final decision or saga progress stored durably?
How do operators find prepared, stuck, or compensating work?
What timeout triggers investigation, and what does that timeout actually prove?
Which actions are irreversible or require human resolution?
How will recovery behave during a network partition?

The hard part is not the happy path. It is making an uncertain outcome visible and recoverable.

Limits of this course

This course gives you a design and operations map. It does not specify one vendor's XA configuration, a production saga framework, or a formal proof of an atomic commit protocol.

Exact guarantees depend on each database, broker, driver, and transaction manager. Verify those contracts before treating a system as one atomic boundary.