Don't Panic: Microservices Data Patterns

Microservices data patterns are what happen after each service gets to mind its own database. This is healthy for autonomy and mildly inconvenient for anyone who hoped a cross-service operation was still one giant transaction wearing a fake moustache.

The first important idea is owned data. A service controls its schema and access to the information for its business capability. Other services ask through APIs or react to events. That stops a convenient join from becoming a permanent, invisible promise between teams. It also means a business operation, such as accepting an order, now travels through several local decisions.

That journey is a saga: a sequence of local transactions. Each step advances the work with an event or command. If a later step fails, earlier steps use compensating actions with business meaning. Releasing a reservation is not a database rollback. It is a new action, which is why it needs an owner, a state, and sometimes a human when the world refuses to cooperate.

The second surprise is that publishing an event is part of the transaction problem. A service can commit its change and then fall over before it tells anyone. The transactional outbox writes the change and the event record together, then a publisher sends the record later. The publisher may send it twice, because computers have an unhelpful talent for failing between two bits of bookkeeping. Consumers therefore need idempotent effects.

Reading is its own choice. API composition asks several services at query time and combines their answers. CQRS keeps a materialized read model from events. The first favors current source responses with runtime fan-out. The second favors repeated combined reads, while accepting lag and the need to repair or rebuild a projection.

Read the Introduction for the full data paths, Slides for the compact map, and Cheatsheet when choosing a transaction, publication, or read pattern. Then use the practice worksheet and exercise to make the workflow, failure path, and visible state explicit. That is the whole trick, apart from all the details that turn out to be the trick.