System Design Fundamentals

System design is the work of turning a problem into a technical plan that can be built, operated, and changed. You identify requirements, shape the major components, trace data and requests, and make trade-offs explicit.

A good system design is not the diagram with the most boxes. It is a defensible answer to a specific workload. The same feature can need a different design when its traffic, latency, durability, security, cost, or team constraints change.

The mental model: requirements become flows and evidence

Use this sequence:

requirements -> estimates -> interfaces -> data -> components -> failure controls -> evidence

Requirements tell you what success means. Estimates expose likely pressure points. Interfaces define the system boundary. Data and flows reveal where work and state move. Components implement those flows. Failure controls keep one problem from spreading. Evidence tests whether your assumptions hold.

Keep the sequence iterative. A data choice can change an interface. A failure scenario can change the component layout. A load test can disprove a capacity estimate.

Start with requirements

Separate functional requirements from quality requirements.

Functional requirements describe user-visible outcomes. Examples include creating an order, reading a timeline, or receiving a notification.

Quality requirements describe how well the system must produce those outcomes. Common concerns include latency, throughput, availability, durability, consistency, security, cost, and operability.

Replace vague words with measurable statements. "Fast" gives you no design test. A target such as "the chosen percentile of successful reads stays below the agreed latency under the expected peak load" tells you what to measure. The actual numbers must come from stakeholders and workload evidence.

Prioritize the requirements. You cannot maximize every quality at once. More redundancy can improve availability while increasing cost. Synchronous coordination can simplify a consistency rule while increasing latency and reducing availability during a dependency failure.

Estimate before choosing components

Back-of-the-envelope estimates help you find the part of the design that deserves attention. They are directional, not promises.

Estimate:

requests per second at average and peak load;
read-to-write ratio;
payload size and network transfer;
stored data per day and over the retention period;
concurrent work;
acceptable queue depth or processing delay.

Write every assumption beside the estimate. If traffic has bursts, an average rate hides the peak. If latency has a long tail, an average hides slow requests. Google SRE recommends user-relevant service indicators and percentile views for latency because distributions reveal behavior that averages can conceal.

Draw the smallest complete flow

Begin with one end-to-end path:

client -> entry point -> application service -> state -> response

Add a component only when it has a responsibility. Typical responsibilities include:

routing requests;
applying business rules;
owning authoritative data;
caching reusable results;
buffering deferred work;
distributing events;
storing large objects;
observing system behavior.

Label arrows with the interaction type. A synchronous request keeps the caller waiting. A queue accepts work for later processing. An event reports a fact that consumers may handle independently.

Every remote interaction needs a timeout and an error policy. Retries must be limited. Otherwise, retries can amplify overload. Operations that may repeat need idempotent handling so a duplicate attempt does not repeat an unintended effect.

Put data ownership at the center

For each important fact, identify:

its authoritative owner;
the write path that enforces its rules;
the read paths and freshness needs;
the retention and recovery needs;
the way other components receive it.

A relational database is often a sensible starting point when relationships and transactions matter. Other storage models fit different access patterns. Choose from reads, writes, query shapes, consistency needs, and operational constraints. Do not select a database from a popularity list.

Replication keeps copies of data. It can improve read capacity and fault tolerance, but replicas create freshness and failover questions. Partitioning divides data across resources. It can increase capacity, but the partition key must distribute work and support the main access patterns.

A cache trades freshness and invalidation work for fewer repeated reads and lower latency. Define what can be cached, the key, the lifetime, and the behavior after a miss. HTTP specifications define caching semantics for HTTP responses, but application caches still need an explicit freshness policy.

Scale the bottleneck, not the diagram

Horizontal scaling adds instances. It works best when any instance can handle a request and shared state lives behind an explicit data service. Session affinity and in-memory session state can restrict how requests spread across instances.

A load balancer distributes eligible requests. It does not remove a database bottleneck or fix an expensive query. Measure the constrained resource before adding capacity.

Queues separate request acceptance from background processing. They can absorb a burst and let consumers work at a controlled rate. They also add delay, duplicate-delivery concerns, backlog monitoring, and failure recovery.

Apply backpressure when a consumer cannot keep up. Bound queues, limit incoming work, or degrade less important features. An unlimited queue turns overload into a delayed outage.

Design for partial failure

Distributed components communicate across networks, so latency and loss must be part of the design. AWS reliability guidance recommends timeouts, controlled retries, throttling, limited queues, graceful degradation, and stateless components where practical.

Use:

timeouts to stop waiting when a dependency exceeds its budget;
limited retries for transient failures;
exponential backoff and jitter to spread retry attempts;
circuit breakers to stop repeated calls to a failing dependency;
bulkheads to isolate resources so one workload cannot consume all capacity;
graceful degradation to preserve core behavior when an optional dependency fails;
redundancy to avoid a single instance becoming the only path to service.

Each mechanism has a cost. A retry increases load and latency. A replica costs money and can serve stale data. A fallback can return less complete results. State the trade-off.

Make consistency a product decision

Strong consistency makes a completed write visible under the system's stated consistency model before dependent work proceeds. Eventual consistency permits replicas or derived views to converge later.

Choose per invariant and user flow. A payment decision and a recommendation count do not necessarily need the same consistency. If a user can observe stale data, define how stale it may be and how the interface communicates pending work.

Asynchronous delivery can repeat a message. Consumers should record or recognize processed operations when duplicate effects would be harmful. Avoid claiming "exactly once" without defining the boundary and mechanism that support it.

Validate the design

A design is a set of hypotheses. Test the risky ones.

Load tests check throughput, latency percentiles, and saturation.
Failure tests check timeouts, retries, recovery, and degraded behavior.
Data tests check invariants, duplicate handling, and recovery.
Observability checks confirm that metrics, logs, and traces expose the important path.
Cost estimates check that the design remains affordable at expected load.

Define service level indicators around user-relevant behavior. Google SRE identifies latency, error rate, throughput, availability, and durability as common indicators. A service level objective sets a target for an indicator. It gives you a way to decide whether the system is healthy enough for its purpose.

A practical design conversation

When you present a design, lead with the requirements and the main flow. Then explain data ownership, capacity assumptions, failure behavior, and trade-offs. End with unknowns and the evidence needed next.

System design does not end at approval. Production traffic, incidents, and changing requirements provide new evidence. Update the design when an assumption stops being true.