Distributed Systems Fundamentals
Distributed systems are collections of independent computers that appear to users as a single coherent system. This topic covers the fundamental challenges they face: network partitions, partial failures, clock synchronization, consistency models, and the impossibility results that constrain their design.
itDistributed systems, messaging, and integration | OpenSkills.info
Intro
Distributed Systems Fundamentals
A distributed system is a group of independent computers that cooperate through messages. To a user, the group often looks like one service. Inside, work and state cross machine and network boundaries.
That boundary changes the engineering problem. A function call either returns or raises an error in one process. A network request can succeed, fail, arrive late, arrive twice, or complete after the caller stops waiting. The caller cannot always tell which case occurred.
Distributed systems exist because one machine is not always enough. You may need more capacity, lower latency near users, continued service during a machine failure, or data in several locations. Distribution can provide those properties. It also introduces partial failure, concurrency, and uncertainty about time.
Start with messages and state
Each node has local state and observes its own events. Nodes communicate by sending messages. There is no instant, free view of the whole system.
This gives you the first useful mental model:
local state + messages + failures + time assumptions = system behavior
Before choosing a database or consensus algorithm, write down those assumptions. Ask which nodes can fail. Ask whether messages can be delayed, duplicated, reordered, or lost. Ask what the system promises while a network link is broken.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://lamport.azurewebsites.net/pubs/time-clocks.pdf
Supports
- Distributed processes communicating through messages
- Happened-before as a partial order defined by process order, message send and receive, and transitivity
- Concurrent events and logical clocks
- Ordered commands as a basis for distributed state machines
- https://groups.csail.mit.edu/tds/papers/Lynch/jacm85.pdf
Supports
- Fully asynchronous process and message model
- Crash-failure assumptions and admissible executions
- Impossibility of guaranteed deterministic consensus termination with one possible faulty process
- Separation between consensus safety conditions and termination
- https://lamport.azurewebsites.net/pubs/byz.pdf
Supports
- Byzantine failures as malfunctioning components that can send conflicting information
- Agreement requirements in the presence of arbitrary faulty behavior
- https://groups.csail.mit.edu/tds/papers/Gilbert/Brewer2.pdf
Supports
- CAP definitions of atomic consistency, availability, and partition tolerance
- Impossibility of guaranteeing both atomic consistency and availability during a partition
- Practical limits of the permanent pick-two interpretation
- https://raft.github.io/raft.pdf
Supports
- Consensus for replicated logs and replicated state machines
- Leader election, log replication, terms, commitment, and safety
- Majority quorum overlap and progress requirements
- Behavior of majority and minority sides during a partition
- https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
Supports
- Partitioning with consistent hashing and replicated key-value data
- Tunable replica, read, and write thresholds
- Eventual consistency, concurrent versions, conflict resolution, hinted handoff, and anti-entropy repair
- Availability, durability, and latency tradeoffs in a production storage system
- https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
Supports
- Timeouts as a bound on client waiting rather than proof that side effects did not occur
- Retries as a response to partial and transient failures
- Idempotent APIs for safely retrying operations with side effects
- High-percentile downstream latency as an input to timeout selection
- https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/
Supports
- Stable client request identifiers for duplicate recognition
- Semantic equivalence across retry responses
- Atomic recording of an idempotency token and the associated mutation
- Parameter validation when one request identifier is reused for different intent
- https://research.google/pubs/spanner-googles-globally-distributed-database-2/
Supports
- Synchronous replication at global scale
- Externally consistent distributed transactions
- Explicit clock uncertainty through the TrueTime API
- Partitioned data managed through consensus-backed replication
- https://research.google/pubs/the-chubby-lock-service-for-loosely-coupled-distributed-systems/
Supports
- Coarse-grained distributed locking and reliable low-volume storage
- Availability and reliability as design goals for a production coordination service
- Operational lessons from large client populations
- https://research.google/pubs/mapreduce-simplified-data-processing-on-large-clusters/
Supports
- Parallel data processing across clusters
- Runtime-managed partitioning, scheduling, communication, and machine-failure handling
- Distribution hidden behind a narrow map and reduce programming model
