Distributed Storage
Distributed storage spreads data across multiple machines so that no single node holds all the data or represents a single point of failure. It addresses replication, partitioning, consistency models, and fault tolerance to provide durable, scalable data persistence.
itStorage, backup, and data protection | OpenSkills.info
Intro
Distributed Storage
Distributed storage keeps data across multiple machines while presenting a useful storage service to clients. You may see a file system, an object API, a key-value interface, or a database. Behind that interface, the system must place data, coordinate updates, detect failures, and restore protection.
The goal is not to make failure disappear. The goal is to define which failures the service can tolerate and what clients observe while those failures happen.
Why one machine stops being enough
A single machine has finite capacity, throughput, and fault tolerance. Adding machines can increase capacity and parallelism. It also creates network delays, partial failures, and copies that can temporarily disagree.
Distributed storage is useful when you need one or more of these outcomes:
- More data than one machine can hold
- More read or write traffic than one machine can serve
- Continued service after a disk, host, rack, or site fails
- Data access from many clients or locations
- Independent scaling of storage and compute
These outcomes are not automatic. They depend on the data model, placement policy, protection scheme, consistency model, and operating discipline.
The five jobs every system must do
First, the system divides data into manageable units. A file system may use blocks or chunks. An object store uses objects. A key-value store uses records or key ranges.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://research.google/pubs/the-google-file-system/
Supports
- Distributed file-system scaling across commodity machines
- Chunking, metadata coordination, replication, fault tolerance, and recovery
- Workload-driven distributed-storage design and measured production behavior
- https://research.google/pubs/bigtable-a-distributed-storage-system-for-structured-data/
Supports
- Distributed storage for structured data across large machine fleets
- Data-model and data-layout choices for workloads with different latency and size demands
- Partitioning and workload distribution in a production storage system
- https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
Supports
- Highly available key-value storage design
- Consistent hashing, replication, configurable quorums, versioning, and conflict resolution
- Explicit tradeoffs among availability, consistency, durability, performance, and cost
- https://raft.github.io/index.html
Supports
- Consensus for replicated state machines
- Majority behavior and progress limits when a majority cannot communicate
- Replicated logs that preserve one command order
- https://raft.github.io/raft.pdf
Supports
- Leader election, log replication, safety, and membership concepts
- Majority-based replicated state-machine behavior
- Safety and availability distinctions during server failure
- https://research.google/pubs/spanner-truetime-and-the-cap-theorem/
Supports
- CAP as a partition-time tradeoff rather than a timeless product taxonomy
- Strong consistency, availability, and wide-area partition behavior
- The relationship between Spanner's guarantees and CAP
- https://docs.ceph.com/en/latest/architecture/
Supports
- RADOS, monitor, object storage daemon, client, and CRUSH responsibilities
- Direct client access to object storage daemons after calculated placement
- Replication, recovery, rebalancing, and cluster-map concepts
- Object, block, and file services over one distributed storage foundation
- https://docs.ceph.com/en/latest/rados/operations/crush-map/
Supports
- Calculated data placement without a central lookup table
- Topology-aware replica and erasure-chunk placement
- Host and other failure-domain rules
- https://docs.ceph.com/en/latest/rados/operations/erasure-code-profile/
Supports
- K data chunks and M coding chunks
- Reconstruction limits and capacity overhead
- Stripe size, plugins, and failure-domain configuration
- https://docs.ceph.com/en/latest/rados/operations/pools/
Supports
- Replicated and erasure-coded protection strategies
- Capacity, performance, workload, and failure-domain tradeoffs
- Pool-level data-protection configuration
- https://hadoop.apache.org/docs/r3.2.4/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html
Supports
- NameNode metadata and DataNode block-storage responsibilities
- Direct client data transfer after block-location lookup
- Configurable block replication, heartbeats, block reports, and rack-aware placement
- https://research.google/pubs/availability-in-globally-distributed-storage-systems/
Supports
- Hardware, software, network, and power failures in large storage systems
- Effects of data placement and replication strategies on availability
- Recovery and correlated-failure considerations at cloud-storage scale