openskills.info
Course catalog
Course Preview

Database Partitioning and Sharding Strategies

Database partitioning divides one logical data set into smaller physical pieces. Sharding places those pieces across database servers so storage and request load can grow beyond one machine.

itDatabases and data storage

Don't Panic: Database Partitioning and Sharding

Database partitioning divides one logical data set into smaller physical pieces. Sharding places horizontal pieces across database servers. Partitioning gives different data to different units. Replication keeps copies of the same unit. Mixing those up is a fine way to buy several servers and still be surprised.

The request path is route, execute, and combine. A router reads a key or predicate and maps it to partition bounds, hash buckets, or a directory entry. One target is pleasant. Several targets are possible. All targets, followed by a merge, is scatter-gather, which is where latency and coordination arrive with luggage.

Partition methods are tradeoffs. Range partitioning keeps order and helps retention, but a growing timestamp can heat the newest range. Hash partitioning spreads point access, but range scans fan out. List partitioning supports explicit placement, but every new value needs a destination. Directory routing supports custom movement, while making the directory part of the critical path.

The partition key is the decision. It needs enough distinct values, no dominant values, alignment with common predicates, and locality for rows used together. One busy tenant can defeat a neat hash. Adding buckets can spread that tenant, but whole-tenant reads then visit multiple buckets. The system is not being difficult; it is charging you for the property you asked for.

Resharding is a migration, not a wish. Create targets, copy data, stream new changes, verify counts and boundaries, switch routing once, and keep a rollback window. Backups need a cluster-wide consistency point when invariants cross shards.

Use the Practice Reference to score keys and plan resharding. Do the Exercise to design tenant event placement. The Cheatsheet keeps routing, strategy, architecture, failure, and monitoring checks together. Shard when measured write, storage, working-set, or failure-domain limits require it, not because the diagram looks more grown-up.

Where this skill leads

Relevant careers

See how this topic contributes to broader role-level skill maps.

Sources