Key-Value Databases
A key-value database stores data as pairs: a unique key and the value stored under it, with no fixed columns and no built-in way to query by anything but that key. It trades the flexibility of a relational database for speed and horizontal scale, and it runs the caches, session stores, and shopping carts behind many large-scale online systems.
itDatabases and data storage | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Intro
Key-Value Databases
A key-value database stores data as pairs: a key you look up by, and a value stored under it. There are no columns, no fixed schema, and — in the base model — no way to query by anything other than the key. You give it a key, it gives you back the value. That narrowness is the point: strip out everything a relational engine has to do for arbitrary queries, and what's left can be extremely fast and easy to spread across many machines.
Why the model exists
The clearest statement of the idea comes from Amazon's Dynamo, the internal system Amazon built to keep its shopping cart working even when disks were failing or a data center went dark. Dynamo exposes exactly two operations: get(key) and put(key, context, object). Values are opaque binary blobs, typically under 1 MB, with no relational schema and no multi-item transactions — only lookup by primary key. Amazon accepted that narrowness deliberately: a shopping cart that rejects a customer's "add to cart" click during a network blip is a lost sale, so Dynamo favors write availability over strict consistency, resolving conflicting versions later instead of blocking writes now.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
Supports
- Dynamo's get/put interface as the base key-value model
- Values as opaque objects with no relational schema or multi-item transactions
- Shopping cart availability motivation and write-availability-over-consistency tradeoff
- Consistent hashing, vector clocks, and quorum as underlying techniques
- Introduction origin story, glossary entries, slides, video script, and 06-links rationale
- https://redis.io/docs/latest/develop/data-types/
Supports
- Redis as a data structure server with strings, hashes, lists, sets, sorted sets, and streams
- Redis value-type explanation across intro, slides, cheatsheet, quiz, and video script
- https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/
Supports
- RDB snapshot and AOF append-only file persistence options and their tradeoffs
- Combinability of RDB and AOF, and default fsync-every-second durability bound
- Durability discussion and related quiz answer
- https://redis.io/docs/latest/operate/oss_and_stack/management/scaling/
Supports
- Redis Cluster's 16,384 hash slots and CRC16(key) mod 16384 slot assignment
- Downtime-free resharding by moving slot ranges between nodes
- Hash tags forcing related keys into the same slot for multi-key operations
- Partitioning explanation and related quiz answer
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html
Supports
- DynamoDB tables, items, and attributes vocabulary
- Simple (partition key) vs composite (partition key + sort key) primary keys
- Partition key hashing determining physical placement; sort key ordering within a partition
- Secondary indexes and DynamoDB Streams
- Core-components explanation and related quiz answer
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html
Supports
- Per-partition throughput ceilings independent of total table provisioned capacity
- Hot partition key risk from uneven access patterns
- Partition-key design guidance and related quiz answer
- https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html
Supports
- Eventually consistent (default) vs strongly consistent read semantics and cost difference
- Strongly consistent reads supported on tables and local secondary indexes, not global secondary indexes or streams
- Consistency-model explanation and related quiz answer
- https://etcd.io/docs/latest/learning/data_model/
Supports
- etcd as a multiversion key-value store for infrequently updated, highly reliable data
- Revisions, versions, and watch/point-in-time-read support
- Coordination-store role distinct from application data storage
- https://memcached.org/
Supports
- Memcached as a deliberately simple, free and open-source distributed in-memory object caching system
- Purpose of alleviating backing-database load
- Comparison with Redis in intro and quiz
- https://github.com/facebook/rocksdb/wiki/RocksDB-Overview
Supports
- RocksDB as an embeddable persistent key-value store using a log-structured merge (LSM) tree
- Memtable, write-ahead log, and SST file components, with background compaction
- Optimization for fast storage media such as flash
- https://rocksdb.org/
Supports
- RocksDB official project description as an embeddable persistent key-value store for fast storage
- 09-awesome-links entry and rationale
- https://www.symas.com/lmdb
Supports
- LMDB as an embedded key-value store using a memory-mapped B-tree
- ACID/MVCC semantics, zero-copy reads, and absence of background compaction or cleanup
- Comparison against RocksDB's LSM-tree design
- https://riak.com/products/riak-kv/index.html
Supports
- Riak KV's masterless, ring-distributed architecture with buckets of key-value pairs
- Availability-first guarantee (reads/writes succeed during hardware failures or partitions)
- Dotted version vector conflict resolution as an eventual-consistency mechanism
- https://aerospike.com/products/database/
Supports
- Aerospike as a multi-model database including a key-value access pattern
- Sub-millisecond response times and in-memory/flash/hybrid storage tiers
- 09-awesome-links entry and rationale
- https://github.com/numetriclabz/awesome-db
Supports
- Curated discovery of Memcached, RocksDB, LMDB, Riak, and Aerospike as ecosystem key-value projects
- Awesome Links selection