CubeFS
CubeFS is a cloud-native distributed storage system that provides both file and object storage interfaces from a single cluster. It is designed for containerized workloads requiring scalable, high-performance shared storage with strong consistency.
itCloud native tools and technologies | OpenSkills.info
Intro
CubeFS
CubeFS is a distributed storage system for unstructured data. It presents the same storage platform through file and object interfaces, including POSIX, HDFS, and S3-compatible access.
The central mental model is one logical volume, split metadata and data services, several access paths.
applications
|-- POSIX through a CubeFS client
|-- HDFS-compatible access
`-- S3-compatible access through ObjectNode
|
volume
+---------+---------+
| |
metadata partitions data partitions or blobs
on MetaNodes on DataNodes or BlobNodes
\ /
`---- Masters ----'
coordinate
CubeFS exists for teams that need shared storage across many machines and applications. It separates file metadata from file data, partitions both, and distributes those partitions across a cluster. You can expand the relevant subsystem as demand grows.
This is infrastructure, not a hosted storage account. Your team deploys the components, supplies disks and network capacity, monitors them, and plans failure recovery.
What a volume means
A volume is the unit users work with. Through a file client, it looks like a file system that can be mounted by multiple clients. Through the object gateway, the same volume corresponds to a bucket.
That shared backing does not make POSIX and object semantics identical. The object gateway converts bucket and key operations into volume and path operations. A key such as a/b.txt maps to a file path. Some object names that are legal in a pure key-value namespace can conflict after this conversion. Treat cross-protocol use as a design decision and test the exact naming and update patterns you need.
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.cubefs.io/docs/master/overview/introduction.html
Supports
- CubeFS scope, access protocols, storage engines, and application scenarios
- Stated POSIX compatibility boundary and multi-protocol positioning
- Kubernetes CSI integration and caching feature overview
- https://www.cubefs.io/docs/master/overview/architecture.html
Supports
- Master, metadata, data, object, and volume architecture
- Replica and erasure-code subsystems coexisting or operating independently
- Volume presentation as a file-system instance or object bucket
- Raft, RocksDB, Multi-Raft, partition, and horizontal-scaling roles
- https://cubefs.io/docs/master/design/master.html
Supports
- Master responsibilities for node health, partitions, and volumes
- Raft replication and RocksDB persistence for Master metadata
- Resource-based placement of metadata and data partitions
- https://cubefs.io/docs/master/design/metanode.html
Supports
- Inode and dentry B-Tree structures in metadata partitions
- Inode-range partitioning, Multi-Raft replication, splitting, and recovery
- https://cubefs.io/docs/master/design/datanode.html
Supports
- DataNode, data partition, extent, replica, and storage-topology concepts
- Separate handling of large and small files
- Replica consistency and fault-recovery responsibilities
- https://cubefs.io/docs/master/design/blobstore.html
Supports
- BlobStore component roles and erasure-code architecture
- Volume, chunk, blob, and shard definitions
- Encoding, read, repair, degraded-read, and storage-engine tradeoffs
- https://www.cubefs.io/docs/master/design/objectnode.html
Supports
- Stateless ObjectNode architecture and S3-compatible access
- Bucket-to-volume and key-to-path semantic conversion
- Cross-protocol object-name conflict and atomic object-write design
- https://www.cubefs.io/docs/master/design/client.html
Supports
- FUSE client architecture and topology or metadata caching
- Concurrent reads and the restriction on concurrent same-file writes from different mounts
- Replica and erasure-coded client data paths
- https://cubefs.io/docs/master/user-guide/file.html
Supports
- FUSE dependencies, client configuration, mounting, and unmounting
- Cache consistency warning for changing datasets
- https://www.cubefs.io/docs/master/quickstart/single-deploy.html
Supports
- Roles of cfs-server, cfs-client, and cfs-cli
- Standalone mount and object-access orientation
- Single-copy nature of standalone data and metadata
- https://www.cubefs.io/docs/master/deploy/k8s.html
Supports
- Kubernetes deployment roles for Master, MetaNode, DataNode, and ObjectNode
- StatefulSet, DaemonSet, Deployment, host-network, and host-path design
- https://www.cubefs.io/docs/master/user-guide/cli/overview.html
Supports
- CLI management of cluster, nodes, partitions, volumes, users, quotas, and configuration
- https://github.com/cubefs/cubefs
Supports
- Upstream source, releases, issues, and contribution materials