Don't Panic — CubeFS

CubeFS is a distributed storage system for unstructured data. That is a fairly calm description for a machine that may be asked to behave like a mounted file system, an HDFS target, and an S3-compatible object service at the same time. The useful part is not that it can wear several hats. It is that those hats point at one logical volume.

The volume is the map, not the territory. A file client sees a file system. ObjectNode sees a bucket. Neither view means there is one heroic disk under the floorboards. CubeFS splits metadata from file data, then splits both across the cluster. MetaNodes keep the file-system records: inodes and directory entries. DataNodes store replicated file data. BlobStore stores erasure-coded data as shards on BlobNodes. Masters coordinate the topology without carrying every file byte, which is a good way to prevent the control plane from becoming a very expensive filing cabinet.

The first surprise is that compatible interfaces do not abolish semantics. ObjectNode, the S3-compatible gateway, maps a bucket to a volume and an object key to a path. An object named a/b can therefore occupy the place that another key, a/b/c, needs to be a directory. Several clients may read a file, but concurrent writers to that same file are not a safe default either. The storage system is not being difficult for sport; the path and cache boundaries are real.

Then comes the redundancy decision. Replication keeps complete copies and makes recovery direct, at the cost of full-copy capacity. Erasure coding uses data and parity shards and can reduce that capacity cost, but adds encoding, reconstruction, repair work, and a larger component set. The winning choice depends on file sizes, latency needs, failure recovery, and the operational skill available. A healthy empty cluster, sadly, has not yet been persuaded to demonstrate repair traffic under load.

CubeFS fits when several access paths must share scalable storage and the team can operate disks, nodes, networking, monitoring, upgrades, and recovery. It is a weak fit when one protocol or a managed service already meets the need with fewer moving parts.

Read the Course intro for the full architecture and fit decision. Use Slides to trace interfaces through volumes, partitions, and disks. Keep the Cheatsheet nearby when comparing the replica and BlobStore paths. The Reference tab then leads from the overview into official design, deployment, and operational material.

Don't Panic — CubeFS

Relevant careers