Storage fundamentals
Storage node
A storage node is a single server that takes part in a distributed storage system. It contributes its drives, processors, memory and network connections to the cluster, which combines many nodes into one logical pool of storage and spreads data across them.
What does a storage node consist of?
A node is usually a standard server chosen for drive density.
| Component | Role in the node |
|---|---|
| Data drives | Bulk capacity, usually hard drives or flash |
| Metadata and cache drives | Fast flash for indexes and small objects |
| Processors | Checksums, erasure coding, encryption, request handling |
| Memory | Caching and metadata operations |
| Network interfaces | Client traffic and traffic between nodes |
| Operating system and storage software | The services that make the server part of the cluster |
What roles do nodes play?
Designs vary. In some systems every node is identical and runs every service, which keeps the architecture simple and means no node is special. In others, roles are separated: data nodes hold object contents, metadata nodes hold the index, and access or gateway nodes terminate client protocols such as S3. Separated roles let each type be sized for its job, and identical nodes are easier to operate at small scale. Either way, the software presents the cluster to applications as one system.
What happens when a node is added or lost?
Adding a node increases capacity and performance together, which is the principle of scale-out storage. The cluster moves a share of existing data onto the new node in the background so that load stays even. Losing a node removes its share of every protection group. The cluster keeps serving requests from the remaining pieces and starts a data rebuild to restore full redundancy on the surviving nodes. The protection scheme and the number of nodes together decide how many can be lost at once.
How are nodes sized and replaced?
Node size is a trade-off. Large, dense nodes lower the cost per terabyte and concentrate more data behind each failure. Smaller nodes spread risk and need more rack space and network ports. A minimum number of nodes is set by the protection scheme. Because the software and hardware are separate, nodes of different generations can usually run in one cluster, and old nodes are retired by draining their data onto newer ones. Mixed hardware clusters and replacing aging nodes cover the operations.
Frequently asked questions
Is a storage node the same as a server?
Physically, yes. The term describes a server in its role as a member of a storage cluster.
How many nodes does a cluster need?
It depends on the protection scheme. A few nodes are enough to start, and more are needed for schemes that tolerate several simultaneous failures.
Can nodes from different vendors be mixed?
With software-defined storage, generally yes, within the software's list of supported hardware.
Related terms
- Scale-out storage: growth by adding nodes.
- Distributed storage: the system nodes form together.
- JBOD: drive enclosures attached to a node.
- Consistent hashing: one way data is assigned to nodes.
- Rack awareness: spreading data across the racks nodes sit in.