Data protection & recovery
Rack awareness
Rack awareness is the ability of a distributed storage system to know which rack each server sits in and to use that knowledge when placing data, so that the copies or fragments of any one object are spread across different racks. A failure that takes out a whole rack then removes at most a tolerable share of each object.
Why does the rack matter?
Servers in the same rack usually share a power distribution unit and a top-of-rack network switch. When either fails, every server in the rack disappears at once. A storage system that protects only against individual drive or server failures might have placed all copies of an object in that one rack. Rack awareness treats the rack as a unit that can fail together and places data accordingly.
What is a failure domain?
A failure domain is any group of components that can fail from a single cause. Storage systems model a hierarchy of them.
| Failure domain | Shared dependency | Typical failure |
|---|---|---|
| Drive | The device itself | Media or electronics fault |
| Server | Motherboard, power supply, controller | Hardware fault, reboot |
| Rack | Power feed, top-of-rack switch | Power or network loss |
| Room or hall | Cooling, fire suppression zone | Environmental event |
| Site | Building, utility connection | Fire, flood, regional outage |
How does it interact with the protection scheme?
The protection scheme sets how many pieces of an object can be lost. Rack awareness sets how many pieces a single rack may hold. With three replicas, each goes to a different rack. With erasure coding, the fragments are spread so that no rack holds more than the scheme can lose. That in turn dictates a minimum number of racks: a scheme that tolerates two lost fragments needs enough racks that each holds no more than two. Where there are too few racks, the system has to fall back to server-level separation.
What makes a rack-aware layout hold in practice?
The system knows only the topology it has been told. Racks declared as separate while sharing one power feed, servers moved without updating the map and uneven racks that fill at different rates all undermine the model. Keeping the declared hierarchy in step with the building, checking fill levels per rack and recalculating what the loss of the largest rack would mean are the routine checks. The rules themselves are a data placement policy. Data placement across racks and sites covers the practice.
Frequently asked questions
Is rack awareness automatic?
The placement is automatic once the topology is defined. Someone has to describe which server is in which rack and keep that description current.
How many racks are needed?
At least as many as the protection scheme requires to keep each rack's share within what can be lost. Three is a common minimum.
Does rack awareness affect performance?
It can add network traffic between racks during writes and rebuilds, which the data center network has to carry.
Related terms
- Data placement policy: the rules that implement rack awareness.
- Erasure coding: the scheme whose fragments are spread across racks.
- Data durability: what correct placement preserves.
- Storage node: the unit placed within a rack.
- Stretched cluster: the same principle applied across sites.