Data protection & recovery
Data placement policy
A data placement policy is the set of rules that determines where a distributed storage system puts each copy or fragment of an object: on which drives, servers, racks and sites. It turns requirements about resilience, performance, cost and location into concrete placement decisions that the system applies automatically.
What does a placement policy express?
A policy combines several kinds of constraint.
| Constraint | Example rule | Purpose |
|---|---|---|
| Protection scheme | Three replicas, or erasure coding with a given layout | How many failures can be tolerated |
| Failure domain separation | No two fragments in the same rack | Survive rack or site loss |
| Location | All copies within one country | Meet data localization rules |
| Media type | Metadata on flash, data on hard drives | Match performance to need |
| Balance | Fill nodes in proportion to capacity | Avoid hot spots and early exhaustion |
| Affinity | Keep one copy near the application | Reduce read latency |
How is a policy applied?
The administrator describes the physical topology to the system: which drives are in which servers, racks and sites. Policies are then defined against that hierarchy and attached to a scope, typically a bucket, a tenant or a storage class. When an object is written, the placement algorithm, often based on consistent hashing with topology rules, chooses locations that satisfy the policy. Different data sets on the same cluster can follow different policies.
How does placement interact with protection?
The protection scheme and the placement rules have to be considered together. Erasure coding that tolerates four lost fragments gives little benefit if six fragments sit in one rack. Placement decides which real-world failures the scheme's arithmetic covers. It also sets requirements on the estate: separating fragments across racks needs enough racks, and across sites needs enough sites with adequate links. Rack awareness is the commonest form of this.
What happens when a policy changes?
Changing a policy, or the topology beneath it, means existing data may no longer comply. The system has to move data until it does, a background task similar to a rebalance, which uses disk and network capacity and takes time in proportion to the data affected. A sound practice is to model the change first: what each failure scenario means before and after, how much data moves and how long the cluster will be in between. Data placement across racks and sites and erasure coding layouts work through examples.
Frequently asked questions
Who defines the data placement policy?
The storage operator, based on resilience, compliance and performance requirements agreed with data owners.
Can different buckets have different placement policies?
In most distributed object stores, yes. Policies are usually selectable per bucket, tenant or storage class.
Does a placement policy guarantee compliance with location rules?
It enforces where the storage system puts data. Compliance also depends on replicas, backups and metadata handled outside that system.
Related terms
- Rack awareness: separating data across racks.
- Erasure coding: the scheme whose fragments are placed.
- Data localization: location constraints a policy can enforce.
- Consistent hashing: a common placement mechanism.
- Storage class: a user-facing name for a placement and protection choice.