Erasure coding is usually described with two numbers, such as 8+3 or 12+4, and compared on a single property: how much raw capacity it consumes for each unit of usable data. That ratio matters, but it is the least interesting thing about a layout. The same overhead can produce very different results depending on how the fragments are spread across drives, servers, racks and sites, and on how much work the system has to do to repair a missing piece.
Choosing a layout is a balance between three things that pull against each other: storage efficiency, the number and kind of failures the data can survive, and the cost of recovery when something does fail. The general trade-off between erasure coding and replication is well covered. This post looks at the choice within erasure coding itself.
An erasure coding layout written as k+m splits each object, or each chunk of a large object, into k data fragments and computes m parity fragments. Any k of the k+m fragments are enough to reconstruct the original. The scheme therefore tolerates the loss of up to m fragments at once, and its raw-to-usable overhead is (k+m)/k.
Two layouts with the same m tolerate the same number of fragment losses, but they distribute risk differently. A wider stripe, with a larger k, uses capacity more efficiently, yet each fragment represents a smaller share of the object and a repair must read more fragments to rebuild one. A narrower stripe costs more capacity but repairs faster and places fewer demands on the network during recovery.
The figures below are arithmetic properties of each layout, independent of any product.
| Layout (k+m) | Raw-to-usable overhead | Usable share of raw | Fragment losses tolerated | Fragments read to repair one | Minimum fault domains for one fragment each |
|---|---|---|---|---|---|
| 4+2 | 1.50x | 67% | 2 | 4 | 6 |
| 6+3 | 1.50x | 67% | 3 | 6 | 9 |
| 8+3 | 1.375x | 73% | 3 | 8 | 11 |
| 10+4 | 1.40x | 71% | 4 | 10 | 14 |
| 12+4 | 1.33x | 75% | 4 | 12 | 16 |
| 16+4 | 1.25x | 80% | 4 | 16 | 20 |
Two rows stand out. A 4+2 layout and a 6+3 layout have identical overhead, yet 6+3 survives one more simultaneous loss at the cost of reading six fragments instead of four during repair. And 16+4 is the most efficient on paper, but it needs twenty separate fault domains before each fragment can sit on its own node, and every repair reads sixteen fragments across the network.
The "losses tolerated" column assumes each fragment lives on a different drive or server. Real clusters are rarely laid out that cleanly. If two fragments of the same stripe sit on one server, losing that server costs two fragments at once, and the effective tolerance drops.
The rule that follows is simple: to survive the loss of an entire fault domain, no fault domain may hold more than m fragments of any stripe. Applied at different levels it gives different minimum cluster shapes:
This is why a layout cannot be chosen in isolation from the physical design. A cluster that grows from three racks to five changes which layouts are safe, a theme that also comes up when planning object storage expansion.
When a drive fails, every stripe with a fragment on that drive is missing one piece. To rebuild a fragment, the system reads k surviving fragments, recomputes the missing one and writes it elsewhere. For a 4 TB fragment population on a failed drive under 12+4, the cluster reads roughly 48 TB to rebuild it. Under 4+2 it reads roughly 16 TB.
That read amplification is spread across many drives in a well-distributed system, so the wall-clock time can still be short, but it is real network and disk load that competes with client traffic. Wider stripes trade capacity savings for heavier repairs, and the trade becomes more visible as drive sizes grow. The factors that drive recovery time in practice are covered in the post on object storage rebuilds.
Erasure coding has a fixed cost per stripe: k+m fragments, each with its own metadata and placement. For large objects that cost disappears into the bulk of the data. For small objects it dominates. A 16 KB object split into sixteen data fragments of 1 KB each, plus four parity fragments, generates twenty separate writes and twenty metadata records to store a single small file.
Many systems therefore apply erasure coding only above a size threshold and use replication below it, or pack small objects together before coding them. When estimating usable capacity, the real overhead is the blend of both, weighted by the object size distribution, and it can be noticeably higher than the headline k+m ratio for workloads dominated by small files. Raw versus usable capacity should be calculated against that blend.
Scality RING supports both erasure coding and replication, selectable per policy, and data placement across nodes and sites is configured by the operator rather than fixed by the product. That makes the fault domain rules above something the operator sets deliberately: a layout can be matched to the number of servers, racks and sites actually deployed, and changed as the cluster grows.
RING distributes data across the nodes in the system, so reconstruction after a failure draws on many devices in parallel rather than funnelling through a single replacement. That property keeps repair load spread out, which matters more as stripe width and drive capacity increase. RING can also span multiple sites, which is where the site-level placement arithmetic becomes the deciding factor in layout choice.
A practical selection usually works backwards from the failure model rather than forwards from the overhead number:
The layout that results may not be the most efficient one on paper, but it will survive the failures the business actually cares about and recover from them at a cost the cluster can carry.