Glossary
Erasure coding
Erasure coding protects stored data by splitting it into pieces, calculating additional parity pieces from them, and spreading all the pieces across drives, servers or sites. When pieces are lost, the survivors are enough to rebuild them. It tolerates several simultaneous failures while using far less raw capacity than keeping full copies.
Erasure coding and replication compared
Both approaches protect against hardware loss. A layout of four data pieces and two parity pieces, written 4+2, survives the same number of failures as three full copies.
| Three full copies | 4+2 erasure coding | |
|---|---|---|
| What is stored | Three complete copies | Six pieces, each a quarter of the original size |
| Losses survived | Any two copies | Any two pieces |
| Raw capacity used | Three times the data | One and a half times the data |
| Work on write | Send the whole object to three places | Calculate parity, send six pieces |
| Work on rebuild | Copy from a surviving copy | Read several surviving pieces and recalculate the missing one |
At a few terabytes the capacity difference hardly matters. At petabyte scale it decides how many servers, how much power and how much floor space a system needs, which is the main reason most large object storage systems use erasure coding.
How a rebuild works
When a drive or server fails, the system identifies every object that had a piece there, reads enough surviving pieces of each from other servers, recalculates the missing pieces and writes them to healthy hardware. Applications keep reading and writing meanwhile, and a read that needs a missing piece reconstructs it on the fly. Because the work is spread across many servers, a large system can restore full protection quickly, which matters for both data durability and storage high availability.
Where each approach fits
Erasure coding suits large volumes of data written once and read many times, which describes most object storage. Replication suits very small objects, where splitting adds overhead for little saving, and data that must be served with the least possible work per read. Many systems combine the two: erasure coding within a site and replication between sites, so each site can rebuild locally. Spreading erasure-coded pieces across three or more sites is also possible, at the cost of more traffic between them.
What it does not protect against
Erasure coding answers hardware loss. A file that has been deleted or encrypted by ransomware is stored just as faithfully, across just as many drives, as any other, so it does not substitute for backup or for point-in-time copies.
Frequently asked questions
How is erasure coding different from RAID?
RAID protects drives inside one server or array. Erasure coding in distributed storage spreads pieces across servers, racks or sites, so it can survive the loss of whole servers and rebuilds using many machines at once.
Does erasure coding slow storage down?
Writes need parity calculated and pieces sent to several servers, and some reads gather pieces from several places. On current processors the calculation is rarely the bottleneck; network traffic between servers, especially during rebuilds, usually matters more.