Glossary
Read/write performance
Read/write performance is the speed at which a storage system completes read operations and write operations, measured separately because the two follow different paths through the system. Reads usually finish faster, and the size of the gap depends on the media, the data protection scheme and the point at which the system acknowledges a write.
Why read/write performance matters at scale
A single headline speed figure hides the fact that most large platforms are limited in one direction. A backup landing zone or a telemetry store spends most of its life absorbing writes. A media archive or an AI training corpus spends most of its life serving reads. Sizing a platform on a blended figure leaves one of the two paths short, and at petabyte scale the shortfall turns into missed backup windows, idle GPUs or extra nodes bought late to compensate.
The direction also decides what data protection costs. Redundancy is generated when data is written, so every choice about replication or erasure coding shows up first as write overhead and as traffic between nodes.
How the read and write paths differ
A read resolves where the data lives, fetches one valid copy (or enough fragments to rebuild the requested bytes) and returns it. In a distributed system any healthy copy will do, so reads can be served by whichever node answers first and spread across the whole cluster.
A write has more stages. The system accepts the data, chooses where to place it, generates replicas or parity, persists every piece to non-volatile media, commits the metadata that makes the object visible and only then acknowledges the client. A read is complete when one copy arrives. A write is complete when the durability promise is met, which involves more devices, more nodes and more round trips.
The media adds its own asymmetry. Hard drives read and write with the same mechanical motion. NAND flash programs a page several times more slowly than it reads one and cannot overwrite in place, so sustained writes trigger garbage collection inside the drive, which also delays reads queued behind it. The link between write load and drive wear is covered under flash storage endurance.
Data protection and the write multiplier
Every protection scheme multiplies the bytes and device operations behind one logical write. A healthy read usually costs a single fetch whatever the scheme.
| Scheme | Bytes written per byte stored | Effect on the write path |
|---|---|---|
| Three-way replication | 3 | Three full copies sent across the network and persisted |
| Erasure coding, 8 data + 4 parity | 12 ÷ 8 = 1.5 | Parity computed, twelve fragments placed on twelve devices |
| Erasure coding, 9 data + 3 parity | 12 ÷ 9 ≈ 1.33 | Lower capacity overhead, same fan-out to coordinate |
| RAID 6, small random update | Read-modify-write of data and two parity blocks | Six device operations per logical write |
Erasure coding trades a lower capacity multiplier for parity computation and wider fan-out on every write. Small objects are the expensive case: an object smaller than the stripe still pays for coordinating a full set of fragments, so a workload of millions of small writes per hour stresses CPU and metadata long before it stresses disk bandwidth. Replication has a fixed multiplier equal to its copy count, simple to reason about and costly in raw capacity.
Read/write mix by workload
| Workload | Mix and pattern | What usually sets the limit |
|---|---|---|
| Backup ingest | Write-dominated, large sequential | Aggregate write bandwidth and protection overhead |
| Restore | Read-dominated, large sequential | Read bandwidth across many nodes at once |
| AI training data loading | Read-dominated, random across a large data set | Read concurrency and per-request latency |
| Model checkpointing | Bursts of large writes | Write bandwidth during the burst |
| Logs and telemetry | Write-dominated, small appends | Small-write overhead and metadata rate |
| Media archive | Written once, read occasionally | Capacity cost first, read latency second |
The mix shifts over a platform's life. A repository that took mostly writes during migration becomes read-heavy once analytics or AI teams start using it. The sequential vs random I/O entry covers the access-pattern side of the same question.
What read/write asymmetry means for large-scale storage
For an architect sizing a multi-petabyte platform, reads and writes amount to two separate capacity plans. Write throughput depends on how much protected data the cluster can place per second, which ties it to the protection scheme, the network between nodes and the CPU available for parity. Read throughput depends on how many nodes and drives serve requests in parallel, so it grows close to linearly as nodes are added and data is spread evenly.
That asymmetry runs through several decisions:
- The protection scheme is a write-performance decision as much as a capacity one. A wider erasure-coded stripe saves raw capacity across petabytes and adds fan-out to every PUT.
- A cluster serving both backup and AI sees its heaviest writes (nightly ingest) and heaviest reads (training epochs) at different hours, and contention appears when the two overlap.
- Small-object writes are bounded by metadata rate, so billions of small files call for a different design from a few million large objects at identical capacity.
- Restores are reads. A system tuned only for ingest can meet its targets every night and still miss its recovery time when a full restore arrives.
Read and write latency in Scality RING
Scality RING is software-defined object and file storage on standard x86 servers. Scality publishes measured latencies of 511 microseconds for a GET and 741 microseconds for a PUT. The PUT figure is about 1.45 times the GET (741 ÷ 511), a margin consistent with the extra placement, protection and metadata work on the write path.
RING defines erasure coding schemes per storage class, so data sets with different read/write profiles can carry different protection overheads inside one system.