Glossary

Storage performance

Storage performance describes how quickly a storage system completes work and how much work it completes per second, expressed through three linked measurements: latency, IOPS and throughput. Which of the three sets the limit depends on the workload placed on the system.

At enterprise scale the question moves from how fast one drive is to how a whole cluster behaves with many concurrent clients, during failures and as it grows.

The component that saturates first

Latency, IOPS and throughput are three views of one system. Data rate equals operation rate times size, so 100,000 operations of 4 KiB move about 410 MB/s while 2,000 operations of 1 MiB move about 2.1 GB/s, and a figure quoted at one size says little about the other. Concurrency joins them: a client issuing one request at a time and waiting 0.5 ms for each cannot pass 2,000 operations a second, whatever storage sits behind it.

Which component limits the result depends on the workload. Clients run out of threads, connections or protocol CPU in single-stream jobs. Networks cap large transfers and remote clients, so a host with one 25 Gb/s link reads no faster than about 3.1 GB/s. Node CPU limits small-object and erasure-coded writes through checksums, parity and encryption. The metadata service limits lookups and listings once objects number in the billions, and media limits cache-miss reads and sustained writes. The bottleneck moves whenever the workload does, so a platform tuned for one consumer can disappoint the next. Storage performance tuning covers the adjustments made once the limiting component is known.

Storage performance degradation as a cluster fills and ages

Storage performance degradation is a decline in the latency, IOPS or throughput a system delivers compared with its own earlier behaviour under the same workload. Large platforms seldom fail by stopping. They slow down: a backup that used to finish at 4 a.m. finishes at 7, a nightly ingest runs into the business day, an AI pipeline that kept its GPUs fed starts leaving them idle. Consumers frequently notice before dashboards do, and by then the remedy is hardware on a short lead time.

Several gradual causes compound. Flash slows as it fills, because garbage collection copies more still-valid data for each block it reclaims. Hard drives lose sequential throughput as data lands on inner tracks. Queueing delay climbs steeply as utilisation rises, so a platform can feel comfortable for a year and degrade within a quarter. Cache effectiveness erodes as datasets outgrow it: with 0.1 ms hits and 8 ms back-end reads, average read latency is about 0.26 ms at a 98 percent hit ratio and about 0.5 ms at 95 percent, so a three-point drop nearly doubles it. Metadata growth slows listings and small-object operations long before capacity runs out.

Degraded reads and rebuild traffic after a failure

A cluster with tens of thousands of drives loses some every week, so it spends a real share of its life rebuilding. After a failure, a protected system serves data in degraded mode, reconstructing missing pieces from replicas or parity at extra cost per read, then restores the lost redundancy while competing with applications for drives, CPU and network. A full 20 TB drive rebuilt at 100 MB/s takes 20 × 10¹² ÷ 10⁸ = 200,000 seconds, about 56 hours, and longer when throttled to protect foreground work. Losing a whole server puts every drive it held into reconstruction at once, and how widely the protection scheme spreads data decides how thinly that load lands on the survivors. A shorter window also shrinks the period of reduced protection that data durability depends on.

Sizing for a full, busy cluster with a node down

For a lean team running petabytes on behalf of many consumers, platform performance means behaviour at the worst ordinary moment: 80 percent full, busy, with a rebuild under way. Capacity plans built from figures on an empty, healthy cluster overstate what the same hardware delivers eighteen months later, and the gap arrives as a missed service level for whichever team is busiest that week.

The scaling model decides what growth costs. A scale-up array has a fixed controller pair, so added shelves bring capacity until the controllers saturate, after which more speed means another system and another silo. A scale-out cluster adds CPU, ports and media with every node, so aggregate throughput rises with node count when data and requests are spread evenly.

Tail latency is what applications feel. GPU pipelines and interactive services wait on the slowest request in a batch, and a p99 that doubles while the median holds points to queueing or background interference months before averages move. Scrubbing, replication, lifecycle transitions and mass deletes draw on the same drives as tenants, so on a multi-tenant platform one tenant's cleanup reaches everyone. Across sites, each synchronously protected write pays the inter-site round trip before any storage component is involved, a floor on write latency that no choice of media removes.

Scality RING and storage performance

When a drive fails, Scality RING writes data across the remaining drives in that server and rebuilds only data that was written, so reconstruction stays within the server. A failed 20 TB drive holding 6 TB needs 6 TB rebuilt, 30 percent of a full-drive rebuild, with the writes landing on several drives at once.

Scality describes the hyperscale RING topology with the statement that latency stays flat as the workload grows. Erasure coding schemes set per storage class let the balance between capacity overhead and rebuild work differ between datasets held on one RING.