Servers bought five years ago still hold data, still appear in warranty renewals, and are still the nodes most likely to fail next. Replacing them is not primarily a purchasing decision. It is a data movement exercise on a cluster that keeps serving requests throughout, where every node retired is one whose data must already exist elsewhere, and where the order of operations decides whether durability dips.
The obvious plan is to buy a new cluster, copy everything across and switch over. That is clean on a diagram and expensive in practice, since it funds two clusters at once, concentrates risk into one cutover, and guarantees a repeat in five years. Replacing hardware continuously inside one cluster spreads cost and risk, but makes retirement an ongoing process rather than a project.
The part most often underestimated is that retiring a node is the same class of event as losing one. The cluster reconstructs or relocates that node's share of the data while still tolerating an unrelated failure. A refresh is a durability exercise wearing a procurement label.
Draining a node versus letting it fail
Draining copies data off a node before it leaves the cluster, so the protection level holds throughout and the operation can be paused. Letting a node fail, by removing it and letting self-healing reconstruct its data from surviving fragments, is faster to start and leaves the cluster degraded for the length of the rebuild.
The difference matters most in erasure-coded systems. A drain reads existing fragments and writes copies; a reconstruction reads from every remaining node and recomputes what was lost. Reconstruction costs more in network and CPU, and is the state where a second failure has least margin. For a planned retirement there is rarely reason to choose it, and what governs its duration also governs any unplanned recovery.
The counterargument for drains is duration. Draining a dense node moves a large amount of data at a throttled rate and can take days, during which the cluster carries extra load. Draining several at once shortens the calendar but reduces several failure domains at the same time, a trade to make explicitly rather than by default.
Why a refresh is a durability event
Every hour a cluster spends moving data is an hour with less margin than at rest. During a drain the data exists in both places, which is safe. During a reconstruction it exists in fewer places than the policy specifies, which is not.
Drive failure rates are not uniform across a node's life, and aging hardware fails in correlated ways because it was bought, racked and powered on together. A batch of drives from one manufacturing run reaching end of life is not a set of independent events, and independence is the assumption protection schemes rest on. The chance of a second failure during a rebuild is therefore highest exactly when the oldest hardware is being retired.
The protection policy in force should therefore be judged against the failure rate of the hardware being retired rather than the cluster average. If a scheme tolerates two concurrent failures and retiring one node consumes one, the margin for that cohort is thinner than the policy suggests, and how replication and erasure coding behave under that condition is covered in the comparison of the two approaches.
| Retirement method | What the cluster does | Exposure during the operation |
|---|---|---|
| Planned drain, one node at a time | Copies data off before removal, protection maintained | Lowest risk, longest duration, days of background load per node |
| Planned drain, several nodes at once | Parallel movement, shorter calendar | Several failure domains reduced at once, thinner margin |
| Remove node and let self-healing rebuild | Treats retirement as a failure, reconstructs from surviving fragments | Degraded until rebuild completes, loads every remaining node |
| Drive-by-drive replacement in place | A small localized rebuild per drive | Many small exposure windows, accumulating across hundreds |
| Full migration to a new cluster | Copy everything, cut over, retire the old | Two clusters funded at once, one cutover, limited rollback |
Refresh cadence against warranty and failure rates
Warranty period and useful life are different numbers that get conflated. A warranty defines who pays for a replacement part. It says nothing about whether the failure rate has begun to climb. Drives typically show an early-life failure period, a long flat period and a rise at the end, and that rise does not begin on the warranty expiry date.
A continuous cadence, retiring a fixed fraction of the cluster each year, keeps the average age of the fleet stable and avoids a large cohort reaching end of life together. It also turns refresh into a routine the team practices rather than a project undertaken every several years, which matters because the failure modes of a drain are only obvious to people who have run one recently.
The fraction to retire each year follows from the intended service life, and the achievable cadence from how much data movement the cluster can absorb without affecting clients. The two are rarely compatible at first, and the resolution is usually a longer service life with closer monitoring of the aging cohort rather than a faster refresh rate.
Capacity planning so a refresh does not collide with growth
A refresh performed by drain requires free space. The data on the retiring node has to land somewhere, so the cluster needs headroom to absorb one node's worth before the replacement is in service. A cluster near its fullness threshold cannot drain anything.
That constraint interacts badly with growth. Capacity is added when utilization approaches a trigger, and refresh when hardware approaches an age. If those schedules converge, the cluster needs to drain a node at the moment it has no space to drain into. Planning refresh against the capacity curve rather than the calendar puts the headroom for retirement into the sizing model from the start, as in any capacity model that accounts for more than raw stored bytes.
Sequencing helps as well. Adding replacement nodes before draining the old ones means the free space arrives first, making expansion and retirement two phases of one operation rather than competing ones. That ordering has consequences for placement and rebalancing, described in what happens as nodes join a live cluster.
Where Scality RING fits
RING is designed so that nodes and servers can be added to a live system and old hardware retired without taking the namespace offline, the property that makes continuous refresh possible instead of one large migration. Because the namespace spans nodes and sites, retiring hardware does not change the S3 endpoint or the bucket structure applications depend on.
Durability comes from erasure coding and replication selectable per policy, and self-healing detects and repairs lost or corrupted data as a background task. During a refresh that work and the retirement traffic share the same disks and network, so the sequencing above applies directly.
Because RING runs on standard x86 servers in the operator's own facility, hardware generations can be mixed and replaced on the operator's schedule rather than an appliance vendor's cycle, and the metrics showing how a drain is progressing stay local.
What to schedule and record
A refresh works better as a standing annual entry than an occasional project. The entry names the cohort to retire, the date range, the free space required, and the protection policy that must hold throughout. Scheduling it away from the periods when growth lands removes most conflicts before they arise.
For each node retired, the record should hold the drain start and finish times, the volume of data moved, whether an unrelated failure occurred inside the window, and the protection margin at its narrowest. Those entries accumulate into a realistic estimate of the next drain, more useful than any figure supplied at purchase.
Two checks belong before every cohort. That the cluster has headroom to drain a node without crossing a fullness threshold, and that the age of the remaining hardware has not quietly concentrated into one group needing replacement at once. The second is what stops a continuous refresh turning back into the migration it was meant to avoid.