Deleting data sounds like the cheapest thing a storage system can do. At small scale it is. At the scale of hundreds of millions of objects expiring under a lifecycle rule, or a retention period ending for a whole year of backups at once, deletion becomes a sustained background workload with its own effect on performance, capacity reporting and, occasionally, data that was meant to be kept.
Most teams discover this the first time a large expiry runs. Capacity does not fall when expected, metadata operations slow down for other tenants, or a lifecycle rule removes more than intended. None of these are failures of the storage system as such. They follow from how object stores handle deletion internally, and they can be planned for.
Why deleting an object is not instant
In most object stores a delete request does not erase data on disk. It updates metadata so the object is no longer visible, and the space it occupied is reclaimed later by a separate process, usually called garbage collection or space reclamation. The split exists for good reasons: it keeps the delete request fast, it allows replication to propagate the delete safely, and it avoids rewriting erasure-coded stripes every time one object in them goes away.
The consequence is a gap between when an object disappears from listings and when its capacity becomes available again. For a handful of objects the gap is invisible. For a mass expiry it can run to days, and capacity dashboards that report used space rather than reclaimable space will show the cluster as fuller than it effectively is.
How lifecycle expiry actually runs
Lifecycle rules are evaluated by a background process that scans buckets, identifies objects matching the rule, and issues deletions. The mechanics of testing a lifecycle policy before it deletes data are covered separately. What matters at scale is that expiry is not a single event. It is a queue of work processed at a rate the system can sustain, and that rate determines how quickly both visibility and capacity change.
Several details shape how a mass expiry behaves:
- Versioning. On a versioned bucket, expiring the current version creates a delete marker and turns the old data into a noncurrent version. Capacity is only freed when a separate noncurrent-version rule removes it, and the hidden capacity in versioned buckets often explains why an expiry freed far less than expected.
- Incomplete multipart uploads. Parts of uploads that never completed are not objects and are not removed by ordinary expiry rules. They need their own abort rule.
- Retention locks. Objects under an active retention period cannot be deleted. A lifecycle rule that targets them simply has no effect until the lock expires.
- Replication. On replicated buckets, whether deletes propagate to the replica depends on configuration. A mass expiry at the primary may or may not free capacity at the second site.
What a mass expiry costs the cluster
Deletion at scale consumes the same resources as other metadata-heavy operations. Listing, marking, propagating and reclaiming each involve metadata lookups and updates, and reclamation involves reading and rewriting data where partially freed stripes are compacted.
| Stage | What happens | Main resource used | What to watch |
|---|---|---|---|
| Rule evaluation | Buckets scanned for matching objects | Metadata reads | Listing latency for other workloads |
| Logical delete | Objects hidden, delete markers written | Metadata writes | Request latency on the affected buckets |
| Replication of deletes | Delete operations sent to replica sites | Inter-site bandwidth, remote metadata | Replication lag during the expiry |
| Space reclamation | Freed space returned, stripes compacted | Disk reads and writes | Rebuild and client throughput competing with compaction |
| Capacity reporting | Used and free figures updated | Accounting processes | Gap between expected and reported free space |
Running a large expiry during a rebuild or a rebalance stacks background work, and the three compete for the same disks. Scheduling them apart, or throttling one while another runs, keeps client latency predictable.
When deletion goes wrong
The more serious risk is not slowness but scope. A lifecycle rule with a missing prefix filter, a retention period entered in days instead of years, or a rule copied from a test bucket to production can remove data that was meant to stay. Because deletion is asynchronous and often logged only in aggregate, the mistake may not surface until someone tries to read the data.
Safeguards worth having in place before any large expiry:
- Dry runs. Count matching objects and bytes before enabling the rule, and compare the result with what the team expects.
- Staged rollout. Apply the rule to one bucket or prefix first, confirm the effect, then widen it.
- Versioning or a short grace period. Keeping noncurrent versions for a few days after expiry gives a window to reverse a mistake before capacity is reclaimed.
- Retention locks on data that must survive. A lock prevents a misconfigured rule from deleting protected data at all.
- Evidence of deletion. Where deletion is a compliance obligation, the record of what was removed and when matters as much as the removal itself, as discussed in what storage can prove about deletion.
Planning capacity around expiry
Capacity forecasts often assume that data reaching the end of its retention frees space on the day it expires. In practice the freed space arrives later and in a less tidy curve. Forecasts are more accurate when they track three numbers separately: logical data visible to clients, data awaiting reclamation, and physical capacity in use. The difference between the first and the third is where mass expiry hides, and it is the figure to check before ordering hardware on the strength of an expiry that has not finished reclaiming.
Where Scality RING fits
Scality RING exposes an S3 interface, so the lifecycle, versioning and retention settings described here are managed through the same API that applications already use. Data protection is configured per policy with erasure coding or replication, and background tasks such as space reclamation run on the same cluster that serves clients, drawing on the same resources as self-healing and rebalancing.
RING emits operational metrics and logs that the operator keeps locally, which is how the progress of a mass expiry, the space still awaiting reclamation, and the effect on client latency become visible while the expiry is running rather than after it has finished.
A checklist before the next large expiry
- How many objects and bytes match the rule, including noncurrent versions?
- Are incomplete multipart uploads covered by a separate rule?
- Do deletes propagate to replica sites, and is that intended?
- Is any rebuild, rebalance or migration scheduled for the same window?
- Which capacity figure will show the effect, and when is it expected to move?
- What record of the deletion will exist afterwards, and who needs it?
Treated as a planned workload rather than a side effect of a policy, mass expiry becomes predictable: capacity arrives when forecast, other tenants do not notice, and nothing disappears that was meant to stay.