Scality Blog | Object storage, AI data infrastructure & cyber resilience

S3 lifecycle policies: Testing before deleting data

Written by Joshua Silvia | Sep 18, 2026, 5:09:20 PM

A lifecycle policy is a small piece of configuration that deletes data continuously, without confirmation, and without any human involvement after the moment it is saved. That is precisely why it is valuable, since manual cleanup at object storage scale does not happen. It is also why a mistake in one is more consequential than a mistake in almost any other storage setting.

The risk is not the syntax. Lifecycle rules are simple to write. The risk is that the set of objects a rule matches is larger than the author expected, and nothing in the process makes that visible before the deletions begin.

What lifecycle rules can get wrong

Prefix scope is the most common error. A rule intended for objects under a path can match a wider set if the prefix is written slightly differently from how keys are actually named, and object storage prefixes are literal string matches rather than directory paths. A rule with an empty or missing prefix applies to the entire bucket.

Tag based rules carry a different risk, because they depend on every object being tagged correctly by whatever wrote it. An application that stops applying a tag after an upgrade will produce objects that fall outside a retention rule or inside a deletion rule without anyone changing the policy.

Age is measured from the object's creation or last modification, not from when it was last read. An archive that is accessed regularly but never rewritten looks old to a lifecycle rule, and a rule that expires objects older than a year will remove data that is in active use.

Testing before it runs

The essential step is to determine, in advance, exactly which objects a rule will match, and to have a person look at that list. Every other check is secondary.

StepWhat it catchesEffort
List objects matching the rule criteriaScope errors, unexpected matchesLow, a scripted listing
Count and total the size of matchesA rule matching far more than intendedLow
Review a sample of matched keysData nobody realized was in scopeLow, and the most valuable check
Run on a copy or a limited prefix firstBehavior differences from expectationModerate
Apply as a transition before an expiryLets the effect be observed reversiblyModerate
Confirm with the data ownerBusiness context the storage team lacksLow, one conversation
Check for Object Lock and legal holdsRules that will fail or conflictLow

The staged approach in the fifth row is worth using whenever a rule affects a large volume. Moving objects to a different storage class or prefix first makes the consequence observable and reversible, and the expiry can be added once nothing has broken for a period.

The interaction with versioning

In a versioned bucket, an expiry rule on current objects does not delete anything. It creates delete markers, and the previous versions remain stored. A team that applies an expiry rule expecting to free capacity will find that capacity unchanged and the object count higher.

Removing the data requires a separate rule for non-current versions, and often a third for expired delete markers. Writing all three together, with the relationships understood, avoids the common sequence of applying one rule, seeing no effect, and applying increasingly aggressive rules until something unintended is removed.

Incomplete multipart uploads need their own rule again, since neither of the above touches them. Most environments benefit from one that aborts incomplete uploads after a few days as a standing policy.

Rules that cannot be undone, and rules that can

It is worth separating lifecycle actions by reversibility when reviewing them. Transitioning objects between storage classes or tiers is reversible, at the cost of moving the data back. Expiring a current version in a versioned bucket is reversible, since the previous version remains. Expiring non-current versions is not reversible, and neither is any expiry in a bucket without versioning.

Applying more review effort to the irreversible actions than to the reversible ones is a reasonable allocation of attention. In practice this means a transition rule can be approved quickly while a permanent expiry deserves the full list of checks and a named approver.

Where Object Lock is in force, expiry actions will not remove locked objects before their retention period ends, which is the intended behavior and can produce rules that appear to be failing when they are in fact being correctly refused. Knowing that in advance prevents a rule from being escalated in response.

Where Scality RING fits

Scality RING supports S3 lifecycle rules for transition and expiration, including non-current versions and incomplete multipart uploads, which means capacity management can be automated per prefix rather than handled through periodic cleanup projects. Object Lock provides the counterweight: data that must not be removed early is protected at the storage layer, so a lifecycle rule cannot delete it regardless of how it is written.

Using both together is the arrangement worth aiming for. Lifecycle rules keep ordinary working data under control automatically, and Object Lock guarantees that the material with a retention obligation is outside their reach. That combination makes a mistake in a lifecycle rule an inconvenience rather than an incident for the data that matters most.

Review rules on a schedule, not only when writing them

Lifecycle policies are written once and then run for years while the data they act on changes. A rule that was correct when a bucket held one application's output can behave very differently after a second application starts writing to the same prefix.

An annual review that re-runs the matching listing for every active rule takes an afternoon and finds the drift. The output worth keeping is a record of what each rule currently matches, how much it removes per month, and who agreed to it, since that is what allows a future question about missing data to be answered rather than investigated.