Versioning is one of the most useful features in object storage and one of the most common causes of capacity that nobody can account for. The mechanism is simple: when an object is overwritten or deleted, the previous state is retained rather than discarded. The consequence is that a bucket's stored capacity bears no necessary relationship to what a listing of it shows, and the difference can be several times over.
The gap is not a defect. It is the feature working as designed, protecting against overwrites and accidental deletion. What causes trouble is enabling it without a corresponding expiry policy, at which point capacity grows in proportion to how often applications rewrite objects rather than to how much data the business has.
Why a listing does not show the real size
A standard object listing returns current versions. Previous versions exist alongside them and are only visible through a listing that specifically asks for them. An application that overwrites the same 50 MB object every hour creates 24 versions a day and a listing that still shows one object of 50 MB.
Deletion behaves in a way that surprises people the first time. Deleting an object in a versioned bucket does not remove anything. It adds a marker indicating that the current state is deleted, while every prior version remains stored. A bucket can therefore appear empty and still consume its full capacity, and a cleanup that deletes millions of objects can free nothing at all.
Understanding this mechanism is what turns an unexplained growth curve into an explainable one, which is usually the first thing anyone wants when reviewing capacity on an object store.
Where the capacity actually accumulates
| Source | How it grows | Visible in a normal listing |
|---|---|---|
| Previous versions of overwritten objects | One per overwrite, at full object size | No |
| Versions behind delete markers | Retained indefinitely after deletion | No |
| Delete markers themselves | Small, but numerous in high churn buckets | No |
| Incomplete multipart uploads | Parts of failed uploads never cleaned up | No |
| Applications rewriting whole objects | Every small edit stores a full new copy | Only the current version |
| Sync tools that recopy unchanged data | Identical content stored repeatedly | Only the current version |
The last two rows describe the most expensive pattern. Object storage has no concept of modifying part of an object, so an application that changes one field in a file and writes it back stores the entire object again. A workload that edits a 200 MB file ten times a day is storing 2 GB a day in a bucket that appears to hold 200 MB.
Synchronization tools configured to compare by timestamp rather than by content can produce the same effect at scale, rewriting objects that have not changed and creating a new version each time.
Finding out what a bucket really holds
The first step is a listing that includes all versions, which most tools and clients support explicitly. Summing the sizes returned gives the actual stored capacity and, compared against the current version total, the size of the retained history.
The second is a separate check for incomplete multipart uploads, which are invisible to both listings and are a common source of capacity in environments where large uploads sometimes fail. They deserve their own investigation and cleanup rule because nothing removes them automatically unless configured to.
The third is to identify which prefixes are producing the versions. The distribution is usually uneven: one or two applications with a rewrite pattern generate the majority, and addressing those is more effective than applying a policy uniformly across the bucket.
Expiry policy, not a cleanup project
The durable answer is a lifecycle rule that expires non-current versions after a defined period, chosen for what the versions are actually protecting against. If the purpose is recovery from accidental overwrite, a few weeks is usually sufficient. If the purpose is reproducibility of a dataset, the period has to match the retention of whatever references those versions.
Different prefixes can carry different rules, and they usually should. Applying a 7 day expiry to a working area and a much longer one to prefixes that feed regulated processes is more defensible than a single number that is either too short for the important data or too long for the churn.
Any expiry rule should be tested before it runs at scale, because expiring a non-current version is permanent. Running the rule against a copy or a limited prefix first is the same discipline that applies to any lifecycle policy that deletes data.
Where Scality RING fits
Scality RING supports S3 versioning and lifecycle rules, so non-current versions can be expired automatically on a schedule that reflects what each prefix is protecting. Because versioning is applied per bucket and lifecycle rules can be scoped to prefixes, an environment can keep deep history where it is needed without paying for it everywhere.
The operational recommendation is to enable versioning and an expiry rule at the same time rather than enabling versioning first and adding the rule later, since the interval between those two events is where most of the unexplained capacity is created. Reporting stored capacity including all versions, alongside the current version total, keeps the difference visible from the outset rather than emerging as a surprise at a capacity review.
Report both numbers, permanently
Capacity reporting for versioned buckets should always show two figures: the size of current versions and the total including history and incomplete uploads. A single number hides exactly the thing that is growing, and teams that report both catch a change in application behavior within days rather than at the next review.
A useful ratio to track is history as a percentage of current data, per prefix. A prefix where retained versions are three times the current data is telling you something specific about an application, and that is a more actionable finding than a bucket that is larger than expected.