Glossary
Storage compression
Storage compression stores data in fewer bytes by encoding repeated patterns, and it restores the original exactly when the data is read. It shrinks text and logs a great deal, but it does almost nothing for media files or encrypted data, so planning needs a measurement of the real data.
What compresses and what does not?
Compression looks for repetition, such as the same field names appearing in every line of a log, and replaces it with short references. On read, the references expand back to the original. Data with a lot of repetition shrinks well. Data that is already compact, or that has no pattern left to find, barely changes.
| Shrinks well | Barely changes |
|---|---|
| Text, logs and documents | Photos, video and audio |
| CSV and JSON | ZIP and other archives |
| Databases with repeated values | Parquet and other columnar formats |
| Source code | Encrypted data, which looks random |
A single ratio quoted for a whole environment is therefore an average over very different content. An archive of logs and one of video can sit on the same system with ratios far apart.
Why does the order of reduction matter?
The sequence that works is deduplication first, then compression, then encryption. Deduplication removes whole repeated blocks. Compression then shrinks what is left. Encryption comes last because its output looks random, and random data has no repetition to remove.
The failure case is an application team enabling client-side encryption in front of the storage. A capacity plan built on a 2 to 1 ratio for 500 TB of data assumed about 250 TB of storage. Once the data arrives already encrypted, the ratio drops close to 1 to 1, and the system needs about 500 TB. The savings disappear and nothing in the storage reports an error. The effect is the same for backup data, covered under backup compression.
How much will it save?
Only a sample of the organization's own data answers that. Compression also costs processing, usually small on current hardware, and it changes what a terabyte means for capacity numbers. Planning with an unmeasured ratio turns capacity planning into a guess. Where a plan relies on reduction, the ratio is better treated as a bonus than as a baseline, with protection overhead, described under erasure coding, accounted for separately.
Compression is also separate from tiering, which changes where data lives, not how many bytes it takes.
Frequently asked questions
Does compression slow down storage?
It uses processor time on write and on read. On current hardware the cost is usually small, and for compressible data the smaller volume can even reduce the data written to drives.
Is compression lossless?
Storage compression is lossless. The data read back is identical to the data written.
Can encrypted data be compressed?
Hardly. Encryption makes data look random, so compression has to happen before it. Data encrypted by the application before it reaches the storage cannot be reduced there.
Related terms
- Backup compression: compression applied to backup data.
- Deduplication: removing repeated blocks before compression.
- Storage management: where reduction ratios feed capacity planning.
- Storage tiering: a different way to lower cost per terabyte.
- Erasure coding: protection overhead counted alongside reduction.
- Thin provisioning: another way capacity is stretched.