Glossary

Data gravity

Data gravity is the tendency of applications, services and compute to cluster around large concentrations of data, because moving the data grows slower, costlier and riskier as it accumulates. Dave McCrory coined the term in 2010 to describe how data attracts the services that use it.

Where the pull comes from

Two things grow together: the dataset, and the number of systems that depend on it. Every analytics job, application, backup policy and report that attaches to the data adds a reason to leave it in place and to put the next system beside it. Reads across a boundary are slower than local reads and, in a public cloud, billed. The dataset remains movable in principle while becoming too expensive to move in practice.

Four costs that grow with the dataset

  • Transfer time. Bandwidth stays fixed while data grows, so a move that took a weekend at 100 TB takes months at several petabytes.
  • Exit charges. The egress fees for reading a large dataset out of a public cloud can exceed a year of its storage cost.
  • Dependencies. Each attached application has to be repointed, retested and cut over, sometimes by teams that did not know the data was moving.
  • Fidelity. Metadata, permissions and retention settings have to arrive intact, the central problem of data mobility.

A worked example

Twenty petabytes of video land in one cloud region. Over three years, editing tools, transcoding jobs and a recommendation model are deployed in that region because the data is there. Moving it now means weeks of continuous transfer even at 100 Gb/s, a bill for reading every byte out and a coordinated cutover of every service that touches it. The video has not changed; each service added has raised the cost of leaving.

When does compute move instead?

Because the cost of moving only rises, the decision with the most leverage is where a dataset lands when it is created. Once a dataset has gravity, the usual response reverses the default: new compute is placed next to the data rather than the data being copied to the compute. That reasoning puts training data on private cloud or hybrid cloud storage beside the processors that use it. Data gravity describes the force; data locality is the deliberate placement decision that works with it, and data mobility is the capability that works against it.

Frequently asked questions

Does data gravity apply on premises?

Yes. A multi-petabyte archive in a data center has the same pull. A public cloud adds egress fees to the cost of leaving, but transfer time and dependencies are the same.

Can data gravity be reduced after the fact?

Partly. Standard interfaces such as the S3 API make applications less tied to one platform, and replication to a second location keeps an exit ready, but a mature dataset stays costly to move.