AI Infrastructure

Vector database backups: What else must you protect?

Backing up a vector index is not enough to restore a RAG system. What else has to be protected to rebuild it correctly.

7 min read
Glowing cluster of connected nodes above a server rack under violet lighting

Vector databases are often treated as derived data, on the reasoning that if the index is lost it can simply be rebuilt from the source documents. That reasoning is sound and it is also incomplete. Rebuilding requires the source documents, the embedding model that produced the vectors, the chunking rules that split the documents, and the metadata that governs which results a given user is allowed to see. If any one of those is missing, the rebuilt index will not behave like the one that was lost.

For infrastructure teams supporting retrieval systems in production, this makes the protection scope wider than the database itself. The question is not whether the vector store is backed up, but whether the organization could reproduce a working retrieval layer that returns the same answers it returned before the incident.

Why the index alone is not a recovery plan

An index restore is fast and a rebuild is slow. That difference matters when the retrieval system sits in front of a customer facing application. Re-embedding several million document chunks takes hours or days depending on the model and the compute available, and during that time the application either runs without retrieval or does not run at all.

The reverse problem also exists. An index restored from an old backup is internally consistent but stale, and it will confidently return content that has since been corrected or deleted. In a system that answers questions from company documents, returning a withdrawn policy is worse than returning nothing.

The workable position is to protect both paths: keep a restorable copy of the index for speed, and keep everything needed for a rebuild for correctness. The two serve different incidents.

The six things a retrieval system depends on

Most recovery failures come from protecting the first item on this list and assuming the rest will be available.

ComponentWhat is lost without itWhere it usually lives
Source documentsThe ability to rebuild anything at allObject storage, a content system, or both
Chunking configurationChunk boundaries change, so retrieval results changeApplication code or a config file
Embedding model and versionNew vectors are incompatible with any retained onesModel registry or a container image
Vector index and its parametersFast restore, and reproducible ranking behaviorThe vector database itself
Metadata and filtersPermission filtering and source attributionA relational store or the vector database
Access control mappingWho is allowed to retrieve which documentsIdentity system, replicated into filters

The embedding model row is the one that surprises teams most often. Vectors produced by one model version are not comparable with vectors produced by another, so a rebuild with a newer model produces a different index rather than a restored one. If the model was pulled from an external source and that version is no longer available, the original index cannot be reproduced at all.

Metadata is the part that breaks quietly

In most production retrieval systems, the vector search is only half the operation. The other half is a filter: restrict to documents this user may see, or to this customer's data, or to the current version of a policy. Those filters run against metadata that frequently lives outside the vector database.

When the two are backed up separately and restored to different points in time, the result is a system that runs without error and returns wrong results. A chunk whose permission metadata has been rolled back can be returned to someone who should not see it. This is a data exposure, and it will not appear in any monitoring that watches for failures rather than for incorrect answers.

The mitigation is to treat the vector store and its metadata store as one recovery unit with a consistent recovery point, in the same way a database and its transaction log are treated together. Where that is not possible, the rebuild path should be the primary plan and the restore path should be reserved for cases where both can be recovered to the same moment.

What to protect and how often

Source documents deserve the strongest protection because everything else can be derived from them. They are also usually already protected, since they are ordinary business content, and the useful step is to confirm that the copy the retrieval system indexed is the copy that is being backed up rather than a staging copy somewhere else.

Configuration and model references are tiny and should be versioned alongside application code, which most organizations already do. The gap is usually the model artifact itself: a reference to an external model is not a protected copy of it, and retaining the actual weights removes a dependency on something outside the organization's control.

The index deserves periodic snapshots rather than continuous protection in most cases, because it can be rebuilt and because snapshot frequency has a real cost at scale. The right interval is the one where the age of the snapshot plus the time to catch up with recent changes stays inside the recovery objective. This is the same reasoning applied to how quickly a system has to be back in service rather than how recently it was copied.

The dependency order in a recovery

Recovering a retrieval system has a required sequence, and it is worth writing down before it is needed. Identity and access control have to work before permission filters mean anything. The metadata store has to be available before the index is useful. The embedding service has to be running before any new or changed content can be added. The index itself comes last, whether restored or rebuilt.

That ordering is a specific instance of a general problem in recovery planning, which is that systems depend on other systems coming back first. A retrieval layer is unusually dependent because it sits on top of content, identity and model infrastructure simultaneously, and a plan that covers only the database will stall on the first of those that is not yet available.

It is worth confirming whether the application degrades gracefully when retrieval is unavailable. A system that returns a clear message is in a much better position during a rebuild than one that returns answers with no supporting context and no indication that anything is missing.

Where Scality RING fits in protecting retrieval systems

The source documents behind a retrieval system are ordinary unstructured content at scale, which is what object storage is designed to hold. Keeping them on Scality RING gives the rebuild path a single durable namespace to read from, with versioning so that a document corrected after indexing does not erase the state the index was built against, and with object lock available where the content needs to be defensible.

Index snapshots and retained model artifacts are also a natural fit for the same platform, which means a rebuild reads everything it needs from one system rather than assembling it from several. As the storage design behind a retrieval system matures, keeping the documents, the snapshots and the model artifacts together is what makes the recovery procedure short enough to be tested regularly.

Judge the plan by a rebuild you have actually run

The test is to rebuild the index from protected sources into a separate environment and compare its answers against production for a set of known queries. Teams that do this usually find one missing dependency, most often the exact model version or the chunking parameters, and finding it during an exercise costs an afternoon rather than a customer facing outage.

Record how long the rebuild took, because that number is the real recovery time for the retrieval layer regardless of what the backup schedule suggests. If it is longer than the business will accept, the answer is either faster restore of the index or a smaller gap to close, and both of those are design decisions that have to be made before the incident rather than during it.

See Scality in action

Exabyte-scale object storage for AI data and cyber resilience. Talk to our team about what it can do for yours.

Request a demo