Glossary

Vector database

A vector database stores numeric vectors that represent meaning and returns the ones closest to a query. It finds content by similarity, not exact words. It holds derived data: the source corpus remains the system of record, and the index can be rebuilt from it.

How does a vector database work?

Documents are split into chunks through document chunking. An embedding model turns each chunk into a vector, usually hundreds to thousands of numbers. A query is embedded the same way, and the database returns its nearest neighbors by distance. That lets a search for cancel find the contract clause that says termination, which keyword matching would miss.

Comparing a query with every stored vector is too slow at scale. Approximate indexes such as HNSW, which builds a layered graph of neighbors, skip most comparisons and still find close matches.

What does approximate search cost?

Speed is bought with recall, the share of true nearest neighbors the index actually returns. An approximate index can miss a relevant chunk, and the model then answers without it. Recall at k, how many of the true top k results come back, is the figure to measure, set against the latency the application can tolerate. Metadata filters, such as date or department, narrow the search, and hybrid keyword scoring sharpens it.

Why is the index derived data?

Every vector depends on the embedding model that produced it. If the model changes, every vector has to be recomputed, which means reading the entire source corpus again. The index is therefore treated as rebuildable, and the corpus underneath it is what is kept durable and versioned. A lost index is an inconvenience measured in compute time. A lost or overwritten corpus cannot be recovered. Keeping earlier versions of the corpus also lets an index be rebuilt as it stood at a given date, which matters when an answer has to be reproduced for an audit.

This puts the storage question in the layer below. Source documents sit on object storage, and the index above them is rebuilt from it. How quickly that corpus can be read is a limit on how quickly a model upgrade finishes.

Where does it fit in the AI stack?

It is the retrieval layer in retrieval-augmented generation, serving an AI knowledge base. It is not a general replacement for relational or document databases, since it answers similarity questions and not exact queries. Hybrid search and GraphRAG extend it for cases where plain vector lookup falls short.

Frequently asked questions

Is a vector database the same as a vector index?

A vector index is the search structure. A vector database adds storage, filtering, updates, access control and operations around it. Some existing databases and search engines now include vector indexes.

How large do vector collections get?

They scale with the number of chunks, not documents. One document can produce dozens of chunks, so a corpus of millions of documents yields far more vectors.

What happens to vectors when a document is deleted?

They have to be removed too, or the content stays retrievable. Linking each vector to its source document and version is what makes that possible.