Vector Databases Explained: Indexes, Metadata and Retrieval

What a vector database stores

A typical record contains a vector, the original text or a pointer to it, and metadata such as tenant, document type, timestamp or permissions. The vector supports similarity search; metadata makes the results useful and safe.

Why approximate search exists

Comparing a query with every vector in a large collection becomes expensive. Approximate nearest-neighbour indexes reduce the search space so the system can return strong candidates quickly. The trade-off is small differences in recall for large gains in latency and scale.

Metadata is not optional

If two customers share an index, tenant filtering should happen during retrieval. The same applies to language, document status and access permissions. A semantically similar document that the user is not allowed to see is still a wrong result.

Top-k is only the beginning

Fetching the top 20 candidates does not mean you should put all 20 into the model context. Rerank them, remove duplicates, apply diversity rules, then build the final context around the best evidence.

Measure retrieval separately from generation

Track retrieval recall, precision, latency and the percentage of answers with sufficient supporting evidence. If retrieval is weak, changing the LLM can hide the symptom without fixing the system.

The best vector database is the one whose indexing, filtering and latency characteristics match your workload. Start from the retrieval contract, then pick the technology.