What Is a Vector Database?

A vector database is a data store built to hold embeddings, the lists of numbers that represent the meaning of text, images or other content, and to find the stored items whose vectors are closest to a query vector. It answers “what is most similar to this?” rather than “what exactly matches this?”. Vector databases are the search layer behind semantic search, recommendations and most retrieval-augmented generation (RAG) systems.

The term covers two kinds of product: dedicated vector databases, and general-purpose databases with vector support added, such as PostgreSQL with the pgvector extension, or search engines that index vectors alongside text.

How a Vector Database Works

  1. Store. Each record holds a vector (for example, 1,024 or 1,536 numbers produced by an embedding model), the original content or a pointer to it, and metadata such as document ID, date, language and access group.
  2. Index. The database builds an index that makes similarity search fast. Comparing a query with every stored vector (exact search) gives perfect results but slows down as the collection grows. Approximate nearest neighbour (ANN) indexes trade a little accuracy for a large gain in speed.
  3. Query. The application embeds the user’s query with the same model, sends the vector, and receives the top matches ranked by a distance measure such as cosine similarity or dot product.
  4. Filter. Most real queries combine similarity with conditions: “closest matches, but only documents this user may read, published after January”.

The most widely used ANN index is HNSW (Hierarchical Navigable Small World), a layered graph described by Malkov and Yashunin, in which search starts on a sparse top layer and moves down to denser layers, giving logarithmic scaling (arXiv 1603.09320). Other options include IVF (inverted file) indexes, which group vectors into clusters, and quantized indexes that compress vectors to save memory.

Filtering Is Where Projects Go Wrong

Filtering and approximate search do not combine as naturally as people expect. The pgvector documentation shows the problem clearly: with an approximate index, the filter can be applied after the index scan, so with the default HNSW search size of 40 candidates and a filter that matches 10% of rows, only about 4 rows come back on average. pgvector added iterative index scans to keep searching until enough rows match, and it recommends partial indexes or partitioning for common filters (pgvector). Every vector store handles this differently, and it is the first thing to test with real data and real permission rules.

Dedicated Store or Existing Database

SituationUsual choice
Up to millions of vectors, data already in PostgreSQLPostgreSQL with pgvector
Need for joins with business data and one backup processPostgreSQL with pgvector
Very large collections, high query rates, multi-tenant search as a serviceDedicated vector database
Search product that already runs on a text search engineThat engine’s vector features

Keeping vectors next to the business data has a practical advantage: access rules, transactions and deletions apply in one place. When a document is removed, its vectors go with it. With a separate store, the team has to keep the two in sync.

Common Failure Modes

  • Mixed embedding models. Vectors from different models, or different versions of one model, are not comparable. Changing the model means re-embedding the collection.
  • No keyword search. Vector search is weak on exact identifiers, part numbers and rare names. Hybrid search, combining vectors with keyword (BM25) search, fixes most of this.
  • Recall never measured. ANN settings trade speed for recall. Without a test set, nobody knows how many relevant documents are silently missed.
  • Permissions applied after retrieval. Filtering results in the application after the search can leak data through counts or summaries, and wastes the result slots.
  • Stale data. Updated documents keep old vectors if the indexing pipeline does not track changes.

A vector database stores what an embedding model produces. It is one retrieval option among several; a knowledge graph stores explicit relationships instead of similarity, and text-to-SQL answers questions over structured tables. In an agent system, vector search is usually exposed as a tool the agent can call.

Vector Databases in Practice

The store is picked last, after retrieval quality has been measured on real questions. In many projects PostgreSQL with pgvector is enough and keeps the architecture simple: one database, one access model, one backup. A dedicated vector store is chosen when collection size or query volume outgrows it, and in either case the user’s permissions are applied as filters inside the query, so retrieval can never return a passage the user could not open.

Related services: RAG and CAG knowledge assistants and private LLM and data residency.

Back to Glossary

Let's talk about your project

We'll help you harness AI and ML technologies to create innovative software solutions

File requirements: pdf, doc, docx, rtf, ppt, pptx — max 15 MB

No file selected