FAISS Index Architecture for Vector Similarity Search

One million 768-dimensional float32 vectors require roughly 3 GB for the raw vectors alone. At much larger scales, “compute cosine similarity” stops being an implementation plan and becomes a memory-layout and index-architecture problem.

I use FAISS to reason about this trade-off space. In image, face and speech embedding systems, the practical question is not merely which distance function is available, but which recall loss is acceptable under a defined latency and memory budget.

Exact search as the baseline

IndexFlatL2 and flat inner-product search store vectors and scan every candidate. Construction is inexpensive, but query work grows approximately with:

O(N × d)

I keep an exact index as the quality baseline before evaluating approximate methods. Otherwise a faster query result says little about how many true neighbors were lost.

IVF: reduce the candidate space

An inverted-file index assigns vectors to coarse centroids and probes only selected partitions. nlist controls partitioning and nprobe controls how much of that partitioning is searched.

Increasing nprobe generally raises recall and latency together. I therefore select it from a measured recall/latency curve rather than a fixed recipe.

Product Quantization: memory for approximation

Product Quantization replaces full vectors with compact subspace codes. This can greatly reduce RAM and improve cache behavior, but quantization error becomes part of the retrieval contract.

For that reason I compare PQ results against the exact baseline on representative data.

HNSW and graph search

HNSW can provide strong recall/latency behavior, but its graph consumes memory and takes time to construct. Build time, serialized size, reload time and update behavior matter in a service that frequently replaces embeddings.

Cosine similarity is not an index type

For normalized vectors:

cos(x,y) = x · y

so cosine search can be implemented through inner product when the normalization contract is correct. Mixing normalized and non-normalized vectors silently changes ranking semantics.

GPU does not automatically win

FAISS has optimized GPU paths, but end-to-end latency can include host/device copies and batching. For small requests, transfer can dominate computation.

I therefore measure search time separately from host-to-device transfer, device-to-host transfer and queueing.

What I measure

A useful comparison records:

• Recall@k against exact search,
• median and tail latency,
• RAM and index overhead,
• build/reload time,
• update/rebuild behavior,
• CPU/GPU utilization and transfer cost.

A vector database can add persistence, filtering and distribution, but it does not remove these index-level trade-offs.

References

• Douze et al., The Faiss Library
• Johnson, Douze and Jégou, Billion-Scale Similarity Search with GPUs
• Meta AI, FAISS repository and wiki