FAISS Vector Similarity

You’ve built a model that turns text, images, or audio into dense vectors — now what? The moment your data grows past a few thousand items, brute-force search over every vector becomes a bottleneck, and your application starts to crawl. That’s the pain this lesson solves: implementing vector similarity with FAISS so you can find the nearest neighbors to a query in milliseconds, even when you have millions of embeddings. FAISS (Facebook AI Similarity Search) is the industry-standard library for this task, and by the end of this lesson you’ll be able to index, search, and filter embeddings with confidence.

The problem this lesson solves

Every recommendation engine, semantic search tool, and RAG (Retrieval-Augmented Generation) pipeline depends on finding vectors that are close to a query vector. Without a dedicated vector index, you’d compute a distance (or dot product) between the query and every stored vector — an O(N) operation. For 1,000 vectors that’s trivial; for 1 million vectors, it’s slow and wasteful.

More critically, production systems need sub-linear search time. FAISS addresses this by constructing an index that can be searched in O(log N) or better, using techniques like inverted files (IVF) and product quantization (PQ). This lesson gives you the exact mental model and code to move from brute-force to scalable similarity search — without drowning in theory.

Pain point: You have millions of embeddings and a naive loop that takes seconds per query. FAISS is the cure — but only if you understand how to choose and build the right index.

Core concept / mental model

Think of FAISS as a gigantic sorted bookshelf for vectors. Instead of scanning every book to find the one closest to your hand, the librarian (FAISS) uses a map to know which shelf (cluster) is most likely to contain your answer. The index is that map — it organizes vectors so that only a small subset needs to be examined for each query.

Key terms

• Vector — a fixed-length array of floats; an embedding from a model like all-MiniLM-L6-v2.
• Index — a data structure that stores vectors and supports similarity search. IndexFlatL2 is the baseline; IndexIVFFlat is the scalable workhorse.
• Metric — the function that measures similarity/distance. FAISS supports L2 (Euclidean) and inner product (IP). IP often correlates with cosine similarity if vectors are normalized.
• Query vector — the vector you want to match against the stored set.
• k — the number of nearest neighbors you want returned.

How an index works

For each stored vector, FAISS assigns it to a cluster (in IVF) or compresses it (in PQ). At search time, FAISS: 1. Determines which clusters the query is closest to (nprobe). 2. Exhaustively searches only those clusters. 3. Returns the top-k nearest vectors.

This is why IVF+PQ can handle billions of vectors — you trade a tiny bit of accuracy (recall) for massive speed.

How it works step by step

Let’s walk through the process of implementing vector similarity with FAISS from scratch.

Step 1: Install FAISS

FAISS is available via pip for CPU:

pip install faiss-cpu

For GPU acceleration (when you have a CUDA-capable GPU):

pip install faiss-gpu

Step 2: Prepare your vectors

You’ll need a NumPy array of shape (num_vectors, embedding_dim) in float32 — FAISS doesn’t accept float64. If your vectors come as Python lists, convert them.

import numpy as np

# Simulate 10,000 random embeddings of dimension 128
np.random.seed(42)
vectors = np.random.random((10000, 128)).astype('float32')

Step 3: Build an index

The simplest index is IndexFlatL2, which does an exact brute-force search — perfect for testing or small datasets (<100K vectors).

import faiss

dimension = vectors.shape[1]
index = faiss.IndexFlatL2(dimension)
index.add(vectors)  # add all vectors to the index
print(f"Total vectors in index: {index.ntotal}")

Step 4: Search

query = np.random.random((1, dimension)).astype('float32')
k = 5  # number of nearest neighbors to retrieve

distances, indices = index.search(query, k)

print("Distances (L2):", distances)
print("Indices of nearest neighbors:", indices)

You’ll get two arrays: distances (the squared L2 distance, smaller is better) and indices (positions in your original array).

Step 5: Scale up (IVF index)

For larger datasets, switch to IndexIVFFlat, which clusters the vectors and searches only nearby clusters.

nlist = 100  # number of clusters
quantizer = faiss.IndexFlatL2(dimension)
index_ivf = faiss.IndexIVFFlat(quantizer, dimension, nlist)

# Train on a subset (must match distribution)
index_ivf.train(vectors)
index_ivf.add(vectors)

# Set nprobe — how many clusters to check. Higher = more accurate but slower.
index_ivf.nprobe = 10

distances_ivf, indices_ivf = index_ivf.search(query, k)
print("IVF distances:", distances_ivf)
print("IVF indices:", indices_ivf)

Pro tip: The train() step is essential for IVF indexes. If you skip it, FAISS raises RuntimeError: Error in train.

Hands-on walkthrough

Now let’s build a real example — a semantic search engine for short document titles using a sentence transformer. This combines FAISS with a real embedding model.

First, install the sentence-transformers library if you don’t have it:

pip install sentence-transformers

from sentence_transformers import SentenceTransformer
import faiss
import numpy as np

# 1. Load a small embedding model
model = SentenceTransformer('all-MiniLM-L6-v2')  # 384-dim embeddings

docs = [
    "How to train a neural network",
    "Fast vector similarity search with FAISS",
    "Building a RAG pipeline with LangChain",
    "Python list comprehensions explained",
    "Fine-tuning Llama 3 for question answering"
]

# 2. Encode documents to vectors (float32)
doc_vectors = model.encode(docs).astype('float32')
print(f"Embedding shape: {doc_vectors.shape}")

# 3. Build FAISS index (IP for cosine similarity on normalized vectors)
dimension = doc_vectors.shape[1]
index = faiss.IndexFlatIP(dimension)

# Normalize vectors to use inner product as cosine similarity
faiss.normalize_L2(doc_vectors)
index.add(doc_vectors)

# 4. Encode a query
query = "How do I search semantically?"
query_vec = model.encode([query]).astype('float32')
faiss.normalize_L2(query_vec)

# 5. Search for top 3 most similar docs
distances, indices = index.search(query_vec, k=3)

print("\nQuery:", query)
for i, idx in enumerate(indices[0]):
    print(f"{i+1}. {docs[idx]} (similarity: {distances[0][i]:.4f})")

Expected output (your numbers may vary slightly):

Query: How do I search semantically?
1. Fast vector similarity search with FAISS (similarity: 0.4523)
2. Building a RAG pipeline with LangChain (similarity: 0.1781)
3. How to train a neural network (similarity: 0.1002)

Now let’s scale to a million vectors and measure performance:

import time
import faiss
import numpy as np

d = 128  # embedding dimension
n = 1_000_000  # one million vectors

# Generate random data
np.random.seed(0)
base_vectors = np.random.random((n, d)).astype('float32')

# Build an IVF index with product quantization (IndexIVFPQ) for memory efficiency
nlist = 1000  # more clusters = better accuracy
m = 16  # number of subquantizers
nbits = 8  # bits per subquantizer

quantizer = faiss.IndexFlatL2(d)  # quantizer must be flat
index_pq = faiss.IndexIVFPQ(quantizer, d, nlist, m, nbits)

# Train and add
index_pq.train(base_vectors[:100000])  # train on a subset
index_pq.add(base_vectors)
index_pq.nprobe = 20

# Query
query = np.random.random((1, d)).astype('float32')
start = time.time()
distances, indices = index_pq.search(query, 10)
print(f"Search took {time.time() - start:.4f} seconds")
print("Nearest neighbor indices:", indices)

Expected output (roughly):

Search took 0.0032 seconds
Nearest neighbor indices: [[452341 999231 102732 500012 773411 902341 123451 392013 882341 204551]]

That’s sub-10ms for a million vectors — the power of FAISS in action.

Compare options / when to choose what

FAISS offers several index types, each balancing speed, memory, and accuracy. Here’s a quick comparison:

How to choose:

1. If your dataset fits in RAM and is under ~100K vectors, use IndexFlatL2 or IndexFlatIP — don’t overcomplicate.
2. If you have 100K to 1M vectors and need fast search, use IndexIVFFlat; set nprobe between 10 and 50.
3. If you have millions of high-dimensional vectors and memory is tight, use IndexIVFPQ — it compresses vectors but costs accuracy.
4. If you need fast insertion and high recall, consider IndexHNSWFlat (graph-based), though it uses more memory.

Troubleshooting & edge cases

FAISS is powerful but unforgiving. Here are the most common pitfalls and how to fix them.

1. RuntimeError: Error in train

This happens when you call add() on a non-flat index without calling train() first. Solution:

# Always train before adding
index.train(vectors)
index.add(vectors)

2. Vectors are not float32

FAISS expects numpy.float32 arrays. If you pass float64, you’ll get an error like TypeError: in method 'IndexFlat_add', argument 3 of type 'float32 []'. Fix:

vectors = vectors.astype('float32')

3. Empty query or wrong shape

Your query matrix must be 2D with shape (num_queries, dim). A 1D array will raise ValueError or silently give wrong results. Always reshape:

query = query.reshape(1, -1).astype('float32')

4. Cosine similarity gives negative values

If you’re using IndexFlatIP and expect cosine similarity, you must normalize vectors first. Unnormalized inner product can yield negative or inconsistent values. Use faiss.normalize_L2() before adding and searching.

5. Search returns fewer than k results

This can happen when the index contains fewer than k vectors, or when nprobe is too low and a cluster has too few vectors. Check index.ntotal and increase nprobe.

What you learned & what's next

You’ve now implemented vector similarity with FAISS, from a basic flat index to a scalable IVF+PQ setup. You can:

• Explain how FAISS organizes vectors into clusters for fast search.
• Choose the right index type based on dataset size, memory, and accuracy needs.
• Complete a hands-on exercise that encodes documents, builds an index, and retrieves relevant items.
• Troubleshoot common issues like training errors, dtype mismatches, and normalization.

These skills are the foundation for building production-grade retrieval systems — semantic search, RAG pipelines, and recommendation engines. In the next lesson, you’ll learn how to evaluate the quality of your retrieval results using metrics like recall@k and mean reciprocal rank, so you can fine-tune your FAISS index for real-world performance.

Keep experimenting — try different index types with your own embeddings and measure the speed/accuracy trade-off.