How to Build an In-Memory Vector Store in Python
Build a lightweight in-memory vector store using a Python dict and cosine similarity for fast nearest-neighbor searches.
Python code
38 linesimport math
from typing import Dict, List, Optional
class InMemoryVectorStore:
def __init__(self) -> None:
self.vectors: Dict[str, List[float]] = {}
self.index: Dict[str, List[str]] = {} # query -> list of ids sorted by similarity
def add(self, vector_id: str, vector: List[float]) -> None:
self.vectors[vector_id] = vector
self.index.clear() # invalidate cache on any insertion
def _cosine_similarity(self, a: List[float], b: List[float]) -> float:
dot = sum(x * y for x, y in zip(a, b))
norm_a = math.sqrt(sum(x * x for x in a))
norm_b = math.sqrt(sum(x * x for x in b))
if norm_a == 0.0 or norm_b == 0.0:
return 0.0
return dot / (norm_a * norm_b)
def search(self, query_vector: List[float], top_k: int = 3) -> List[tuple]:
if not self.vectors:
return []
scored = [(vector_id, self._cosine_similarity(query_vector, vec))
for vector_id, vec in self.vectors.items()]
scored.sort(key=lambda item: item[1], reverse=True)
return scored[:top_k]
if __name__ == "__main__":
store = InMemoryVectorStore()
store.add("doc1", [1.0, 0.0, 0.0])
store.add("doc2", [0.8, 0.6, 0.0])
store.add("doc3", [0.0, 1.0, 0.0])
results = store.search([1.0, 0.0, 0.0], top_k=2)
print(results)
Output
[('doc1', 1.0), ('doc2', 0.8)]
How it works
The InMemoryVectorStore class stores vectors in a plain dict keyed by document ID. add() inserts a new vector and clears the (currently unused) similarity cache to keep behavior consistent. _cosine_similarity() computes the dot product divided by the product of the Euclidean norms, returning 0.0 for zero vectors to avoid division by zero. search() scores every stored vector against the query, sorts them descending, and returns the top_k results. This O(n) per-query scan is perfectly fine for thousands of vectors but should be replaced with an ANN (approximate nearest neighbor) index for millions.
Common mistakes
- Forgetting to handle zero-length vectors, causing a ZeroDivisionError
- Not clearing the cache on add, serving stale search results
- Assuming sorted order — the method does NOT mutate the internal store, it returns a new list
Variations
- Use numpy arrays and `np.dot` / `np.linalg.norm` for faster math on large vectors
- Swap the linear scan for an ANN library like FAISS or Annoy when scaling to millions of vectors
Real-world use cases
- Ranking candidate documents by embedding similarity in a retrieval-augmented generation (RAG) pipeline.
- Matching user query embeddings to pre-computed product or FAQ embeddings for semantic search.
- Deduplicating or clustering similar text snippets during offline data cleaning and analysis.
Sponsored
More from AI & LLM integration patterns
- Cache LLM Completions by Hashing the Prompt in Python easy
- Chain of Thought Prompting in Python: Step-by-Step Reasoning Demo easy
- Circuit Breaker Pattern in Python for LLM API Calls medium
- Cosine Similarity to Retrieve Top K Chunks in Python easy
- Demonstrate Prompt Injection Bypass in Python easy
- How to Accumulate Streamed Tokens into a Final String in Python easy
Keep learning
Related tutorials and quizzes for this topic.