How to Build an In-Memory Vector Store in Python

Build a lightweight in-memory vector store using a Python dict and cosine similarity for fast nearest-neighbor searches.

Easy Python 3.9+ Aug 9, 2026 AI & LLM integration patterns 13 views 0 copies

Python code

38 lines
Python 3.9+
import math
from typing import Dict, List, Optional


class InMemoryVectorStore:
    def __init__(self) -> None:
        self.vectors: Dict[str, List[float]] = {}
        self.index: Dict[str, List[str]] = {}  # query -> list of ids sorted by similarity

    def add(self, vector_id: str, vector: List[float]) -> None:
        self.vectors[vector_id] = vector
        self.index.clear()  # invalidate cache on any insertion

    def _cosine_similarity(self, a: List[float], b: List[float]) -> float:
        dot = sum(x * y for x, y in zip(a, b))
        norm_a = math.sqrt(sum(x * x for x in a))
        norm_b = math.sqrt(sum(x * x for x in b))
        if norm_a == 0.0 or norm_b == 0.0:
            return 0.0
        return dot / (norm_a * norm_b)

    def search(self, query_vector: List[float], top_k: int = 3) -> List[tuple]:
        if not self.vectors:
            return []
        scored = [(vector_id, self._cosine_similarity(query_vector, vec))
                  for vector_id, vec in self.vectors.items()]
        scored.sort(key=lambda item: item[1], reverse=True)
        return scored[:top_k]


if __name__ == "__main__":
    store = InMemoryVectorStore()
    store.add("doc1", [1.0, 0.0, 0.0])
    store.add("doc2", [0.8, 0.6, 0.0])
    store.add("doc3", [0.0, 1.0, 0.0])

    results = store.search([1.0, 0.0, 0.0], top_k=2)
    print(results)

Output

stdout
[('doc1', 1.0), ('doc2', 0.8)]

How it works

The InMemoryVectorStore class stores vectors in a plain dict keyed by document ID. add() inserts a new vector and clears the (currently unused) similarity cache to keep behavior consistent. _cosine_similarity() computes the dot product divided by the product of the Euclidean norms, returning 0.0 for zero vectors to avoid division by zero. search() scores every stored vector against the query, sorts them descending, and returns the top_k results. This O(n) per-query scan is perfectly fine for thousands of vectors but should be replaced with an ANN (approximate nearest neighbor) index for millions.

Common mistakes

  • Forgetting to handle zero-length vectors, causing a ZeroDivisionError
  • Not clearing the cache on add, serving stale search results
  • Assuming sorted order — the method does NOT mutate the internal store, it returns a new list

Variations

  1. Use numpy arrays and `np.dot` / `np.linalg.norm` for faster math on large vectors
  2. Swap the linear scan for an ANN library like FAISS or Annoy when scaling to millions of vectors

Real-world use cases

  • Ranking candidate documents by embedding similarity in a retrieval-augmented generation (RAG) pipeline.
  • Matching user query embeddings to pre-computed product or FAQ embeddings for semantic search.
  • Deduplicating or clustering similar text snippets during offline data cleaning and analysis.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from AI & LLM integration patterns

Related tutorials and quizzes for this topic.