How to cache embeddings with a Python dict to avoid recomputation

Caches embeddings computed from text in a dictionary keyed by SHA-256 hash, returning cached results for repeated calls.

Medium Python 3.9+ Aug 9, 2026 AI & LLM integration patterns 15 views 0 copies

embedding cache dict hashing llm

Python code

33 lines

Python 3.9+

import hashlib
import time


class EmbeddingCache:
    def __init__(self):
        self.cache = {}

    def _hash_text(self, text):
        return hashlib.sha256(text.encode()).hexdigest()

    def get_embedding(self, text, compute_func):
        key = self._hash_text(text)
        if key not in self.cache:
            start = time.time()
            self.cache[key] = compute_func(text)
            print(f"Computed embedding in {time.time() - start:.6f}s")
        else:
            print("Used cached embedding")
        return self.cache[key]


if __name__ == "__main__":
    def mock_embedding(text):
        return [len(text), sum(ord(c) for c in text)]

    cache = EmbeddingCache()
    text = "Hello world"

    emb1 = cache.get_embedding(text, mock_embedding)
    emb2 = cache.get_embedding(text, mock_embedding)
    print(f"Embeddings equal: {emb1 == emb2}")
    print(f"Cache size: {len(cache.cache)}")

Output

stdout

Computed embedding in 0.000012s
Used cached embedding
Embeddings equal: True
Cache size: 1

How it works

The cache dictionary maps a SHA-256 hash of the input text to the computed embedding vector. On each call, the key is derived from the text via _hash_text, and if the key is missing, the compute function is invoked and the result stored. Subsequent calls with identical text hit the cache and skip the expensive computation, which is key in LLM pipelines where embedding calls are slow. Printing the timing confirms which path was taken.

Common mistakes

Using the raw text as the dict key, which wastes memory for large strings
Ignoring hash collisions — use a strong hash like SHA-256 and accept negligible risk
Not handling exceptions when the compute function fails, leaving partial state

Variations

Use `functools.lru_cache` on a function that returns the embedding for a single argument
Swap the dict for a SQLite or Redis store to persist embeddings across restarts

Real-world use cases

Avoiding repeated API calls to a paid embedding service when processing the same product descriptions in a batch job.
Caching user-query embeddings in a recommendation service so the same search terms don't hit the model twice.
Storing chunk embeddings during document indexing so re-indexing is skipped for unchanged text segments.

How to cache embeddings with a Python dict to avoid recomputation

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from AI & LLM integration patterns

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from AI & LLM integration patterns

Keep learning

Tutorials

Quizzes