Reference library

AI & LLM integration patterns

Call LLM APIs, structure prompts, parse responses, and ship AI features safely.

3 matches

Cache LLM Completions by Hashing the Prompt in Python

A simple in-memory cache that stores LLM completions keyed by a SHA-256 hash of the prompt to avoid recomputing identical requests.

llm caching hashing

Python

import hashlib
import json

class PromptCache:
    def __init__(self):
        self.cache = {}

    def _hash_prompt(self, prompt: str) -> str:
        return hashlib.sha256(prompt.encode("utf-8")).hexdigest()

    def get(self, prompt: str) -> str | None:
        key = self._hash_prompt(prompt)
        return self.ca…

14 0 Open

AI & LLM integration patterns easy

How to Build a Simple Semantic Cache for Similar Prompts in Python

Mock a semantic cache that finds the closest matching prompt using word-overlap similarity and returns cached results above a threshold.

semantic cache prompt matching llm

Python

prompt_cache = [
    "What is the capital of France?",
    "How does recursion work?",
    "Best practices for Python logging?",
    "Explain binary search in one line.",
    "How to reverse a string in Python?"
]

def normalize(text):
    return " ".join(text.lower().split())

def similarity(a, b):
    a_words = set(…

14 0 Open

AI & LLM integration patterns medium

How to cache embeddings with a Python dict to avoid recomputation

Caches embeddings computed from text in a dictionary keyed by SHA-256 hash, returning cached results for repeated calls.

embedding cache dict

Python

import hashlib
import time


class EmbeddingCache:
    def __init__(self):
        self.cache = {}

    def _hash_text(self, text):
        return hashlib.sha256(text.encode()).hexdigest()

    def get_embedding(self, text, compute_func):
        key = self._hash_text(text)
        if key not in self.cache:
          …

15 0 Open

Browse by section

Each section groups closely related Python snippets.