Reference library

AI & LLM integration patterns

Call LLM APIs, structure prompts, parse responses, and ship AI features safely.

4 matches

Circuit Breaker Pattern in Python for LLM API Calls

Implements a circuit breaker class that wraps LLM client calls to fail fast when the service is degrading, then recover automatically after a timeout.

circuit-breaker llm resilience

Python

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=3, recovery_timeout=5):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failure_count = 0
        self.state = "closed"
        self.last_failure_time = None

    def call(self, …

15 0 Open

AI & LLM integration patterns easy

How to Build an In-Memory Vector Store in Python

Build a lightweight in-memory vector store using a Python dict and cosine similarity for fast nearest-neighbor searches.

vector-store cosine-similarity embeddings

Python

import math
from typing import Dict, List, Optional


class InMemoryVectorStore:
    def __init__(self) -> None:
        self.vectors: Dict[str, List[float]] = {}
        self.index: Dict[str, List[str]] = {}  # query -> list of ids sorted by similarity

    def add(self, vector_id: str, vector: List[float]) -> None:
…

12 0 Open

AI & LLM integration patterns easy

How to Filter Toxic Keywords in Python

Filter toxic keywords from text by replacing each occurrence with asterisks, useful as a basic guardrail for LLM inputs.

guardrails text-filtering llm-safety

Python

TOXIC_KEYWORDS = ["insult", "threat", "hate", "violence", "spam"]


def guardrails_filter(text: str, keywords: list[str] | None = None) -> str:
    """Filter out toxic keywords from the given text.

    Args:
        text: The input text to filter.
        keywords: Optional keyword list. Defaults to TOXIC_KEYWORDS.

…

12 0 Open

AI & LLM integration patterns easy

How to Keep Last K Turns in a Memory Buffer in Python

A TurnBuffer class using deque with maxlen to keep only the most recent k conversation turns in memory for LLM context.

deque llm-context memory-buffer

Python

from collections import deque

class TurnBuffer:
    def __init__(self, k):
        self.k = k
        self.turns = deque(maxlen=k)

    def add(self, turn):
        self.turns.append(turn)

    def last_k(self):
        return list(self.turns)


if __name__ == "__main__":
    buffer = TurnBuffer(3)
    buffer.add("tu…

14 0 Open

Browse by section

Each section groups closely related Python snippets.