AI & LLM integration patterns
Call LLM APIs, structure prompts, parse responses, and ship AI features safely.
Circuit Breaker Pattern in Python for LLM API Calls
Implements a circuit breaker class that wraps LLM client calls to fail fast when the service is degrading, then recover automatically after a timeout.
import time
class CircuitBreaker:
def __init__(self, failure_threshold=3, recovery_timeout=5):
self.failure_threshold = failure_threshold
self.recovery_timeout = recovery_timeout
self.failure_count = 0
self.state = "closed"
self.last_failure_time = None
def call(self, …How to Build a Prompt Template with Variable Slots in Python
Create a reusable LLM prompt template with named variable slots using Python's string.Template class and fill them with render() calls.
from string import Template
class PromptTemplate:
def __init__(self, template_text):
self.template = Template(template_text)
def render(self, **kwargs):
return self.template.substitute(**kwargs)
if __name__ == "__main__":
template = PromptTemplate(
"You are a helpful assistant …How to Build an Entity Memory Dict to Store Facts in Python
Store and recall facts about entities using nested dictionaries with remember, recall, and forget functions in Python.
facts = {}
def remember(entity, attribute, value):
if entity not in facts:
facts[entity] = {}
facts[entity][attribute] = value
def recall(entity, attribute):
return facts.get(entity, {}).get(attribute, None)
def forget(entity, attribute=None):
if attribute is None:
facts.pop(entity, …How to Mock OpenAI Tool Call Messages in Python
Create an assistant message with a function tool call in OpenAI's chat format, useful for testing and mocking.
from openai import OpenAI
def mock_tool_call(tool_name: str, arguments: dict) -> dict:
"""Simulate a tool call message in OpenAI style."""
return {
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_" + "a1b2c3d4e5f6",
"type…How to Repair Malformed JSON Braces Heuristically in Python
Heuristically fix malformed JSON by balancing braces and quotes, using a stack-based approach to add missing closing characters.
import json
import re
def repair_json(text: str) -> str:
"""Heuristically repair malformed JSON by balancing braces and quotes."""
# Trim whitespace and handle leading/trailing garbage
text = text.strip()
# Remove common non-JSON decorations
text = re.sub(r'^(How to Retry LLM Calls on Rate Limit Errors in Python
Implement a retry mechanism with exponential backoff for LLM API calls that raises a custom RateLimitError, using a mock function to demonstrate the pattern.
import time
import random
def mock_llm_call():
"""Simulates an LLM API call that may raise a rate limit error."""
if random.random() < 0.4: # 40% chance of rate limit
raise RateLimitError("Rate limit exceeded. Try again later.")
return {"response": "Hello world from mock LLM"}
class RateLimitE…How to build a function calling schema dict in Python
Build an OpenAI-compatible function calling schema dictionary with a helper function that takes name, description, parameters, and required fields.
import json
from typing import Dict, Any, List, Optional
def build_function_schema(
name: str,
description: str,
parameters: Optional[Dict[str, Any]] = None,
required: Optional[List[str]] = None
) -> Dict[str, Any]:
"""Build an OpenAI-compatible function calling schema dictionary."""
schema: …How to cache embeddings with a Python dict to avoid recomputation
Caches embeddings computed from text in a dictionary keyed by SHA-256 hash, returning cached results for repeated calls.
import hashlib
import time
class EmbeddingCache:
def __init__(self):
self.cache = {}
def _hash_text(self, text):
return hashlib.sha256(text.encode()).hexdigest()
def get_embedding(self, text, compute_func):
key = self._hash_text(text)
if key not in self.cache:
…How to compute ROUGE recall in Python
Compute ROUGE recall by counting token overlap between a reference and candidate summary with pure Python.
def rouge_recall(reference, candidate):
ref_tokens = reference.lower().split()
cand_tokens = candidate.lower().split()
ref_counts = {}
for token in ref_tokens:
ref_counts[token] = ref_counts.get(token, 0) + 1
cand_counts = {}
for token in cand_tokens:
cand_counts[token] = cand…How to implement exponential backoff for LLM API calls in Python
A decorator that retries flaky LLM API calls with exponential delay, using a mock client to demonstrate the pattern.
import time
import random
class MockLLM:
def call(self, prompt):
if random.random() < 0.7: # 70% chance of transient failure
raise ConnectionError("API unavailable")
return f"LLM response for: {prompt}"
def with_exponential_backoff(max_retries=5, base_delay=0.1):
def decorator(fu…How to parallel map embeddings with a thread pool in Python
Run embedding computations in parallel using ThreadPoolExecutor, collect results into a dict keyed by the original item.
import threading
from concurrent.futures import ThreadPoolExecutor
import time
def compute_embedding(item: int) -> tuple[int, int]:
time.sleep(0.05) # Simulate embedding work
return item, item * 10
def parallel_map_embed(items, max_workers=3):
results = {}
with ThreadPoolExecutor(max_workers=max_w…Route Tool Call Name to Python Handler Dict
Routes a tool call name to the correct Python handler function using a dictionary lookup, returning an error for unknown tools.
def get_name():
return {"name": "Alice"}
def get_age():
return {"age": 30}
def get_email():
return {"email": "alice@example.com"}
handlers = {
"get_name": get_name,
"get_age": get_age,
"get_email": get_email,
}
def route(tool_call):
handler = handlers.get(tool_call["name"])
if handl…Serialize and Format Data for LLM Prompts in Python
Use dataclasses and the json module to convert Python objects to JSON strings, parse them back, and format structured data into prompt-friendly text for LLM calls.
import json
from dataclasses import dataclass, asdict
@dataclass
class Recipe:
"""Simple data model to represent a recipe."""
name: str
cuisine: str
prep_minutes: int
def to_json(recipe: Recipe) -> str:
"""Serialize a Recipe to a JSON string."""
return json.dumps(asdict(recipe), indent=2)
…Browse by section
Each section groups closely related Python snippets.
AI & LLM integration patterns — Python code examples
What you will find here
This page collects ai & llm integration patterns snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.