Reference library

AI & LLM integration patterns

Call LLM APIs, structure prompts, parse responses, and ship AI features safely.

4 matches

Circuit Breaker Pattern in Python for LLM API Calls

Implements a circuit breaker class that wraps LLM client calls to fail fast when the service is degrading, then recover automatically after a timeout.

circuit-breaker llm resilience

Python

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=3, recovery_timeout=5):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failure_count = 0
        self.state = "closed"
        self.last_failure_time = None

    def call(self, …

14 0 Open

AI & LLM integration patterns easy

How to Log Prompts and Completions as JSONL Audit Files in Python

Read a JSONL file of LLM prompt–completion pairs, compute totals and averages, then write an audit summary with timestamps.

jsonl audit llm

Python

import json
from pathlib import Path
from datetime import datetime


def audit_jsonl(filepath):
    logs = []
    with open(filepath, encoding="utf-8") as f:
        for line in f:
            line = line.strip()
            if not line:
                continue
            entry = json.loads(line)
            logs.ap…

15 0 Open

AI & LLM integration patterns easy

How to Serialize Chat Messages to a JSON File in Python

Writes a list of chat message dicts to a JSON file with metadata like export time and message count.

json serialization chat

Python

import json
from pathlib import Path
from datetime import datetime

def serialize_messages(messages, output_path):
    data = {
        "exported_at": datetime.now().isoformat(),
        "count": len(messages),
        "messages": messages
    }
    Path(output_path).write_text(
        json.dumps(data, indent=2, ensu…

16 0 Open

AI & LLM integration patterns easy

How to Stream Tokens from a Mock LLM in Python

Simulate real-time LLM streaming by yielding tokens one at a time with a delay, making it easy to test streaming UIs.

generator llm streaming

Python

import time
from typing import Generator


def stream_tokens(text: str, delay: float = 0.05) -> Generator[str, None, None]:
    """Simulate an LLM streaming tokens word by word."""
    for word in text.split():
        yield word
        time.sleep(delay)


if __name__ == "__main__":
    sample = "Hello world! This is…

15 0 Open

Browse by section

Each section groups closely related Python snippets.