Reference library

AI & LLM integration patterns

Call LLM APIs, structure prompts, parse responses, and ship AI features safely.

3 matches

Circuit Breaker Pattern in Python for LLM API Calls

Implements a circuit breaker class that wraps LLM client calls to fail fast when the service is degrading, then recover automatically after a timeout.

circuit-breaker llm resilience

Python

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=3, recovery_timeout=5):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failure_count = 0
        self.state = "closed"
        self.last_failure_time = None

    def call(self, …

14 0 Open

AI & LLM integration patterns easy

How to Mock an LLM Client in Python

Create a simple mock LLM client that returns a canned completion for testing or development without a real API.

llm mock testing

Python

from dataclasses import dataclass


@dataclass
class MockLLMClient:
    canned_response: str = "This is a canned completion."

    def complete(self, prompt: str) -> str:
        return f"{self.canned_response} [to: {prompt[:20]}]"


if __name__ == "__main__":
    client = MockLLMClient()
    result = client.complete(…

16 0 Open

AI & LLM integration patterns medium

How to implement exponential backoff for LLM API calls in Python

A decorator that retries flaky LLM API calls with exponential delay, using a mock client to demonstrate the pattern.

exponential-backoff retries llm

Python

import time
import random

class MockLLM:
    def call(self, prompt):
        if random.random() < 0.7:  # 70% chance of transient failure
            raise ConnectionError("API unavailable")
        return f"LLM response for: {prompt}"

def with_exponential_backoff(max_retries=5, base_delay=0.1):
    def decorator(fu…

14 0 Open

Browse by section

Each section groups closely related Python snippets.