Reference library

AI & LLM integration patterns

Call LLM APIs, structure prompts, parse responses, and ship AI features safely.

4 matches

How to Batch Embed a List of Strings in Python

Batch embed a list of strings into deterministic pseudo-random vectors using a mock encoder class.

class MockEncoder:
    def __init__(self, dim=8, seed=42):
        self.dim = dim
        self.seed = seed

    def embed(self, text):
        # Deterministic pseudo-random embedding based on text content
        hash_val = hash(text)
        import random
        rng = random.Random(hash_val + self.seed)
        retu…

13 0 Open

AI & LLM integration patterns easy

How to Chunk a Long Document for RAG Retrieval in Python

Split text into overlapping chunks at sentence boundaries using a custom Python function suitable for RAG retrieval pipelines.

rag text-chunking nlp

Python

import re
from pathlib import Path

def chunk_document(text, chunk_size=500, overlap=100):
    """Split text into overlapping chunks suitable for RAG retrieval."""
    # Normalize whitespace
    text = re.sub(r'\s+', ' ', text).strip()
    
    chunks = []
    start = 0
    while start < len(text):
        end = min(s…

15 0 Open

AI & LLM integration patterns easy

How to Estimate Token Count in Python

Estimates tokens in a text string using a whitespace and punctuation heuristic without external libraries.

token-count llm heuristic

Python

def estimate_tokens(text: str) -> int:
    """Estimate token count using whitespace and punctuation heuristics."""
    if not text:
        return 0

    words = text.split()
    total_punctuation = sum(1 for char in text if char in ".,!?;:")
    special_tokens = sum(1 for char in text if char in "\n\t")

    # Rough …

12 0 Open

AI & LLM integration patterns easy

How to Redact Emails and Phones Before Sending to an LLM in Python

This code uses regular expressions to replace email addresses and US phone numbers with [EMAIL] and [PHONE] placeholders before any LLM processing.

pii redaction regular-expressions

Python

import re

def redact_pii(text: str) -> str:
    # Replace email addresses with [EMAIL]
    text = re.sub(r'[\w.+-]+@[\w-]+\.[\w.-]+', '[EMAIL]', text)
    # Replace phone numbers (US format) with [PHONE]
    text = re.sub(r'\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}', '[PHONE]', text)
    return text

if __name__ == "__main…

15 0 Open

Browse by section

Each section groups closely related Python snippets.