Reference library

Database scaling & optimization

Indexing, connection pooling, read replicas, query tuning, and throughput-aware SQL.

6 matches

Broadcast a Small Reference Table in Python

Simulates SQL-style broadcasting of a small lookup table against a larger fact table in memory for mockups or load tests.

broadcast mock-data data-engineering

Python

import random

def broadcast_mock(target, source, columns):
    result = {}
    for col in columns:
        if col in target and col in source:
            result[col] = target[col] + [source[col][i % len(source[col])] for i in range(len(target[col]))]
        elif col in target:
            result[col] = target[col]
…

15 0 Open

Database scaling & optimization easy

Geo shard by region in Python

Maps users to database shards based on geographic region with a deterministic hash fallback.

sharding geolocation database

Python

import json
from collections import defaultdict

REGION_SHARD_MAP = {
    "na": ["shard-01", "shard-02"],
    "eu": ["shard-03", "shard-04", "shard-05"],
    "ap": ["shard-06"],
    "sa": ["shard-07", "shard-08"],
}

# user_id -> region (mock lookup)
USER_REGIONS = {
    "u_1001": "na",
    "u_1002": "eu",
    "u_1003…

13 0 Open

Database scaling & optimization medium

How to Eager Load with JOIN to Reduce N+1 Queries in Python

Demonstrates eager loading with a SQL JOIN to reduce N+1 query patterns down to a single database call when fetching related data.

eager-loading n-plus-1 join

Python

import sqlite3


def eager_load_join_reduce(mock_db_path=":memory:"):
    """Demonstrate eager loading where joins reduce query count from N+1 to 1."""
    conn = sqlite3.connect(mock_db_path)
    cursor = conn.cursor()
    cursor.executescript(
        """
        CREATE TABLE authors (id INTEGER PRIMARY KEY, name TE…

16 0 Open

Database scaling & optimization easy

How to Mock a Function Call in Python with unittest.mock

Use unittest.mock.Mock to wrap a function and spy on its call count and arguments in Python.

mock unittest spy

Python

import random
from unittest.mock import Mock, patch


def select_n_plus_one(numbers: list[int]) -> int:
    """Return the first number that appears more than once, if any."""
    seen = set()
    for num in numbers:
        if num in seen:
            return num
        seen.add(num)
    return -1


def detect_mock(se…

14 0 Open

Database scaling & optimization easy

How to Replicate Data Across All Shards in Python

Mocks a global table that replicates a key-value pair to every shard, ensuring reads return the same value from any shard.

sharding replication distributed systems

Python

from dataclasses import dataclass
from typing import Dict, List


@dataclass
class Shard:
    id: str
    data: Dict[str, int]


class GlobalTable:
    def __init__(self, shards: List[Shard]):
        self._shards = {s.id: s for s in shards}

    def set_value(self, key: str, value: int) -> None:
        """Replicate …

14 0 Open

Database scaling & optimization easy

How to Shard Data by User ID Hash in Python

Deterministically map user IDs to shard indexes using an MD5 hash modulo the shard count in Python.

sharding hashing md5

Python

import hashlib

def shard_id(user_id: str, num_shards: int = 4) -> int:
    """Deterministically map a user_id to a shard index using MD5."""
    digest = hashlib.md5(user_id.encode("utf-8")).hexdigest()
    return int(digest[:8], 16) % num_shards

if __name__ == "__main__":
    user_ids = ["alice", "bob", "carol", "d…

12 0 Open

Browse by section

Each section groups closely related Python snippets.