Reference library

Database scaling & optimization

Indexing, connection pooling, read replicas, query tuning, and throughput-aware SQL.

6 matches

Geo shard by region in Python

Maps users to database shards based on geographic region with a deterministic hash fallback.

import json
from collections import defaultdict

REGION_SHARD_MAP = {
    "na": ["shard-01", "shard-02"],
    "eu": ["shard-03", "shard-04", "shard-05"],
    "ap": ["shard-06"],
    "sa": ["shard-07", "shard-08"],
}

# user_id -> region (mock lookup)
USER_REGIONS = {
    "u_1001": "na",
    "u_1002": "eu",
    "u_1003…

13 0 Open

Database scaling & optimization easy

How to Batch Load JSON Data in Python for Database Optimization

This code parses JSON data into records and loads them in batches to simulate efficient database insertion, reducing load and improving performance.

json batching database

Python

import json
import time

def parse_and_load(data, batch_size=100):
    """
    Parse JSON data and batch-load into a list of dicts.
    Demonstrates batching for database efficiency.
    """
    records = json.loads(data)
    batches = []

    for i in range(0, len(records), batch_size):
        batch = records[i:i + …

12 0 Open

Database scaling & optimization easy

How to Mock Date Sharding by Range in Python

Split a date interval into fixed-size contiguous shards, returning each window as an ISO date string pair.

date datetime sharding

Python

from datetime import date, timedelta

def shard_ranges(start_date, end_date, shard_days=7):
    if start_date > end_date:
        raise ValueError("start_date cannot be after end_date")

    shards = []
    current = start_date
    while current <= end_date:
        shard_end = min(current + timedelta(days=shard_days …

13 0 Open

Database scaling & optimization easy

How to Replicate Data Across All Shards in Python

Mocks a global table that replicates a key-value pair to every shard, ensuring reads return the same value from any shard.

sharding replication distributed systems

Python

from dataclasses import dataclass
from typing import Dict, List


@dataclass
class Shard:
    id: str
    data: Dict[str, int]


class GlobalTable:
    def __init__(self, shards: List[Shard]):
        self._shards = {s.id: s for s in shards}

    def set_value(self, key: str, value: int) -> None:
        """Replicate …

14 0 Open

Database scaling & optimization easy

How to mock directory-based sharding in Python

Simulates distributing files into logical shards using a deterministic hash of each filename, mocking how a database might shard rows across nodes.

sharding hash partitioning

Python

import os
import hashlib
from collections import defaultdict
from pathlib import Path


def get_shard_for_key(key: str, num_shards: int) -> int:
    """Return a deterministic shard index (0..num_shards-1) for a key."""
    digest = hashlib.md5(key.encode('utf-8')).hexdigest()
    return int(digest, 16) % num_shards


…

14 0 Open

Database scaling & optimization easy

Route SELECT Queries to Read Replicas in Python

A mock round-robin router that forwards SELECT queries to read replicas and sends writes to the primary.

database replication routing

Python

import random

class ReadReplicaRouter:
    """Round-robin router that sends SELECT queries to read replicas."""
    
    def __init__(self, replicas):
        self.replicas = replicas
        self.counter = 0
    
    def route(self, sql):
        if sql.strip().upper().startswith("SELECT"):
            replica = sel…

13 0 Open

Browse by section

Each section groups closely related Python snippets.