Reference library

Database scaling & optimization

Indexing, connection pooling, read replicas, query tuning, and throughput-aware SQL.

4 matches

How to Replicate Data Across All Shards in Python

Mocks a global table that replicates a key-value pair to every shard, ensuring reads return the same value from any shard.

sharding replication distributed systems

Python

from dataclasses import dataclass
from typing import Dict, List


@dataclass
class Shard:
    id: str
    data: Dict[str, int]


class GlobalTable:
    def __init__(self, shards: List[Shard]):
        self._shards = {s.id: s for s in shards}

    def set_value(self, key: str, value: int) -> None:
        """Replicate …

14 0 Open

Database scaling & optimization easy

How to mock directory-based sharding in Python

Simulates distributing files into logical shards using a deterministic hash of each filename, mocking how a database might shard rows across nodes.

sharding hash partitioning

Python

import os
import hashlib
from collections import defaultdict
from pathlib import Path


def get_shard_for_key(key: str, num_shards: int) -> int:
    """Return a deterministic shard index (0..num_shards-1) for a key."""
    digest = hashlib.md5(key.encode('utf-8')).hexdigest()
    return int(digest, 16) % num_shards


…

14 0 Open

Database scaling & optimization easy

Rebalance Shard Ranges Across Nodes in Python

A mock rebalancing function that shuffles shard ranges and distributes them evenly across nodes using round-robin assignment.

sharding rebalancing dataclass

Python

import random
from dataclasses import dataclass

@dataclass
class Shard:
    id: int
    start: int
    end: int

def rebalance_shards(shards: list[Shard], node_count: int) -> dict[int, list[Shard]]:
    """Mock rebalancing of shard ranges across nodes."""
    all_ranges = [(s.start, s.end) for s in shards]
    random…

11 0 Open

Database scaling & optimization easy

Simulate PostgreSQL Vacuum to Reclaim Space in Python

A Python class that safely rewrites a data file to remove deleted rows and reclaim physical space, mimicking PostgreSQL's VACUUM operation.

vacuum file-management database

Python

import shutil
import os

class VacuumCleaner:
    """Simulates PostgreSQL-style vacuum reclaiming dead space in a file."""
    
    def __init__(self, filepath, fill_ratio=0.7, dead_marker="[DELETED]"):
        self.filepath = filepath
        self.fill_ratio = fill_ratio
        self.dead_marker = dead_marker
    
  …

13 0 Open

Browse by section

Each section groups closely related Python snippets.