Database scaling & optimization
Indexing, connection pooling, read replicas, query tuning, and throughput-aware SQL.
Cross Shard Query Scatter Gather Mock in Python
Simulate a distributed database cross-shard query using a scatter-gather pattern with a mock Python implementation.
from dataclasses import dataclass
from typing import List, Dict
@dataclass
class NodeResponse:
node_id: int
data: Dict[str, float]
def mock_query_shard(shard_id: int, shard_data: Dict[str, float], query: str) -> NodeResponse:
"""Simulate querying a single shard, returning matches whose value > 50."""
…Database Helper in Python with SQLite Scaling Optimization
Build a beginner-friendly SQLite database helper class with WAL, indexed queries, and efficient batch inserts for scaling.
import sqlite3
from contextlib import contextmanager
class DatabaseHelper:
"""Beginner-friendly helper for SQLite database operations with scaling tips."""
def __init__(self, db_path):
self.db_path = db_path
@contextmanager
def connection(self):
"""Context manager for automatic comm…Hash index equality mock concept in Python
A simple hash index class in Python that stores key-value pairs in buckets and demonstrates basic equality-based lookup.
class HashIndex:
def __init__(self):
self._buckets = {}
def insert(self, key, value):
"""Insert a key-value pair into the hash index."""
index = hash(key) % 10
if index not in self._buckets:
self._buckets[index] = []
self._buckets[index].append((key, value))…How to Build a Shard Map Mock Dict in Python
Implement a dictionary-like class that distributes keys across multiple shards using Python's hash() for realistic data partitioning.
class ShardMap:
def __init__(self, shard_count):
self.shards = {i: {} for i in range(shard_count)}
self.shard_count = shard_count
def _shard_for(self, key):
return hash(key) % self.shard_count
def __getitem__(self, key):
return self.shards[self._shard_for(key)][key]
d…How to Create a Data Helper Class in Python for JSON Files
Build a beginner-friendly Python helper class to read, write, filter, and summarize JSON data files with clean, reusable methods.
import json
from pathlib import Path
class DataHelper:
"""Simple beginner-friendly helper for reading and writing JSON data files."""
@staticmethod
def read_json(filename):
file_path = Path(filename)
if file_path.exists():
with file_path.open("r", encoding="utf-8") as f:
…How to Create a Database Helper Class for Beginners in Python
Build a beginner-friendly SQLite helper class with indexing and batch inserts to optimize database queries in Python.
import sqlite3
class DatabaseHelper:
def __init__(self, db_path):
self.connection = sqlite3.connect(db_path)
self.cursor = self.connection.cursor()
def create_table_with_index(self, table_name, columns, indexed_column):
columns_sql = ", ".join(f"{name} {dtype}" for name, dtype in col…How to Mock Replica Lag Monitoring in Python
Simulates database replica lag with a mock monitor class that generates realistic lag metrics and health statuses.
import time
import random
from datetime import datetime, timedelta
class MockReplicaLagMonitor:
def __init__(self, replicas=3, base_lag=0.5, jitter=0.2):
self.replicas = [f"replica-{i}" for i in range(replicas)]
self.base_lag = base_lag
self.jitter = jitter
self.last_write = dateti…How to Mock a Cross-Shard Saga in Python
Simulate a distributed saga with compensating transactions across multiple database shards using a lightweight Python class that tracks executed steps and rolls them back in reverse on failure.
import json
class SagaState:
def __init__(self, saga_id):
self.saga_id = saga_id
self.executed_steps = []
self.compensations = []
def execute_step(self, shard, step_name, operation):
self.executed_steps.append((shard, step_name))
print(f"[Saga {self.saga_id}] Executin…How to Mock a Hot Shard Split in Python
Simulate a database hot shard splitting into two shards by key ranges when it exceeds a threshold, with a mock class for testing.
import random
from collections import defaultdict
class HotShardMock:
"""Mock implementation of a hot shard split in a distributed database."""
def __init__(self, shard_id="shard_1", max_entries=5):
self.shard_id = shard_id
self.max_entries = max_entries
self.entries = {}
def ad…How to Replicate Data Across All Shards in Python
Mocks a global table that replicates a key-value pair to every shard, ensuring reads return the same value from any shard.
from dataclasses import dataclass
from typing import Dict, List
@dataclass
class Shard:
id: str
data: Dict[str, int]
class GlobalTable:
def __init__(self, shards: List[Shard]):
self._shards = {s.id: s for s in shards}
def set_value(self, key: str, value: int) -> None:
"""Replicate …How to Simulate Distributed Transactions in Python with a Mock
Model distributed transaction behavior with a mock Transaction class that supports commit, rollback, and failure simulation.
class Transaction:
def __init__(self, id):
self.id = id
self.operations = []
self.committed = False
def add_operation(self, op, data):
self.operations.append((op, data))
def commit(self):
if not self.operations:
raise ValueError("No operations to commit…Mock CQRS Read/Write Split in Python
Separate order mutations from queries using a read model and write model to mock CQRS-style separation of concerns.
from dataclasses import dataclass, field
from typing import List, Dict
@dataclass
class Order:
id: int
amount: float
status: str = "pending"
class OrderWriteModel:
"""Handles all mutations (writes) to orders."""
def __init__(self):
self._orders: Dict[int, Order] = {}
self._next…Rebalance Shard Ranges Across Nodes in Python
A mock rebalancing function that shuffles shard ranges and distributes them evenly across nodes using round-robin assignment.
import random
from dataclasses import dataclass
@dataclass
class Shard:
id: int
start: int
end: int
def rebalance_shards(shards: list[Shard], node_count: int) -> dict[int, list[Shard]]:
"""Mock rebalancing of shard ranges across nodes."""
all_ranges = [(s.start, s.end) for s in shards]
random…Simulate PostgreSQL Vacuum to Reclaim Space in Python
A Python class that safely rewrites a data file to remove deleted rows and reclaim physical space, mimicking PostgreSQL's VACUUM operation.
import shutil
import os
class VacuumCleaner:
"""Simulates PostgreSQL-style vacuum reclaiming dead space in a file."""
def __init__(self, filepath, fill_ratio=0.7, dead_marker="[DELETED]"):
self.filepath = filepath
self.fill_ratio = fill_ratio
self.dead_marker = dead_marker
…Browse by section
Each section groups closely related Python snippets.
Database scaling & optimization — Python code examples
What you will find here
This page collects database scaling & optimization snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.