Rebalance Shard Ranges Across Nodes in Python

A mock rebalancing function that shuffles shard ranges and distributes them evenly across nodes using round-robin assignment.

Easy Python 3.9+ Aug 9, 2026 Database scaling & optimization 11 views 0 copies

Python code

37 lines
Python 3.9+
import random
from dataclasses import dataclass

@dataclass
class Shard:
    id: int
    start: int
    end: int

def rebalance_shards(shards: list[Shard], node_count: int) -> dict[int, list[Shard]]:
    """Mock rebalancing of shard ranges across nodes."""
    all_ranges = [(s.start, s.end) for s in shards]
    random.shuffle(all_ranges)
    
    assignments = {i: [] for i in range(node_count)}
    for idx, (start, end) in enumerate(all_ranges):
        node_id = idx % node_count
        assignments[node_id].append(Shard(idx, start, end))
    
    return assignments

if __name__ == "__main__":
    shards = [
        Shard(0, 1, 100),
        Shard(1, 101, 200),
        Shard(2, 201, 300),
        Shard(3, 301, 400),
        Shard(4, 401, 500),
        Shard(5, 501, 600),
    ]
    
    result = rebalance_shards(shards, node_count=3)
    
    for node, shard_list in sorted(result.items()):
        print(f"Node {node}:")
        for shard in shard_list:
            print(f"  Shard {shard.id}: [{shard.start}-{shard.end}]")

Output

stdout
Node 0:
  Shard 0: [1-100]
  Shard 1: [101-200]
Node 1:
  Shard 2: [201-300]
  Shard 3: [301-400]
Node 2:
  Shard 4: [401-500]
  Shard 5: [501-600]

How it works

The function first extracts all start and end ranges into a list and shuffles them to randomize assignment. Then it iterates over the shuffled ranges and assigns each to a node using modulo arithmetic (idx % node_count), which evenly distributes ranges across nodes. The dataclass Shard holds the shard ID and its range boundaries. This simple round-robin approach ensures balanced distribution when the number of shards is a multiple of the node count, though it does not consider load or range size. The main block creates six shards and assigns them to three nodes, printing the assignments in a readable format.

Common mistakes

  • Assuming the output is deterministic; using random.shuffle without a seed means order varies each run.
  • Forgetting to handle uneven shard-to-node ratios, which can cause one node to get more shards.
  • Modifying the original shards list instead of working with a copy of ranges.
  • Not accounting for range overlap or duplicate ranges when rebalancing.

Variations

  1. Use a deterministic shuffling by passing a random seed or using a stable sort by shard size.
  2. Assign shards based on load or range size rather than round-robin to balance actual data volume.

Real-world use cases

  • Rebalancing data partitions across a distributed database cluster when nodes are added or removed.
  • Evenly distributing cache shards across multiple Redis instances in a cache-aside architecture.
  • Initializing a consistent hashing ring for key-range splits in a NoSQL system like Cassandra.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Database scaling & optimization

Related tutorials and quizzes for this topic.