How to Shard Data by User ID Hash in Python

Deterministically map user IDs to shard indexes using an MD5 hash modulo the shard count in Python.

Easy Python 3.9+ Aug 9, 2026 Database scaling & optimization 12 views 0 copies

sharding hashing md5 scalability databases

Python code

11 lines

Python 3.9+

import hashlib

def shard_id(user_id: str, num_shards: int = 4) -> int:
    """Deterministically map a user_id to a shard index using MD5."""
    digest = hashlib.md5(user_id.encode("utf-8")).hexdigest()
    return int(digest[:8], 16) % num_shards

if __name__ == "__main__":
    user_ids = ["alice", "bob", "carol", "dave", "eve"]
    for uid in user_ids:
        print(f"{uid:8} -> shard {shard_id(uid)}")

Output

stdout

alice    -> shard 3
bob      -> shard 0
carol    -> shard 1
dave     -> shard 0
eve      -> shard 0

How it works

The hashlib.md5 call creates a deterministic hash of the user ID string, and taking the first 8 hex characters as an integer gives a stable numeric value. The modulo operation with num_shards maps that value evenly across shard indexes from 0 to num_shards - 1. Because MD5 is a hash function, the distribution of user IDs across shards is roughly uniform, which helps balance load. The int(digest[:8], 16) conversion avoids floating-point errors and keeps the result reproducible across runs and machines.

Common mistakes

Using Python's built-in `hash()` which is salted and not stable across processes
Not encoding the string to UTF-8 before calling `hashlib.md5`, raising a TypeError
Forgetting that changing `num_shards` later will reshuffle all existing shard assignments

Variations

Use `hashlib.sha256` instead of MD5 for a stronger hash with the same pattern
Return the shard name (e.g., `db-shard-0`) by combining the index with a prefix

Real-world use cases

Routing users to different database partitions in a multi-tenant SaaS application.
Distributing messages to Kafka partitions by user key to preserve ordering per user.
Assigning cache keys to Redis cluster slots for even load distribution.

How to Shard Data by User ID Hash in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Database scaling & optimization

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Database scaling & optimization

Keep learning

Tutorials

Quizzes