Geo shard by region in Python
Maps users to database shards based on geographic region with a deterministic hash fallback.
Python code
36 linesimport json
from collections import defaultdict
REGION_SHARD_MAP = {
"na": ["shard-01", "shard-02"],
"eu": ["shard-03", "shard-04", "shard-05"],
"ap": ["shard-06"],
"sa": ["shard-07", "shard-08"],
}
# user_id -> region (mock lookup)
USER_REGIONS = {
"u_1001": "na",
"u_1002": "eu",
"u_1003": "ap",
"u_1004": "sa",
"u_1005": "na",
"u_1006": "eu",
}
def get_shard_for_user(user_id):
region = USER_REGIONS.get(user_id)
if not region:
return {"user_id": user_id, "region": None, "shard": None, "error": "unknown region"}
shards = REGION_SHARD_MAP.get(region, [])
if not shards:
return {"user_id": user_id, "region": region, "shard": None, "error": "no shards for region"}
# simple hash-based shard selection for determinism
shard_idx = sum(ord(c) for c in user_id) % len(shards)
return {"user_id": user_id, "region": region, "shard": shards[shard_idx]}
if __name__ == "__main__":
results = []
for uid in sorted(USER_REGIONS.keys()):
results.append(get_shard_for_user(uid))
print(json.dumps(results, indent=2))
Output
[
{
"user_id": "u_1001",
"region": "na",
"shard": "shard-01"
},
{
"user_id": "u_1002",
"region": "eu",
"shard": "shard-03"
},
{
"user_id": "u_1003",
"region": "ap",
"shard": "shard-06"
},
{
"user_id": "u_1004",
"region": "sa",
"shard": "shard-07"
},
{
"user_id": "u_1005",
"region": "na",
"shard": "shard-02"
},
{
"user_id": "u_1006",
"region": "eu",
"shard": "shard-05"
}
]
How it works
This code uses a simple dictionary lookup to associate each user with a geographic region, then maps that region to a list of available shards. A deterministic hash based on the user ID's character sum picks a shard index, ensuring the same user always goes to the same shard. The defaultdict import is unused but harmless; the real logic relies on dict.get for safe lookups. This pattern keeps shard selection stable and is easy to extend with new regions or shards.
Common mistakes
- Forgetting to handle users with no region or no shards, leading to KeyError
- Using a non-deterministic hash (like random) that sends users to different shards on each call
- Assuming all regions have the same number of shards, which can cause uneven load
- Not caching the shard map, causing repeated lookups in hot paths
Variations
- Use a consistent hashing library like `hashlib.md5` for a more uniform distribution
- Replace the in-memory dict with a Redis cache for dynamic region updates
Real-world use cases
- Routing user requests to region-specific database clusters in a multi-tenant SaaS platform.
- Partitioning event data by country for compliance with data residency regulations.
- Balancing read replicas geographically for low-latency access in a global application.
Sponsored
More from Database scaling & optimization
- Approximate Count with HyperLogLog in Python medium
- B-Tree Insert and In-Order Traversal in Python hard
- Broadcast a Small Reference Table in Python easy
- Build a Full Text Search Index in Python medium
- Build a Partial Index Mock in Python for Database Filtering easy
- Composite index leftmost prefix in Python medium
Keep learning
Related tutorials and quizzes for this topic.