How to Mock Date Sharding by Range in Python
Split a date interval into fixed-size contiguous shards, returning each window as an ISO date string pair.
Python code
20 linesfrom datetime import date, timedelta
def shard_ranges(start_date, end_date, shard_days=7):
if start_date > end_date:
raise ValueError("start_date cannot be after end_date")
shards = []
current = start_date
while current <= end_date:
shard_end = min(current + timedelta(days=shard_days - 1), end_date)
shards.append((current.isoformat(), shard_end.isoformat()))
current = shard_end + timedelta(days=1)
return shards
if __name__ == "__main__":
begin = date(2024, 1, 1)
end = date(2024, 1, 20)
for start, stop in shard_ranges(begin, end, 7):
print(f"{start} -> {stop}")
Output
2024-01-01 -> 2024-01-07
2024-01-08 -> 2024-01-14
2024-01-15 -> 2024-01-20
How it works
The loop advances current past each shard's end by one day, so the next shard starts exactly where the previous ended without overlapping or leaving gaps. shard_days - 1 is used with timedelta so each shard contains exactly shard_days dates, and min(..., end_date) clamps the final window to the requested range. Storing ISO strings makes the output directly usable for query parameters or partition keys. The function raises ValueError early to avoid infinite loops on inverted ranges.
Common mistakes
- Using `timedelta(days=shard_days)` instead of `shard_days - 1`, which adds an extra day per window
- Forgetting to increment `current` past `shard_end`, causing an infinite loop
- Assuming the last shard always has the same length instead of clamping to `end_date`
- Mutating `start_date` or `end_date` when calling the function
Variations
- Use a generator with `yield (current, shard_end)` to avoid building the full list in memory
- Adjust boundaries to be exclusive (`[start, end)`) if your partition scheme expects non-overlapping half-open intervals
Real-world use cases
- Generating time-partitioned queries for a database when a large date range needs splitting into smaller batches.
- Mocking shard iteration logic in tests before deploying a real distributed storage layer.
- Building CSV export jobs that process one week of data at a time to keep memory usage bounded.
Sponsored
More from Database scaling & optimization
- Approximate Count with HyperLogLog in Python medium
- B-Tree Insert and In-Order Traversal in Python hard
- Broadcast a Small Reference Table in Python easy
- Build a Full Text Search Index in Python medium
- Build a Partial Index Mock in Python for Database Filtering easy
- Composite index leftmost prefix in Python medium
Keep learning
Related tutorials and quizzes for this topic.