How to Order Partition Key Events in Python (Mock Stream)
Generate a mock event stream grouped by partition key and sort it deterministically by key then sequence in Python.
Python code
43 linesimport itertools
import random
def partition_key_events(keys, events_per_key=3, seed=None):
"""Produce a realistic-looking, but mock, event stream grouped by partition key.
Args:
keys: iterable of partition keys (e.g. strings or ints).
events_per_key: how many events we want per key.
seed: optional RNG seed for repeatable output.
Returns:
List of (partition_key, sequence, event_id) tuples.
"""
if seed is not None:
random.seed(seed)
events = []
for key in keys:
for seq in range(1, events_per_key + 1):
event_id = random.randint(1_000_000, 9_999_999)
events.append((key, seq, event_id))
return events
def order_partition_events(events):
"""Sort events deterministically: partition key first, then sequence."""
return sorted(events, key=lambda ev: (str(ev[0]), ev[1]))
if __name__ == "__main__":
mock_stream = partition_key_events(["user-a", "user-b", "user-c"], events_per_key=2, seed=42)
print("Original (shuffled) stream:")
random.shuffle(mock_stream)
for e in mock_stream:
print(e)
print("\nOrdered by partition key (then sequence):")
ordered = order_partition_events(mock_stream)
for e in ordered:
print(e)
Output
Original (shuffled) stream:
('user-a', 1, 1049906)
('user-c', 2, 8665766)
('user-b', 1, 2623710)
('user-c', 1, 4222659)
('user-b', 2, 6064335)
('user-a', 2, 6535903)
Ordered by partition key (then sequence):
('user-a', 1, 1049906)
('user-a', 2, 6535903)
('user-b', 1, 2623710)
('user-b', 2, 6064335)
('user-c', 1, 4222659)
('user-c', 2, 8665766)
How it works
The partition_key_events function generates a list of tuples using nested loops, assigning a random event_id via random.randint. The order_partition_events function uses sorted with a key that extracts the partition key as a string and the sequence number as an integer, ensuring deterministic ordering even with mixed key types. This pattern mirrors how real event streams are often sorted before processing to maintain per-key ordering, which is critical in systems like Kafka where each partition preserves order but interleaving across partitions is common.
Common mistakes
- Forgetting to convert partition keys to a common type (e.g., str) in the sort key, leading to TypeError when mixing ints and strings.
- Shuffling the list in-place with `random.shuffle` without copying, which mutates the original and can cause unintended side effects.
- Assuming the event_id itself is meaningful for ordering; it's random and should not be used unless specifically designed for ordering.
Variations
- Use `itertools.product` to generate all key/sequence combinations more compactly.
- Sort using `operator.itemgetter` for slightly faster performance on large streams.
Real-world use cases
- Simulating user event streams for load testing Kafka consumers where per-key ordering matters.
- Generating mock data to validate event deduplication logic in stream processing pipelines.
- Creating deterministic test fixtures for debugging ordering bugs in microservices that consume partitioned events.
Sponsored
More from Microservices patterns
- BFF aggregation pattern: combine multiple service responses in Python easy
- Backward Compatible Schema Evolution in Python medium
- Bulkhead Thread Pool per Service Mock in Python medium
- CQRS with Separate Read and Write Repositories in Python medium
- Cache-Aside Pattern in Python: Per-Service Mock easy
- Consumer Driven Contract Pact Mock in Python medium
Keep learning
Related tutorials and quizzes for this topic.