How to Order Partition Key Events in Python (Mock Stream)

Generate a mock event stream grouped by partition key and sort it deterministically by key then sequence in Python.

Easy Python 3.9+ Aug 9, 2026 Microservices patterns 11 views 0 copies

Python code

43 lines
Python 3.9+
import itertools
import random


def partition_key_events(keys, events_per_key=3, seed=None):
    """Produce a realistic-looking, but mock, event stream grouped by partition key.

    Args:
        keys: iterable of partition keys (e.g. strings or ints).
        events_per_key: how many events we want per key.
        seed: optional RNG seed for repeatable output.

    Returns:
        List of (partition_key, sequence, event_id) tuples.
    """
    if seed is not None:
        random.seed(seed)

    events = []
    for key in keys:
        for seq in range(1, events_per_key + 1):
            event_id = random.randint(1_000_000, 9_999_999)
            events.append((key, seq, event_id))
    return events


def order_partition_events(events):
    """Sort events deterministically: partition key first, then sequence."""

    return sorted(events, key=lambda ev: (str(ev[0]), ev[1]))


if __name__ == "__main__":
    mock_stream = partition_key_events(["user-a", "user-b", "user-c"], events_per_key=2, seed=42)
    print("Original (shuffled) stream:")
    random.shuffle(mock_stream)
    for e in mock_stream:
        print(e)

    print("\nOrdered by partition key (then sequence):")
    ordered = order_partition_events(mock_stream)
    for e in ordered:
        print(e)

Output

stdout
Original (shuffled) stream:
('user-a', 1, 1049906)
('user-c', 2, 8665766)
('user-b', 1, 2623710)
('user-c', 1, 4222659)
('user-b', 2, 6064335)
('user-a', 2, 6535903)

Ordered by partition key (then sequence):
('user-a', 1, 1049906)
('user-a', 2, 6535903)
('user-b', 1, 2623710)
('user-b', 2, 6064335)
('user-c', 1, 4222659)
('user-c', 2, 8665766)

How it works

The partition_key_events function generates a list of tuples using nested loops, assigning a random event_id via random.randint. The order_partition_events function uses sorted with a key that extracts the partition key as a string and the sequence number as an integer, ensuring deterministic ordering even with mixed key types. This pattern mirrors how real event streams are often sorted before processing to maintain per-key ordering, which is critical in systems like Kafka where each partition preserves order but interleaving across partitions is common.

Common mistakes

  • Forgetting to convert partition keys to a common type (e.g., str) in the sort key, leading to TypeError when mixing ints and strings.
  • Shuffling the list in-place with `random.shuffle` without copying, which mutates the original and can cause unintended side effects.
  • Assuming the event_id itself is meaningful for ordering; it's random and should not be used unless specifically designed for ordering.

Variations

  1. Use `itertools.product` to generate all key/sequence combinations more compactly.
  2. Sort using `operator.itemgetter` for slightly faster performance on large streams.

Real-world use cases

  • Simulating user event streams for load testing Kafka consumers where per-key ordering matters.
  • Generating mock data to validate event deduplication logic in stream processing pipelines.
  • Creating deterministic test fixtures for debugging ordering bugs in microservices that consume partitioned events.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Microservices patterns

Related tutorials and quizzes for this topic.