Reference library

Streaming & messaging

Kafka-style pub/sub, event consumers, async pipelines, and message-driven workflows.

6 matches

Build a Streaming Messaging Helper in Python

Create a simple message stream class that stores recent messages, sends user messages, and retrieves history or latest messages with timestamps.

streaming deque dataclass

Python

from collections import deque
from dataclasses import dataclass
from datetime import datetime
import time


@dataclass
class Message:
    user: str
    text: str
    timestamp: str = ""

    def __post_init__(self):
        if not self.timestamp:
            self.timestamp = datetime.now().strftime("%H:%M:%S")


class…

13 0 Open

Streaming & messaging easy

How to Build a Message Stream Queue in Python

A beginner-friendly MessageStream class built on deque that sends messages one at a time, tracks unread counts, and records sent items.

queue deque streaming

Python

from collections import deque
import time


class MessageStream:
    def __init__(self, messages):
        self._queue = deque(messages)
        self._sent = []

    def send_next(self):
        if not self._queue:
            return None
        message = self._queue.popleft()
        self._sent.append(message)
     …

13 0 Open

Streaming & messaging easy

How to Build a Mock Change Data Capture Event Stream in Python

Generate a deterministic list of mock CDC events with event IDs, stream positions, payloads, and timestamps for testing streaming pipelines.

cdc mock event-stream

Python

from itertools import count
from random import choice, randint, seed
from datetime import datetime, timedelta

seed(42)  # Make output deterministic
event_types = ["INSERT", "UPDATE", "DELETE"]
table_names = ["users", "orders", "products", "payments"]
counter = count(1)

def mock_cdc_event(stream_index: int) -> dict:
…

12 0 Open

Streaming & messaging easy

How to Implement a Tumbling Window Counter in Python

Count events that fall within a fixed-size sliding time window using a deque and pruning logic.

streaming window aggregation

Python

from collections import deque
import time


class TumblingWindowCounter:
    def __init__(self, window_size_seconds):
        self.window_size = window_size_seconds
        self.window = deque()

    def add_event(self, timestamp):
        self.window.append(timestamp)

    def count(self, current_time):
        while…

14 0 Open

Streaming & messaging easy

How to Serialize and Deserialize JSON Event Payloads in Python

Define an EventPayload class with custom to_json and from_json methods to convert event objects to JSON strings and back, using datetime parsing.

json serialization datetime

Python

import json
from datetime import datetime


class EventPayload:
    def __init__(self, event_id, event_type, timestamp, data):
        self.event_id = event_id
        self.event_type = event_type
        self.timestamp = timestamp
        self.data = data

    def to_json(self):
        return json.dumps({
          …

12 0 Open

Streaming & messaging easy

How to Simulate a Micro-Batch Interval Trigger in Python

A dataclass-based mock that emits batch numbers at fixed intervals, mimicking a micro-batch streaming scheduler for testing and development.

streaming mock dataclass

Python

import time
from dataclasses import dataclass, field
from typing import List, Callable


@dataclass
class MicroBatchTriggerMock:
    batch_interval_seconds: float = 0.5
    max_batches: int = 5
    _batches_emitted: int = 0
    _next_emit_time: float = field(init=False, default=0)

    def start(self, on_batch: Callab…

13 0 Open

Browse by section

Each section groups closely related Python snippets.