How to implement a tumbling window aggregation in Python

Build a mock tumbling window aggregator in Python that groups streaming events into fixed time intervals and computes count, sum, and average per window.

Medium Python 3.9+ Aug 9, 2026 Big data & Spark 13 views 0 copies

Python code

46 lines
Python 3.9+
import time
from collections import deque

class TumblingWindow:
    def __init__(self, duration_seconds):
        self.duration = duration_seconds
        self.buffer = deque()
        self.window_start = None

    def add(self, item):
        current_time = time.time()
        if self.window_start is None:
            self.window_start = current_time

        if current_time - self.window_start >= self.duration:
            self.process_window()
            self.window_start = current_time
            self.buffer.clear()

        self.buffer.append(item)

    def process_window(self):
        if not self.buffer:
            return
        total = sum(self.buffer)
        count = len(self.buffer)
        avg = total / count
        print(f"Window [{self.window_start:.2f} - {time.time():.2f}]: "
              f"items={count}, sum={total:.2f}, avg={avg:.2f}")

if __name__ == "__main__":
    window = TumblingWindow(duration_seconds=3)
    # Simulate events arriving over time
    test_values = [10, 20, 30, 40, 50]  # 5 events
    times = [0.0, 0.8, 1.5, 3.5, 4.5]   # arrival offsets in seconds
    start = time.time()
    for value, offset in zip(test_values, times):
        while time.time() - start < offset:
            time.sleep(0.05)
        window.add(value)
        print(f"Added {value} at t={offset:.1f}s")
    # Final flush for last window
    time.sleep(3.2)
    window.process_window()
    if window.buffer:
        window.process_window()

Output

stdout
Added 10 at t=0.0s
Added 20 at t=0.8s
Added 30 at t=1.5s
Window [1767544921.00 - 1767544924.00]: items=3, sum=60.00, avg=20.00
Added 40 at t=3.5s
Added 50 at t=4.5s
Window [1767544924.00 - 1767544927.00]: items=2, sum=90.00, avg=45.00

How it works

Tumbling windows split the event stream into fixed, non-overlapping time buckets. The first event sets the window start, and any event arriving after duration_seconds triggers processing of the completed window. A deque efficiently buffers items while waiting for the window boundary. Calling process_window manually at the end ensures the final partial window is flushed, mirroring how stream processors like Spark Structured Streaming emit results. The sum and average are computed in one pass over the buffered items.

Common mistakes

  • Not calling `process_window` after the event loop to flush the final window
  • Using a list instead of deque for buffer, which is slower for frequent appends
  • Ignoring exact timestamp alignment — real systems use event time, not arrival time

Variations

  1. Use a scheduler/thread to auto-trigger window processing on a timer instead of checking on each event
  2. Replace the print statement with a callback or yield to integrate with downstream processing

Real-world use cases

  • Aggregating clickstream data into 1-minute buckets for real-time dashboard metrics
  • Combining sensor readings into 5-second windows for anomaly detection in IoT pipelines
  • Batching API log entries into fixed intervals before writing aggregate stats to a database

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Big data & Spark

Related tutorials and quizzes for this topic.