How to implement a tumbling window aggregation in Python
Build a mock tumbling window aggregator in Python that groups streaming events into fixed time intervals and computes count, sum, and average per window.
Python code
46 linesimport time
from collections import deque
class TumblingWindow:
def __init__(self, duration_seconds):
self.duration = duration_seconds
self.buffer = deque()
self.window_start = None
def add(self, item):
current_time = time.time()
if self.window_start is None:
self.window_start = current_time
if current_time - self.window_start >= self.duration:
self.process_window()
self.window_start = current_time
self.buffer.clear()
self.buffer.append(item)
def process_window(self):
if not self.buffer:
return
total = sum(self.buffer)
count = len(self.buffer)
avg = total / count
print(f"Window [{self.window_start:.2f} - {time.time():.2f}]: "
f"items={count}, sum={total:.2f}, avg={avg:.2f}")
if __name__ == "__main__":
window = TumblingWindow(duration_seconds=3)
# Simulate events arriving over time
test_values = [10, 20, 30, 40, 50] # 5 events
times = [0.0, 0.8, 1.5, 3.5, 4.5] # arrival offsets in seconds
start = time.time()
for value, offset in zip(test_values, times):
while time.time() - start < offset:
time.sleep(0.05)
window.add(value)
print(f"Added {value} at t={offset:.1f}s")
# Final flush for last window
time.sleep(3.2)
window.process_window()
if window.buffer:
window.process_window()
Output
Added 10 at t=0.0s
Added 20 at t=0.8s
Added 30 at t=1.5s
Window [1767544921.00 - 1767544924.00]: items=3, sum=60.00, avg=20.00
Added 40 at t=3.5s
Added 50 at t=4.5s
Window [1767544924.00 - 1767544927.00]: items=2, sum=90.00, avg=45.00
How it works
Tumbling windows split the event stream into fixed, non-overlapping time buckets. The first event sets the window start, and any event arriving after duration_seconds triggers processing of the completed window. A deque efficiently buffers items while waiting for the window boundary. Calling process_window manually at the end ensures the final partial window is flushed, mirroring how stream processors like Spark Structured Streaming emit results. The sum and average are computed in one pass over the buffered items.
Common mistakes
- Not calling `process_window` after the event loop to flush the final window
- Using a list instead of deque for buffer, which is slower for frequent appends
- Ignoring exact timestamp alignment — real systems use event time, not arrival time
Variations
- Use a scheduler/thread to auto-trigger window processing on a timer instead of checking on each event
- Replace the print statement with a callback or yield to integrate with downstream processing
Real-world use cases
- Aggregating clickstream data into 1-minute buckets for real-time dashboard metrics
- Combining sensor readings into 5-second windows for anomaly detection in IoT pipelines
- Batching API log entries into fixed intervals before writing aggregate stats to a database
Sponsored
More from Big data & Spark
Keep learning
Related tutorials and quizzes for this topic.