How to Implement a Sliding Window Counter in Python
This code implements an approximate sliding window counter using a deque of time-based buckets to track event counts within a recent time window.
Python code
34 linesfrom collections import deque
from time import time
class SlidingWindowCounter:
def __init__(self, window_size, bucket_size=1):
self.window_size = window_size
self.bucket_size = bucket_size
self.buckets = deque()
def _evict_expired(self, now):
while self.buckets and self.buckets[0][0] <= now - self.window_size:
self.buckets.popleft()
def add_event(self):
now = time.time()
self._evict_expired(now)
if self.buckets and now - self.buckets[-1][0] < self.bucket_size:
self.buckets[-1][1] += 1
else:
self.buckets.append([int(now), 1])
self._evict_expired(now)
def count(self):
now = time.time()
self._evict_expired(now)
return sum(count for _, count in self.buckets)
if __name__ == "__main__":
window = SlidingWindowCounter(window_size=10, bucket_size=2)
for _ in range(5):
window.add_event()
print(f"Approximate event count in last 10s: {window.count()}")
Output
Approximate event count in last 10s: 5
How it works
The sliding window counter maintains a deque of buckets, each representing a time slice. When an event arrives, expired buckets are removed, and the event is added to the current bucket if it falls within the same time slice, otherwise a new bucket is created. The count sums the counts of all remaining buckets, giving an approximate count of events in the window. This avoids storing each event timestamp individually, saving memory for high-frequency events.
Common mistakes
- Using a fixed window that resets completely instead of sliding smoothly
- Not handling time drift by checking system time consistency
- Forgetting to evict expired buckets before counting
- Assuming exact counts when using coarser buckets
Variations
- Use a fixed-size list and rotate index instead of a deque
- Implement with a single counter and store only the window start time for simpler but less accurate approximation
Real-world use cases
- Rate limiting API endpoints to prevent abuse within a recent time window.
- Tracking user activity metrics like page views or clicks in a rolling time frame.
- Monitoring system health by counting errors or warnings per sliding minute.
Sponsored
More from Reliability & rate limiting
- At Least Once with Idempotent Consumer in Python medium
- Build a Rate Limiter Decorator in Python easy
- Build a queue-based admission control system in Python easy
- Chaos Inject Random Failures in Python easy
- Circuit breaker failure threshold count in Python medium
- Exactly Once Processing Dedupe Mock in Python easy
Keep learning
Related tutorials and quizzes for this topic.