Rate Limiting with Queue Rejection in Python
Simulates a load shed pattern that rejects tasks when a queue fills up.
Python code
36 linesfrom collections import deque
import time
class RateLimiter:
def __init__(self, max_queue_size=3):
self.queue = deque()
self.max_queue_size = max_queue_size
self.rejected_count = 0
def submit(self, task_name):
if len(self.queue) >= self.max_queue_size:
self.rejected_count += 1
return f"REJECT {task_name} - queue full"
self.queue.append(task_name)
return f"ACCEPT {task_name} - queued"
def process_next(self):
if self.queue:
return f"PROCESS {self.queue.popleft()}"
return "PROCESS - queue empty"
if __name__ == "__main__":
limiter = RateLimiter(max_queue_size=3)
# Simulate a burst of submissions
for i in range(5):
result = limiter.submit(f"task_{i}")
print(result)
print("---")
while limiter.queue:
print(limiter.process_next())
print("---")
print(f"Total rejected: {limiter.rejected_count}")
Output
ACCEPT task_0 - queued
ACCEPT task_1 - queued
ACCEPT task_2 - queued
REJECT task_3 - queue full
REJECT task_4 - queue full
---
PROCESS task_0
PROCESS task_1
PROCESS task_2
---
Total rejected: 2
How it works
The RateLimiter uses a deque with a maximum size to model a bounded buffer. The submit method checks the current queue length against max_queue_size and rejects new tasks when the buffer is full, incrementing a rejection counter. This is a classic producer blocking or load shedding pattern, allowing the system to apply backpressure without unbounded memory growth. Processing logic is decoupled, letting consumer functions pull from the queue when ready, ensuring the queue never exceeds its bound and shedding excess load gracefully.
Common mistakes
- Using a list instead of deque, which is inefficient for popleft operations.
- Not clearing the queue when it's full, leading to unbounded memory consumption.
- Forgetting to check queue size before appending, causing overflow.
Variations
- Using asyncio.Queue with maxsize and try/except asyncio.QueueFull to reject asynchronously.
- Adding a blocking variant that waits for space instead of rejecting instantly.
Real-world use cases
- Rejecting incoming HTTP requests when the worker pool is saturated to prevent overload.
- Message queue consumers dropping messages when the local processing buffer exceeds a limit.
- Rate limiting API calls from a client to avoid exceeding backend quotas or service provider limits.
Sponsored
More from Reliability & rate limiting
- At Least Once with Idempotent Consumer in Python medium
- Build a Rate Limiter Decorator in Python easy
- Build a queue-based admission control system in Python easy
- Chaos Inject Random Failures in Python easy
- Circuit breaker failure threshold count in Python medium
- Exactly Once Processing Dedupe Mock in Python easy
Keep learning
Related tutorials and quizzes for this topic.