Rate Limiting with Queue Rejection in Python

Simulates a load shed pattern that rejects tasks when a queue fills up.

Easy Python 3.9+ Aug 9, 2026 Reliability & rate limiting 15 views 0 copies

Python code

36 lines
Python 3.9+
from collections import deque
import time


class RateLimiter:
    def __init__(self, max_queue_size=3):
        self.queue = deque()
        self.max_queue_size = max_queue_size
        self.rejected_count = 0

    def submit(self, task_name):
        if len(self.queue) >= self.max_queue_size:
            self.rejected_count += 1
            return f"REJECT {task_name} - queue full"
        self.queue.append(task_name)
        return f"ACCEPT {task_name} - queued"

    def process_next(self):
        if self.queue:
            return f"PROCESS {self.queue.popleft()}"
        return "PROCESS - queue empty"


if __name__ == "__main__":
    limiter = RateLimiter(max_queue_size=3)

    # Simulate a burst of submissions
    for i in range(5):
        result = limiter.submit(f"task_{i}")
        print(result)

    print("---")
    while limiter.queue:
        print(limiter.process_next())
    print("---")
    print(f"Total rejected: {limiter.rejected_count}")

Output

stdout
ACCEPT task_0 - queued
ACCEPT task_1 - queued
ACCEPT task_2 - queued
REJECT task_3 - queue full
REJECT task_4 - queue full
---
PROCESS task_0
PROCESS task_1
PROCESS task_2
---
Total rejected: 2

How it works

The RateLimiter uses a deque with a maximum size to model a bounded buffer. The submit method checks the current queue length against max_queue_size and rejects new tasks when the buffer is full, incrementing a rejection counter. This is a classic producer blocking or load shedding pattern, allowing the system to apply backpressure without unbounded memory growth. Processing logic is decoupled, letting consumer functions pull from the queue when ready, ensuring the queue never exceeds its bound and shedding excess load gracefully.

Common mistakes

  • Using a list instead of deque, which is inefficient for popleft operations.
  • Not clearing the queue when it's full, leading to unbounded memory consumption.
  • Forgetting to check queue size before appending, causing overflow.

Variations

  1. Using asyncio.Queue with maxsize and try/except asyncio.QueueFull to reject asynchronously.
  2. Adding a blocking variant that waits for space instead of rejecting instantly.

Real-world use cases

  • Rejecting incoming HTTP requests when the worker pool is saturated to prevent overload.
  • Message queue consumers dropping messages when the local processing buffer exceeds a limit.
  • Rate limiting API calls from a client to avoid exceeding backend quotas or service provider limits.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Reliability & rate limiting

Related tutorials and quizzes for this topic.