How to Implement a Dead Letter Queue Replay in Python

A mock Dead Letter Queue that stores failed messages with retry attempts and replays them with a simple retry counter.

Easy Python 3.9+ Aug 9, 2026 Reliability & rate limiting 13 views 0 copies

Python code

39 lines
Python 3.9+
import json
from collections import deque

class DeadLetterQueue:
    def __init__(self):
        self.messages = deque()
    
    def add_message(self, message_id, payload, attempts=3):
        """Add a message to the DLQ with retry metadata."""
        self.messages.append({
            "id": message_id,
            "payload": payload,
            "attempts_left": attempts
        })
    
    def replay(self, message_id):
        """Replay a specific message, decrementing its attempts."""
        for msg in self.messages:
            if msg["id"] == message_id:
                if msg["attempts_left"] <= 0:
                    print(f"Message {message_id} exhausted (0 retries left)")
                    return None
                msg["attempts_left"] -= 1
                payload = msg["payload"]
                print(f"Replayed {message_id}: {json.dumps(payload)}")
                return payload
        print(f"Message {message_id} not found")
        return None

if __name__ == "__main__":
    dlq = DeadLetterQueue()
    dlq.add_message(1, {"event": "payment_failed", "amount": 99.50})
    dlq.add_message(2, {"event": "email_bounced", "address": "user@example.com"})
    
    dlq.replay(1)
    dlq.replay(2)
    dlq.replay(1)
    dlq.replay(1)
    dlq.replay(99)

Output

stdout
Replayed 1: {"event": "payment_failed", "amount": 99.5}
Replayed 2: {"event": "email_bounced", "address": "user@example.com"}
Replayed 1: {"event": "payment_failed", "amount": 99.5}
Message 1 exhausted (0 retries left)
Message 99 not found

How it works

This mock DLQ uses a deque to store messages in insertion order, allowing efficient appends and iteration. Each message carries an attempts_left field that is decremented on every replay call, simulating a retry budget. When the budget reaches zero, the message is considered exhausted and cannot be replayed further. The replay method simulates the action of sending the message back to a consumer and returns the payload for further processing. This pattern is useful for modeling retry policies without external dependencies.

Common mistakes

  • Not decrementing attempts before checking if it can be replayed, causing off-by-one errors
  • Forgetting to mark exhausted messages as permanently failed or removing them
  • Assuming message order is guaranteed when using plain lists instead of deque
  • Not using `json.dumps` to format payloads for readable logs

Variations

  1. Use a dictionary keyed by message ID for O(1) lookups instead of scanning all messages
  2. Add a `pending` flag to track whether a message is available for replay

Real-world use cases

  • Replaying failed Kafka events after a downstream service recovers from an outage.
  • Retrying webhook deliveries that temporarily failed due to rate limits or timeouts.
  • Requeueing failed API requests in a job queue with a configurable max retry count.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Reliability & rate limiting

Related tutorials and quizzes for this topic.