Filter Messages with Topic Exchanges

You’ve built queues that deliver every message to a single consumer, and fan-out exchanges that blast copies to every bound queue. But what happens when you need selective routing — sending only the messages that match a pattern, like order.europe.created but not order.asia.created? Without this, your consumers drown in irrelevant traffic, waste CPU on filtering, and couple themselves to the details of every event type. Filter messages using topic exchanges solves this by letting exchanges route on wildcard patterns, so your queues receive only what they actually care about.

The problem this lesson solves

Imagine you run an e-commerce platform with events like order.created, order.shipped, payment.failed, and user.registered. A billing service only cares about payment.failed and order.shipped; a notification service needs user.registered and order.created. A naive approach — one queue per event type or a single fan-out to all queues — either explodes your queue count or forces every consumer to filter and discard most messages.

That’s the pain: you need routing that’s more flexible than a direct exchange but more selective than a fan-out. Topic exchanges exist precisely for this. They let you bind queues with patterns like order.* or payment.#, and the exchange matches each message’s routing key to those patterns. The right messages flow to the right queues; everything else is ignored.

This lesson shows you how topic exchanges work, when to reach for them over other exchange types, and how to implement filtering in a real messaging system using RabbitMQ and Python.

Core concept / mental model

Think of a topic exchange as a mailroom with smart sorting rules. A message arrives with a routing key — its address label. The mailroom doesn’t read the whole letter; it just looks at the label and matches it against the subscription patterns each queue has posted. If the label matches, the queue gets a copy; if not, it goes in the trash.

The routing key is a dot-separated list of words, like order.europe.created. The pattern is built from the same words, but with two wildcards:

• * (star) matches exactly one word in that position.
• # (hash) matches zero or more words at that position.

For example, the pattern order.* matches order.created, order.shipped, but not order.europe.created (because that has three words). The pattern order.# matches all three — it swallows everything after order.

A queue binding associates a queue with a pattern on the exchange. When you publish a message with routing key order.europe.created, the exchange evaluates it against all bindings and routes to every queue whose pattern matches.

This mental model is key: you’re not filtering messages in consumer code — you’re filtering at the exchange, before the message ever reaches a queue. That separation makes your consumers simpler and your system more maintainable.

How it works step by step

1. Declare a topic exchange. In RabbitMQ, you set the exchange type to topic. This tells the broker you’ll be using wildcard patterns for routing.

2. Bind queues to the exchange with patterns. Each queue declares its interest by binding to a pattern. Multiple queues can bind the same pattern; one queue can bind several patterns.

3. Publish messages with routing keys. The producer sends each message with a routing key of dot-separated words. The routing key’s structure is arbitrary, but it must be meaningful for your domains and patterns.

4. Exchange matches and routes. The broker compares the routing key to each binding pattern. For every match, it enqueues a copy of the message in the bound queue.

5. Consumers receive only matching messages. Your consumer code doesn’t filter — it just processes whatever arrives.

Let’s trace an example. Suppose you have an exchange named events and three queues:

• billing bound with payment.#
• orders bound with order.*
• audit bound with # (matches everything)

Publish payment.failed → billing and audit get it; orders doesn’t. Publish order.created → orders and audit get it; billing doesn’t. Publish order.europe.created → only audit gets it, because order.* only matches a two-word key.

That’s the power: one exchange, many selective bindings, zero consumer-side filtering logic.

Hands-on walkthrough

Let’s implement this with RabbitMQ and the pika library in Python. First, install the client if you haven’t already:

pip install pika

Start RabbitMQ locally via Docker for a quick test environment:

docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

Step 1: Set up the exchange and bindings

Create a Python script setup.py that declares the topic exchange and binds queues with patterns:

import pika

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

# Declare a topic exchange
exchange_name = 'events'
channel.exchange_declare(exchange=exchange_name, exchange_type='topic')

# Declare queues
channel.queue_declare(queue='billing', durable=True)
channel.queue_declare(queue='orders', durable=True)
channel.queue_declare(queue='audit', durable=True)

# Bind queues with patterns
channel.queue_bind(exchange=exchange_name, queue='billing', routing_key='payment.#')
channel.queue_bind(exchange=exchange_name, queue='orders', routing_key='order.*')
channel.queue_bind(exchange=exchange_name, queue='audit', routing_key='#')

print('Exchange and bindings set up.')
connection.close()

Step 2: Publish messages with routing keys

Now publish a batch of events, each with a routing key that reflects its domain:

import pika

exchange_name = 'events'
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()

messages = [
    ('order.created', 'Order 1234 placed'),
    ('order.shipped', 'Order 1234 shipped'),
    ('order.europe.created', 'Order 5678 placed in Europe'),
    ('payment.failed', 'Payment for order 1234 failed'),
    ('user.registered', 'New user signed up')
]

for routing_key, body in messages:
    channel.basic_publish(
        exchange=exchange_name,
        routing_key=routing_key,
        body=body,
        properties=pika.BasicProperties(delivery_mode=2)  # persistent
    )
    print(f'Published [{routing_key}] {body}')

connection.close()

Step 3: Consume with selective queues

Write a consumer that listens on the billing queue and prints what it gets. Since the exchange already filtered, you don’t need any if statements:

import pika

def callback(ch, method, properties, body):
    print(f'Billing service got: [{method.routing_key}] {body.decode()}')

connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
channel.queue_declare(queue='billing', durable=True)
channel.basic_consume(queue='billing', on_message_callback=callback, auto_ack=True)

print('Billing consumer started. Waiting for messages...')
channel.start_consuming()

Run setup.py, then the publisher, then the billing consumer. Expected output:

Billing service got: [payment.failed] Payment for order 1234 failed

Run the same consumer on orders (change the queue name) and you’ll see only order.created and order.shipped — not order.europe.created. That’s topic exchange filtering in action.

Pro tip: Always declare the exchange and bindings before publishing. If a queue is bound after messages are published, those messages are lost — topic exchanges don’t store messages, they only route.

Compare options / when to choose what

Topic exchanges aren’t the only routing tool. Here’s how they stack up against direct and fan-out exchanges:

When to choose topic over direct: You need multiple consumers to receive subsets of messages, and the routing keys have meaningful hierarchy. Direct would require many bindings or a per-event exchange — messy.

When to choose topic over fan-out: You don’t want every consumer to see every message. Fan-out forces wasteful filtering downstream.

When to use a hybrid: Combine topic and fan-out in the same system — e.g., audit logging uses # on a topic exchange, while operational teams use direct exchanges for exact commands.

Variations worth knowing

• AMQP 0-9-1 topic exchanges (RabbitMQ’s native) are the classic implementation. They use and #, but note that matches exactly one word — there’s no way to match “any two words” without explicit patterns.
• Apache Kafka doesn’t have topic exchanges, but its consumer groups and regex subscription (subscribe(Pattern)) achieve similar filtering, just at the consumer level rather than the broker’s exchange.
• Cloud Pub/Sub has subscription filters with a different syntax, but the concept of “push down filtering” is identical — filter at the broker, not in your code.

Troubleshooting & edge cases

Messages not arriving? Check that the exchange type is topic, not direct or fanout. If you declared it wrong, no pattern will match. Delete and redeclare with the correct type.

Pattern matches too much or too little? Remember * matches exactly one word, # matches zero or more. The key order.europe.created won’t match order.*, but will match order.#. Test with a small script before production.

Queue bound after publishing — lost messages. If a consumer starts after messages were published, those messages are gone. Use queue TTL or a separate “offline queue” if you need replay.

Routing key with fewer than expected segments. If your pattern is order.# and you publish order (one word), it matches. But order.* won’t match order — that’s a matching, not an error. Add validation in your producer to enforce routing key structure.

Performance concerns. Topic exchanges are fast, but thousands of binding patterns per exchange can add overhead. Keep patterns coarse (region.#) and narrow down in consumer code if needed.

Mismatched queue durability. If your queue isn’t durable, it disappears on broker restart, and your bindings vanish too. Use durable=True for queues and messages you can’t afford to lose.

What you learned & what's next

You now know how to filter messages using topic exchanges: you understand the wildcard pattern syntax (* and #), you’ve seen how bindings route only matching messages to queues, and you’ve built a working Python + RabbitMQ example with selective consumers.

You’ve met the learning objectives:

• Explain the core idea behind topic exchange filtering — the broker does the filtering, not your code.
• Complete a practical exercise that sets up bindings, publishes with routing keys, and consumes only relevant messages.

You’ve also connected this to the broader messaging puzzle: topic exchanges sit between direct (exact) and fan-out (broadcast) routing, and they’re your best tool for event-driven architectures where consumers have different interests.

What’s next: In the next lesson, you’ll build on this foundation by learning how to handle message routing failures — think dead-letter exchanges and retry policies. Understanding topic exchange patterns first gives you the precise vocabulary to design robust delivery guarantees.

Now try a small variation on your own: add a region.# binding to route messages like order.europe.created to a separate “regional” queue. Experiment with more complex routing keys and patterns — that hands-on tinkering will cement the concept.