How to Implement an In-Memory Pub/Sub System in Python
This code implements a simple in-memory publish/subscribe system in Python, allowing topics, callbacks, and message broadcasting.
Python code
38 linesclass PubSub:
def __init__(self):
self.topics = {}
def subscribe(self, topic, callback):
if topic not in self.topics:
self.topics[topic] = []
self.topics[topic].append(callback)
return lambda: self.unsubscribe(topic, callback)
def unsubscribe(self, topic, callback):
if topic in self.topics and callback in self.topics[topic]:
self.topics[topic].remove(callback)
if not self.topics[topic]:
del self.topics[topic]
def publish(self, topic, message):
if topic in self.topics:
for callback in list(self.topics[topic]):
callback(message)
if __name__ == "__main__":
pubsub = PubSub()
received = []
unsubscribe_foo = pubsub.subscribe("news", lambda m: received.append(f"news: {m}"))
pubsub.subscribe("sports", lambda m: received.append(f"sports: {m}"))
pubsub.publish("news", "Hello world")
pubsub.publish("sports", "Game update")
publish_again = pubsub.publish("news", "Second article")
unsubscribe_foo()
pubsub.publish("news", "After unsubscribe")
print(received)
Output
['news: Hello world', 'sports: Game update', 'news: Second article']
How it works
The PubSub class keeps a dictionary mapping topic names to lists of callback functions. When you subscribe, the callback is appended to the topic's list, and an unsubscribe closure is returned. publish iterates over a copy of the subscribers list to avoid mutation issues if a callback unsubscribes during delivery. unsubscribe removes the callback and cleans up empty topic entries. The main block demonstrates subscribing, publishing, unsubscribing, and verifying that messages after unsubscribe do not reach the removed callback.
Common mistakes
- Mutating the subscriber list while iterating over it during publish; use a copy.
- Forgetting to clean up empty topics after unsubscribing, causing memory leaks.
- Assuming an unsubscribe function is idempotent; it may silently fail if called twice.
- Not thread-safe; concurrent publish/subscribe can cause race conditions.
Variations
- Use `collections.defaultdict(list)` to simplify topic initialization.
- Add support for wildcard topics or pattern matching (e.g., 'news.*').
Real-world use cases
- Coordinating UI updates across modules in a desktop or web application without tight coupling.
- Implementing an event bus in a microservice to decouple components within a single process.
- Building a lightweight notification system for in-process cron-like tasks or background workers.
Sponsored
More from Streaming & messaging
- At Most Once Fire-and-Forget Mock in Python easy
- Batch Consume Process Commit Pattern in Python medium
- Build a Streaming Messaging Helper in Python easy
- Dead Letter Queue Failed Messages List Mock in Python easy
- Dedupe processed message IDs in Python easy
- Event Envelope with Schema Version Field in Python easy
Keep learning
Related tutorials and quizzes for this topic.