How to Build a Materialized View Updater Consumer Mock in Python
A mock consumer that queues change events and triggers refresh callbacks to simulate materialized view updates.
Python code
38 linesimport time
from collections import deque
from dataclasses import dataclass, field
from typing import Callable, Deque, Optional
@dataclass
class MaterializedViewUpdater:
"""Mock updater that consumes change events and refreshes a view."""
refresh: Optional[Callable[[str], None]] = None
queue: Deque[tuple] = field(default_factory=deque)
def consume(self, event: tuple) -> None:
"""Accept a change event, add to queue, and attempt refresh."""
self.queue.append(event)
if self.refresh:
self.refresh(event[1])
def drain(self) -> int:
"""Process all queued events with simple simulated work."""
count = 0
while self.queue:
_, entity_id = self.queue.popleft()
time.sleep(0.01) # Simulate refresh latency
count += 1
return count
def example_refresh(entity_id: str) -> None:
print(f"Refreshing view for {entity_id}")
if __name__ == "__main__":
updater = MaterializedViewUpdater(refresh=example_refresh)
events = [("INSERT", "user_42"), ("UPDATE", "order_7"), ("DELETE", "order_7")]
for ev in events:
updater.consume(ev)
print(f"Processed {updater.drain()} events")
Output
Refreshing view for user_42
Refreshing view for order_7
Refreshing view for order_7
Processed 3 events
How it works
This code defines a MaterializedViewUpdater dataclass that uses a deque as an internal queue to buffer change events. The consume method appends an event and immediately invokes a refresh callback if provided, simulating async notification. The drain method processes all queued events, simulating the actual view refresh latency with time.sleep. The example demonstrates how the mock can be used to test streaming consumers without a real message broker.
Common mistakes
- Using a list instead of deque, which loses O(1) popleft efficiency
- Forgetting to handle None for the refresh callback
- Mixing up the order of event tuples when accessing elements
Variations
- Use asyncio.Queue with async callbacks for non-blocking consumption
- Add a batch refresh method that accepts multiple events at once
Real-world use cases
- Unit testing event-driven services that react to database changes in a microservice architecture.
- Prototyping a view refresh pipeline before integrating with a real message broker like Kafka.
- Simulating consumer behavior in integration tests for an ETL data pipeline.
Sponsored
More from Streaming & messaging
- At Most Once Fire-and-Forget Mock in Python easy
- Batch Consume Process Commit Pattern in Python medium
- Build a Streaming Messaging Helper in Python easy
- Dead Letter Queue Failed Messages List Mock in Python easy
- Dedupe processed message IDs in Python easy
- Event Envelope with Schema Version Field in Python easy
Keep learning
Related tutorials and quizzes for this topic.