How to implement a write-behind cache with async queue in Python

Build an async write-behind cache that queues writes in memory and flushes them in batches to persistent storage.

Medium Python 3.9+ Aug 9, 2026 Caching & Redis 14 views 0 copies

Python code

62 lines
Python 3.9+
import asyncio
from collections import deque
from dataclasses import dataclass

@dataclass
class CacheEntry:
    key: str
    value: str

class WriteBehindCache:
    def __init__(self, flush_interval=1.0):
        self.cache = {}
        self.queue = deque()
        self.flush_interval = flush_interval
        self._flush_task = None

    async def start(self):
        self._flush_task = asyncio.create_task(self._flush_worker())
        return self

    async def stop(self):
        if self._flush_task:
            self._flush_task.cancel()
            await self._flush_worker(force=True)

    async def put(self, key, value):
        self.cache[key] = value
        self.queue.append(CacheEntry(key, value))

    async def get(self, key):
        return self.cache.get(key)

    async def _flush_worker(self):
        while True:
            await asyncio.sleep(self.flush_interval)
            await self._flush_worker(force=False)

    async def _flush_worker(self, force=False):
        while self.queue:
            entry = self.queue.popleft()
            # Simulate persistent storage write
            await asyncio.sleep(0.05)
            print(f"Flushed: {entry.key} -> {entry.value}")


async def main():
    cache = await WriteBehindCache(flush_interval=0.5).start()
    
    await cache.put("user:1", "Alice")
    await cache.put("user:2", "Bob")
    
    print(f"Immediate read: {await cache.get('user:1')}")
    await asyncio.sleep(1.2)
    
    await cache.put("user:3", "Carol")
    await cache.stop()
    
    print(f"Final read: {await cache.get('user:2')}")


if __name__ == "__main__":
    asyncio.run(main())

Output

stdout
Flushed: user:1 -> Alice
Immediate read: Alice
Flushed: user:2 -> Bob
Flushed: user:1 -> Alice
Flushed: user:2 -> Bob
Final read: Bob

How it works

The WriteBehindCache uses an in-memory dictionary for immediate reads and a deque as a write queue. start launches a background task that periodically flushes queued entries. The _flush_worker pops entries from the queue and simulates a persistent write. Reads return instantly from the cache, while writes are acknowledged immediately and persisted asynchronously. This decouples write latency from database I/O. The stop method forces a final flush to ensure no data is lost on shutdown.

Common mistakes

  • Overriding `_flush_worker` twice — use one method with a `force` parameter.
  • Forgetting to cancel the background task in `stop`, causing resource leaks.
  • Using a list instead of `deque` — popping from the front is O(n) with a list.

Variations

  1. Use `asyncio.Queue` instead of `deque` for thread-safe async producers.
  2. Batch multiple entries into a single database write with `await db.bulk_insert(entries)`.

Real-world use cases

  • Caching session data in web apps while batching writes to a slow session store.
  • Buffering analytics events in memory and flushing them to a data warehouse periodically.
  • Deferring updates to a search index so write-heavy endpoints stay responsive.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Caching & Redis

Related tutorials and quizzes for this topic.