Reference library

Concurrency & performance

asyncio, threading, multiprocessing, and profiling-friendly performance patterns.

3 matches

How to Build a Producer-Consumer Pattern with asyncio.Queue in Python

This code implements a classic producer-consumer pattern using asyncio.Queue to coordinate one producer task that generates items and two consumer tasks that process them concurrently, with a sentinel value to signal completion.

asyncio queue concurrency

Python

import asyncio
import random


async def producer(queue, item_count):
    for i in range(item_count):
        item = random.randint(1, 100)
        await queue.put(item)
        print(f"Produced: {item}")
        await asyncio.sleep(0.1)
    await queue.put(None)  # Sentinel to signal end


async def consumer(queue, n…

15 0 Open

Concurrency & performance medium

How to Implement a Batch Requests Flush Interval in Python

A simple async batcher that accumulates items and flushes them either when a max batch size is reached or after a time-based flush interval.

asyncio batching concurrency

Python

import asyncio
from collections import deque

class Batcher:
    def __init__(self, flush_interval=0.5, max_batch=5):
        self.flush_interval = flush_interval
        self.max_batch = max_batch
        self.queue = deque()
        self.lock = asyncio.Lock()

    async def add(self, item):
        async with self.l…

14 0 Open

Concurrency & performance medium

How to Share a Queue Between Processes in Python

Use multiprocessing.Queue to pass work from a producer process to multiple consumer processes, coordinating with a sentinel stop message.

multiprocessing queue concurrency

Python

import multiprocessing
import time


def producer(queue, items):
    for item in items:
        queue.put(item)
        time.sleep(0.1)
    queue.put("STOP")


def consumer(queue, name):
    while True:
        item = queue.get()
        if item == "STOP":
            break
        print(f"{name} processed: {item}")

…

13 0 Open

Browse by section

Each section groups closely related Python snippets.