Reference library

Concurrency & performance

asyncio, threading, multiprocessing, and profiling-friendly performance patterns.

6 matches

How to Memoize Async Functions with lru_cache in Python

Cache async function results with functools.lru_cache to avoid repeated expensive awaits, cutting total execution from ~0.4s to ~0.2s in this example.

asyncio lru_cache memoization

Python

from functools import lru_cache
import asyncio

@lru_cache(maxsize=128)
async def fetch_data(user_id: int) -> str:
    # Simulate expensive async operation
    await asyncio.sleep(0.1)
    return f"Data for user {user_id}"

async def main():
    start = asyncio.get_event_loop().time()
    
    # First calls (miss cach…

13 0 Open

Concurrency & performance medium

How to Parse JSON Files in Parallel with Python ThreadPoolExecutor

Load and transform JSON records from multiple files concurrently using ThreadPoolExecutor for faster I/O-bound parsing.

threadpool json concurrency

Python

import time
from concurrent.futures import ThreadPoolExecutor
import json

def load_json_file(path):
    with open(path, 'r') as f:
        return json.load(f)

def transform_record(record):
    record['full_name'] = f"{record.pop('first_name', '')} {record.pop('last_name', '')}".strip()
    record['score'] = int(reco…

17 0 Open

Concurrency & performance easy

How to Send and Receive Messages Between Processes with multiprocessing.Pipe in Python

Use multiprocessing.Pipe to create a two-way connection between two processes, send a message from parent to child, and receive a reply back.

multiprocessing pipe interprocess-communication

Python

import multiprocessing


def child_process(conn):
    """Receive from parent and send back a response."""
    message = conn.recv()
    print(f"Child received: {message}")
    conn.send("Hello from child!")


if __name__ == "__main__":
    parent_conn, child_conn = multiprocessing.Pipe()

    process = multiprocessing…

14 0 Open

Concurrency & performance medium

How to Share a Queue Between Processes in Python

Use multiprocessing.Queue to pass work from a producer process to multiple consumer processes, coordinating with a sentinel stop message.

multiprocessing queue concurrency

Python

import multiprocessing
import time


def producer(queue, items):
    for item in items:
        queue.put(item)
        time.sleep(0.1)
    queue.put("STOP")


def consumer(queue, name):
    while True:
        item = queue.get()
        if item == "STOP":
            break
        print(f"{name} processed: {item}")

…

14 0 Open

Concurrency & performance medium

How to Use asyncio Lock to Protect a Shared Counter in Python

This code demonstrates how to use an asyncio.Lock to safely increment a shared counter from multiple concurrent coroutines.

asyncio lock concurrency

Python

import asyncio

async def increment(counter, lock, increments):
    for _ in range(increments):
        async with lock:
            counter[0] += 1

async def main():
    counter = [0]
    lock = asyncio.Lock()
    tasks = [
        increment(counter, lock, 1000)
        for _ in range(5)
    ]
    await asyncio.gath…

16 0 Open

Concurrency & performance easy

How to use ThreadPoolExecutor for concurrent tasks in Python

Run blocking functions in parallel with ThreadPoolExecutor and as_completed, cutting total runtime from 5 sequential sleeps to about 1 second.

concurrency threadpoolexecutor parallel

Python

import time
from concurrent.futures import ThreadPoolExecutor, as_completed


def fetch_data(item):
    """Simulate a slow operation with a fixed delay."""
    time.sleep(0.2)
    return item * 2


def main():
    items = [1, 2, 3, 4, 5]
    start = time.perf_counter()

    with ThreadPoolExecutor(max_workers=3) as ex…

14 0 Open

Browse by section

Each section groups closely related Python snippets.