Concurrency & performance
asyncio, threading, multiprocessing, and profiling-friendly performance patterns.
Build a Python Performance Profiler That Generates Readable Reports
Use cProfile and pstats to profile Python functions and print a sorted performance report showing the top time-consuming calls.
import cProfile
import pstats
import io
from pathlib import Path
def slow_function():
total = 0
for i in range(500_000):
total += i ** 2
return total
def fast_function():
total = sum(i * i for i in range(500_000))
return total
def profile_functions():
profiler = cProfile.Profile()
…How to Demonstrate the GIL with Python Threads vs Processes
Measure and compare wall-clock time for CPU-bound work using Python threads (limited by the GIL) versus multiprocessing (which bypasses the GIL).
import threading
import multiprocessing
import time
import os
def cpu_heavy(n):
return sum(i * i for i in range(n))
def run_threads(n):
threads = [threading.Thread(target=cpu_heavy, args=(n,)) for _ in range(2)]
start = time.perf_counter()
for t in threads:
t.start()
for t in threads:
…How to Implement a Token Bucket Rate Limiter with asyncio in Python
This code implements a thread-safe token bucket rate limiter for asyncio, allowing you to limit the rate of async tasks or API calls.
import asyncio
import time
class TokenBucket:
def __init__(self, rate_per_second, capacity):
self.rate = rate_per_second
self.capacity = capacity
self.tokens = capacity
self.last_refill = time.monotonic()
self.lock = asyncio.Lock()
async def acquire(self):
asy…How to Mock anyio.run Backends (asyncio vs trio) in Python
Demonstrates how to mock anyio.run to verify backend selection (asyncio or trio) without actually running the event loop.
import anyio
from unittest.mock import Mock, patch
async def fetch_data():
await anyio.sleep(0.1)
return {"data": 42}
def run_with_backend(backend: str):
async def main():
result = await fetch_data()
print(f"[{backend}] Result: {result}")
anyio.run(main, backend=backend)
if __nam…How to Parse JSON Files in Parallel with Python ThreadPoolExecutor
Load and transform JSON records from multiple files concurrently using ThreadPoolExecutor for faster I/O-bound parsing.
import time
from concurrent.futures import ThreadPoolExecutor
import json
def load_json_file(path):
with open(path, 'r') as f:
return json.load(f)
def transform_record(record):
record['full_name'] = f"{record.pop('first_name', '')} {record.pop('last_name', '')}".strip()
record['score'] = int(reco…How to Run Blocking Code in an Executor with asyncio in Python
This code runs blocking functions concurrently without stalling the event loop by offloading them to thread pool executors via asyncio.
import asyncio
import time
def blocking_task(name: str, duration: float) -> str:
"""Simulate a blocking operation."""
time.sleep(duration)
return f"Finished {name} after {duration}s"
async def main() -> None:
loop = asyncio.get_running_loop()
results = await asyncio.gather(
loop.run_in_…How to Run Coroutines Concurrently with asyncio.gather in Python
Run multiple async coroutines concurrently and collect their results in the order they were passed.
import asyncio
async def fetch_data(name: str, delay: float) -> str:
"""Simulate an async operation (e.g., API call) with a delay."""
await asyncio.sleep(delay)
return f"{name} data (after {delay}s)"
async def main() -> None:
"""Run multiple coroutines concurrently with asyncio.gather."""
resul…How to Share Memory Between Processes in Python with multiprocessing.Value and Array
Share a numeric value and a list-like array across multiple Python processes using multiprocessing.Value and multiprocessing.Array, with each process modifying the same memory.
import multiprocessing
def worker(shared_value, shared_array, index):
shared_value.value += 10
shared_array[index] = shared_array[index] * 2
if __name__ == "__main__":
shared_value = multiprocessing.Value("i", 5)
shared_array = multiprocessing.Array("i", [1, 2, 3, 4, 5])
processes = []
for i…How to Use ProcessPoolExecutor for CPU Parallel Map in Python
Run a function over a sequence of inputs in parallel across multiple CPU cores with ProcessPoolExecutor.map.
from concurrent.futures import ProcessPoolExecutor
import math
def compute_square(num):
return num * num
def is_prime(n):
if n < 2:
return False
for i in range(2, int(math.sqrt(n)) + 1):
if n % i == 0:
return False
return True
if __name__ == "__main__":
numbers = rang…How to Use Thread Pool Executor map for IO-Bound Tasks in Python
Run multiple I/O-bound tasks concurrently with ThreadPoolExecutor map and collect their results in order.
import time
from concurrent.futures import ThreadPoolExecutor
def io_bound_task(task_id: int) -> str:
time.sleep(0.2) # mock I/O wait
return f"Task {task_id} completed"
def main() -> None:
task_ids = [1, 2, 3, 4, 5]
with ThreadPoolExecutor(max_workers=3) as executor:
results = list(executor.…How to Use multiprocessing Pool map and starmap in Python
Parallelize functions over iterables with Pool.map, and unpack multiple arguments via Pool.starmap.
from multiprocessing import Pool
def square(x):
return x * x
def add_and_multiply(a, b, c):
return (a + b) * c
if __name__ == "__main__":
numbers = [1, 2, 3, 4, 5]
with Pool(processes=2) as pool:
squares = pool.map(square, numbers)
print(f"squares: {squares}")
starmap_arg…How to Use threading.RLock in Python
Demonstrates threading.RLock, a reentrant lock that allows the same thread to acquire it multiple times without deadlocking — essential for recursive functions sharing state across threads.
import threading
import time
lock = threading.RLock()
shared_counter = 0
def recursive_increment(value, depth):
global shared_counter
with lock:
shared_counter += 1
print(f"Depth {depth}: counter = {shared_counter}")
if depth > 1:
recursive_increment(value, depth - 1)
def…Profile Memory Usage with tracemalloc Snapshot Diff in Python
Use tracemalloc to take two memory snapshots, compute a diff, and print the top changes (size and count) by line number.
import tracemalloc
def profile_memory():
tracemalloc.start()
# Allocate some objects to track
data = [i * 2 for i in range(10000)]
text = "x" * 5000
nested = {"key": [1, 2, 3], "value": (4, 5)}
# Take first snapshot
snapshot1 = tracemalloc.take_snapshot()
# Free some mem…asyncio Condition wait notify pattern in Python
Coordinate coroutines with asyncio.Condition: workers wait for notifications and the main task notifies one or all of them.
import asyncio
async def worker(condition, name):
async with condition:
print(f"{name} waiting...")
await condition.wait()
print(f"{name} notified!")
async def main():
condition = asyncio.Condition()
tasks = [asyncio.create_task(worker(condition, f"worker-{i}")) for i in range(3…Browse by section
Each section groups closely related Python snippets.
Concurrency & performance — Python code examples
What you will find here
This page collects concurrency & performance snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.