Concurrency & performance
asyncio, threading, multiprocessing, and profiling-friendly performance patterns.
Benchmark list.append vs deque.append in Python
Measures and compares the performance of appending to a Python list versus a collections.deque using timeit.repeat, showing best and average timings.
"""Benchmark list.append vs collections.deque.append."""
import timeit
def bench(stmt, setup, repeat=5, number=1_000_000):
times = timeit.repeat(stmt, setup=setup, repeat=repeat, number=number)
return min(times), sum(times) / len(times)
if __name__ == "__main__":
number = 1_000_000
list_best, list_a…Build a Python Performance Profiler That Generates Readable Reports
Use cProfile and pstats to profile Python functions and print a sorted performance report showing the top time-consuming calls.
import cProfile
import pstats
import io
from pathlib import Path
def slow_function():
total = 0
for i in range(500_000):
total += i ** 2
return total
def fast_function():
total = sum(i * i for i in range(500_000))
return total
def profile_functions():
profiler = cProfile.Profile()
…How to Convert Data in Parallel with ThreadPoolExecutor in Python
This example demonstrates converting a list of items in parallel using ThreadPoolExecutor, showing performance gains over serial processing.
import time
from concurrent.futures import ThreadPoolExecutor
def convert_data(item):
"""Simulate a CPU/IO-bound conversion task."""
time.sleep(0.05) # simulate work
return item.upper()
if __name__ == "__main__":
items = [f"item_{i}" for i in range(20)]
start = time.perf_counter()
serial_…How to Demonstrate the GIL with Python Threads vs Processes
Measure and compare wall-clock time for CPU-bound work using Python threads (limited by the GIL) versus multiprocessing (which bypasses the GIL).
import threading
import multiprocessing
import time
import os
def cpu_heavy(n):
return sum(i * i for i in range(n))
def run_threads(n):
threads = [threading.Thread(target=cpu_heavy, args=(n,)) for _ in range(2)]
start = time.perf_counter()
for t in threads:
t.start()
for t in threads:
…How to Memoize Pure Functions with functools.lru_cache in Python
Use functools.lru_cache to memoize a pure Fibonacci function and avoid recomputing repeated values.
from functools import lru_cache
@lru_cache(maxsize=128)
def fibonacci(n: int) -> int:
"""Return the nth Fibonacci number (0-indexed) using memoization."""
if n < 2:
return n
return fibonacci(n - 1) + fibonacci(n - 2)
if __name__ == "__main__":
for i in range(10):
print(f"fibonacci({…How to Profile CPU Hot Path in Python with cProfile and sort_stats cumtime
Profile a Python function's CPU usage by running cProfile, sorting stats by cumulative time, and printing a readable report to stdout.
import cProfile
import pstats
import io
def slow_function():
total = 0
for i in range(100_000):
total += i * i
return total
def fast_function():
return sum(i for i in range(100))
def main():
slow_function()
fast_function()
if __name__ == "__main__":
profiler = cProfile.Profi…How to Reduce Instance Memory with __slots__ in Python
Demonstrates that classes with __slots__ use less memory per instance than regular classes because they skip the instance __dict__.
class SlottedPoint:
__slots__ = ('x', 'y', 'z')
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
class RegularPoint:
def __init__(self, x, y, z):
self.x = x
self.y = y
self.z = z
if __name__ == "__main__":
regular = RegularPoint(1, 2, 3)…How to Speed Up Data Filtering with Python ThreadPoolExecutor
This code compares sequential filtering of even numbers with a threaded version using ThreadPoolExecutor, showing a measurable speedup for I/O-bound work.
import time
from concurrent.futures import ThreadPoolExecutor
import random
def is_even(number):
time.sleep(0.001) # simulate work
return number % 2 == 0
def filter_even_sequential(numbers):
return [n for n in numbers if is_even(n)]
def filter_even_threaded(numbers):
with ThreadPoolExecutor(max_…How to Speed Up Downloads with ThreadPoolExecutor in Python
Compare sequential and thread-pool download loops to measure real speedup when I/O s bound.
import time
import threading
from concurrent.futures import ThreadPoolExecutor
def download_file(file_id):
"""Simulate fetching a file by sleeping briefly."""
time.sleep(0.2) # pretend network latency
return f"file_{file_id}"
def sequential_downloads(num_files):
"""Process files one at a time."""
…How to Time Code Performance with timeit in Python
Benchmark two implementations of the same logic using Python's timeit module and compare their execution speeds.
import timeit
# Implementation 1: Using a list comprehension
def list_comprehension_squares(n):
return [i ** 2 for i in range(n)]
# Implementation 2: Using a for loop with append
def loop_squares(n):
result = []
for i in range(n):
result.append(i ** 2)
return result
if __name__ == "__main__"…How to Use Array Typecodes for Compact Numeric Storage in Python
This code demonstrates how to use the `array` module with typecodes to store integers, floats, and bytes in a memory-efficient way compared to standard Python lists.
from array import array
def demonstrate_array_types():
# Compact integer arrays
small_ints = array('i', [1, 2, 3, 4, 5])
unsigned_ints = array('I', [10, 20, 30])
# Floating point arrays
floats = array('f', [1.5, 2.5, 3.5])
doubles = array('d', [1.123456789, 2.987654321])
# Charac…How to Use ThreadPoolExecutor and ProcessPoolExecutor in Python
Compares ThreadPoolExecutor and ProcessPoolExecutor by running CPU-bound and I/O-tolerant tasks over a large list, printing elapsed times and first results.
import time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import math
numbers = list(range(1, 1000001))
def compute_square(n):
return n * n
def compute_sqrt(n):
return math.sqrt(n)
def run_executor(executor, func, data):
start = time.perf_counter()
results = list(executo…How to Use ThreadPoolExecutor for Concurrent Tasks in Python
Compare sequential execution with ThreadPoolExecutor for I/O-bound tasks, measuring speedup and timing with perf_counter.
import time
import threading
from concurrent.futures import ThreadPoolExecutor
def fetch_data(index):
"""Simulate a synchronous data fetch."""
time.sleep(0.1)
return f"data-{index}"
def run_sequential(total=10):
"""Run tasks one after another."""
start = time.perf_counter()
results = [fetch…How to Use a Weakref Cache to Avoid Memory Leaks in Python
This code demonstrates building a value cache with weakref.WeakValueDictionary so objects can be garbage collected when no longer referenced, preventing memory leaks.
import weakref
import gc
class ExpensiveObject:
def __init__(self, name):
self.name = name
def __repr__(self):
return f"ExpensiveObject('{self.name}')"
class ObjectCache:
def __init__(self):
self._cache = weakref.WeakValueDictionary()
def get_or_create(self, name):
…How to Use bisect.insort in Python to Maintain a Sorted List
Insert items into an already sorted list using Python's bisect.insort to keep it sorted efficiently in O(n) time.
import bisect
def maintain_sorted_list():
data = [3, 1, 4, 1, 5, 9, 2, 6]
sorted_list = []
for num in data:
bisect.insort(sorted_list, num)
print("Original data:", data)
print("Sorted list maintained with insort:", sorted_list)
# Insert new values to maintain sorted orde…How to Use functools.cache for Unbounded Memoization in Python
Speed up repeated recursive calls by memoizing function results with Python's built-in functools.cache decorator.
```python
import functools
import time
@functools.cache
def fib(n):
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
if __name__ == "__main__":
start = time.perf_counter()
result = fib(30)
elapsed = time.perf_counter() - start
print(f"fib(30) = {result}")
print(f"computed in {…How to Use pool.map for CPU-Bound Tasks in Python
Distribute CPU-intensive functions across processes with multiprocessing.Pool.map and measure the performance gain.
from multiprocessing import Pool
import time
def cpu_bound_task(n):
"""Mock CPU-bound work: compute sum of squares."""
total = 0
for i in range(n):
total += i * i
return total
if __name__ == "__main__":
numbers = [10_000_000, 12_000_000, 8_000_000, 15_000_000]
start = time.perf_count…How to Use uvloop Faster Event Loop
Install uvloop at startup to replace asyncio's default event loop with a faster libuv-based one, with a graceful fallback when it's unavailable.
import asyncio
try:
import uvloop
uvloop.install()
USING_UVLOOP = True
except ImportError:
USING_UVLOOP = False
async def fetch_data(index):
await asyncio.sleep(0.01)
return f"data-{index}"
async def main():
tasks = [fetch_data(i) for i in range(10)]
results = await asyncio.gather(*…How to Vectorize a Function with a Pure Python Fallback
Create a decorator that calls a scalar function directly for a single value and routes list inputs to a pure-Python fallback for vectorized processing without NumPy.
import math
def fallback_vectorize(func, fallback=None):
"""Vectorize a scalar function with a pure-Python fallback for lists."""
if fallback is None:
fallback = lambda x: [func(i) for i in x]
def wrapped(*args):
if len(args) == 1 and isinstance(args[0], (list, tuple)):
retur…How to use ThreadPoolExecutor for concurrent tasks in Python
Run blocking functions in parallel with ThreadPoolExecutor and as_completed, cutting total runtime from 5 sequential sleeps to about 1 second.
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
def fetch_data(item):
"""Simulate a slow operation with a fixed delay."""
time.sleep(0.2)
return item * 2
def main():
items = [1, 2, 3, 4, 5]
start = time.perf_counter()
with ThreadPoolExecutor(max_workers=3) as ex…Merge K Sorted Lists in Python with heapq
Merge k sorted lists into one sorted list in O(N log k) time using a min-heap of current elements.
import heapq
def merge_k_sorted_lists(lists):
heap = []
for i, lst in enumerate(lists):
if lst: # only push non-empty lists
heapq.heappush(heap, (lst[0], i, 0))
result = []
while heap:
val, list_idx, elem_idx = heapq.heappop(heap)
result.append(val)
if elem…Profile Memory Usage with tracemalloc Snapshot Diff in Python
Use tracemalloc to take two memory snapshots, compute a diff, and print the top changes (size and count) by line number.
import tracemalloc
def profile_memory():
tracemalloc.start()
# Allocate some objects to track
data = [i * 2 for i in range(10000)]
text = "x" * 5000
nested = {"key": [1, 2, 3], "value": (4, 5)}
# Take first snapshot
snapshot1 = tracemalloc.take_snapshot()
# Free some mem…Browse by section
Each section groups closely related Python snippets.
Concurrency & performance — Python code examples
What you will find here
This page collects concurrency & performance snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.