Concurrency & performance
asyncio, threading, multiprocessing, and profiling-friendly performance patterns.
Benchmark list.append vs deque.append in Python
Measures and compares the performance of appending to a Python list versus a collections.deque using timeit.repeat, showing best and average timings.
"""Benchmark list.append vs collections.deque.append."""
import timeit
def bench(stmt, setup, repeat=5, number=1_000_000):
times = timeit.repeat(stmt, setup=setup, repeat=repeat, number=number)
return min(times), sum(times) / len(times)
if __name__ == "__main__":
number = 1_000_000
list_best, list_a…How to Convert Data in Parallel with ThreadPoolExecutor in Python
This example demonstrates converting a list of items in parallel using ThreadPoolExecutor, showing performance gains over serial processing.
import time
from concurrent.futures import ThreadPoolExecutor
def convert_data(item):
"""Simulate a CPU/IO-bound conversion task."""
time.sleep(0.05) # simulate work
return item.upper()
if __name__ == "__main__":
items = [f"item_{i}" for i in range(20)]
start = time.perf_counter()
serial_…How to Share Memory Between Processes in Python with multiprocessing.Value and Array
Share a numeric value and a list-like array across multiple Python processes using multiprocessing.Value and multiprocessing.Array, with each process modifying the same memory.
import multiprocessing
def worker(shared_value, shared_array, index):
shared_value.value += 10
shared_array[index] = shared_array[index] * 2
if __name__ == "__main__":
shared_value = multiprocessing.Value("i", 5)
shared_array = multiprocessing.Array("i", [1, 2, 3, 4, 5])
processes = []
for i…How to Share a Dict and List Between Processes with multiprocessing Manager in Python
This code demonstrates how to share a dictionary and a list between multiple processes using multiprocessing.Manager, enabling safe concurrent updates.
import multiprocessing as mp
def worker(shared_dict, shared_list, name):
shared_dict[name] = name.upper()
shared_list.append(name)
print(f"{name} added to shared structures")
def main():
with mp.Manager() as manager:
shared_dict = manager.dict()
shared_list = manager.list()
…How to Use Array Typecodes for Compact Numeric Storage in Python
This code demonstrates how to use the `array` module with typecodes to store integers, floats, and bytes in a memory-efficient way compared to standard Python lists.
from array import array
def demonstrate_array_types():
# Compact integer arrays
small_ints = array('i', [1, 2, 3, 4, 5])
unsigned_ints = array('I', [10, 20, 30])
# Floating point arrays
floats = array('f', [1.5, 2.5, 3.5])
doubles = array('d', [1.123456789, 2.987654321])
# Charac…How to Use ProcessPoolExecutor for CPU Parallel Map in Python
Run a function over a sequence of inputs in parallel across multiple CPU cores with ProcessPoolExecutor.map.
from concurrent.futures import ProcessPoolExecutor
import math
def compute_square(num):
return num * num
def is_prime(n):
if n < 2:
return False
for i in range(2, int(math.sqrt(n)) + 1):
if n % i == 0:
return False
return True
if __name__ == "__main__":
numbers = rang…How to Use ThreadPoolExecutor and ProcessPoolExecutor in Python
Compares ThreadPoolExecutor and ProcessPoolExecutor by running CPU-bound and I/O-tolerant tasks over a large list, printing elapsed times and first results.
import time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
import math
numbers = list(range(1, 1000001))
def compute_square(n):
return n * n
def compute_sqrt(n):
return math.sqrt(n)
def run_executor(executor, func, data):
start = time.perf_counter()
results = list(executo…How to Use bisect.insort in Python to Maintain a Sorted List
Insert items into an already sorted list using Python's bisect.insort to keep it sorted efficiently in O(n) time.
import bisect
def maintain_sorted_list():
data = [3, 1, 4, 1, 5, 9, 2, 6]
sorted_list = []
for num in data:
bisect.insort(sorted_list, num)
print("Original data:", data)
print("Sorted list maintained with insort:", sorted_list)
# Insert new values to maintain sorted orde…How to Use multiprocessing Pool map and starmap in Python
Parallelize functions over iterables with Pool.map, and unpack multiple arguments via Pool.starmap.
from multiprocessing import Pool
def square(x):
return x * x
def add_and_multiply(a, b, c):
return (a + b) * c
if __name__ == "__main__":
numbers = [1, 2, 3, 4, 5]
with Pool(processes=2) as pool:
squares = pool.map(square, numbers)
print(f"squares: {squares}")
starmap_arg…How to Validate Data with ThreadPoolExecutor in Python
This code shows how to validate a list of numbers concurrently using ThreadPoolExecutor, dramatically speeding up slow validation tasks by running them in parallel threads.
import time
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass
@dataclass
class Result:
is_valid: bool
value: int
def validate(value: int) -> Result:
time.sleep(0.1) # simulate slow validation (API call, DB check)
return Result(is_valid=0 < value < 100, value=value…How to Vectorize a Function with a Pure Python Fallback
Create a decorator that calls a scalar function directly for a single value and routes list inputs to a pure-Python fallback for vectorized processing without NumPy.
import math
def fallback_vectorize(func, fallback=None):
"""Vectorize a scalar function with a pure-Python fallback for lists."""
if fallback is None:
fallback = lambda x: [func(i) for i in x]
def wrapped(*args):
if len(args) == 1 and isinstance(args[0], (list, tuple)):
retur…Merge K Sorted Lists in Python with heapq
Merge k sorted lists into one sorted list in O(N log k) time using a min-heap of current elements.
import heapq
def merge_k_sorted_lists(lists):
heap = []
for i, lst in enumerate(lists):
if lst: # only push non-empty lists
heapq.heappush(heap, (lst[0], i, 0))
result = []
while heap:
val, list_idx, elem_idx = heapq.heappop(heap)
result.append(val)
if elem…Using a Python Generator Instead of a List to Save Memory
Compare a list approach with a generator to stream values lazily, avoiding memory-heavy storage of large sequences.
def fibonacci_generator(limit):
a, b = 0, 1
count = 0
while count < limit:
yield a
a, b = b, a + b
count += 1
def sum_first_n(generator, n):
total = 0
for i, value in enumerate(generator):
if i >= n:
break
total += value
return total
if __…Browse by section
Each section groups closely related Python snippets.
Concurrency & performance — Python code examples
What you will find here
This page collects concurrency & performance snippets — short, copy-ready Python you can paste into our free online IDE and run without installing anything. Each sample includes a plain-English explanation and the full source code.
Samples vs tutorials and challenges
Samples are quick reference — one concept per page. For step-by-step teaching, use our Python tutorials. To test yourself, try quizzes or coding challenges. Clean up style with the Python formatter.