Reference library

Concurrency & performance

asyncio, threading, multiprocessing, and profiling-friendly performance patterns.

6 matches

How to Convert Data in Parallel with ThreadPoolExecutor in Python

This example demonstrates converting a list of items in parallel using ThreadPoolExecutor, showing performance gains over serial processing.

concurrency threadpoolexecutor parallelism

Python

import time
from concurrent.futures import ThreadPoolExecutor


def convert_data(item):
    """Simulate a CPU/IO-bound conversion task."""
    time.sleep(0.05)  # simulate work
    return item.upper()


if __name__ == "__main__":
    items = [f"item_{i}" for i in range(20)]

    start = time.perf_counter()
    serial_…

16 0 Open

Concurrency & performance easy

How to Use ThreadPoolExecutor in Python for Parallel Processing

Use ThreadPoolExecutor with executor.map to run a function over many inputs concurrently and collect ordered results.

concurrency threadpoolexecutor parallel

Python

def worker(item):
    return item * item

if __name__ == "__main__":
    from concurrent.futures import ThreadPoolExecutor
    numbers = list(range(1, 11))
    with ThreadPoolExecutor(max_workers=4) as executor:
        results = list(executor.map(worker, numbers))
    print("Input:  ", numbers)
    print("Results:", …

13 0 Open

Concurrency & performance easy

How to Use pool.map for CPU-Bound Tasks in Python

Distribute CPU-intensive functions across processes with multiprocessing.Pool.map and measure the performance gain.

multiprocessing pool cpu-bound

Python

from multiprocessing import Pool
import time

def cpu_bound_task(n):
    """Mock CPU-bound work: compute sum of squares."""
    total = 0
    for i in range(n):
        total += i * i
    return total

if __name__ == "__main__":
    numbers = [10_000_000, 12_000_000, 8_000_000, 15_000_000]

    start = time.perf_count…

11 0 Open

Concurrency & performance easy

How to Validate Data with ThreadPoolExecutor in Python

This code shows how to validate a list of numbers concurrently using ThreadPoolExecutor, dramatically speeding up slow validation tasks by running them in parallel threads.

concurrency threadpool validation

Python

import time
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass


@dataclass
class Result:
    is_valid: bool
    value: int


def validate(value: int) -> Result:
    time.sleep(0.1)  # simulate slow validation (API call, DB check)
    return Result(is_valid=0 < value < 100, value=value…

12 0 Open

Concurrency & performance easy

How to spawn multiple worker processes in Python with multiprocessing.Process

Spawns three separate worker processes using multiprocessing.Process, runs them concurrently, and waits for all to finish before printing a completion message.

multiprocessing parallel concurrency

Python

import multiprocessing
import time

def worker(name):
    print(f"Worker {name} started")
    time.sleep(1)
    print(f"Worker {name} finished")
    return name

if __name__ == "__main__":
    processes = []
    for i in range(3):
        p = multiprocessing.Process(target=worker, args=(i,))
        processes.append(p…

14 0 Open

Concurrency & performance easy

How to use ThreadPoolExecutor for concurrent tasks in Python

Run blocking functions in parallel with ThreadPoolExecutor and as_completed, cutting total runtime from 5 sequential sleeps to about 1 second.

concurrency threadpoolexecutor parallel

Python

import time
from concurrent.futures import ThreadPoolExecutor, as_completed


def fetch_data(item):
    """Simulate a slow operation with a fixed delay."""
    time.sleep(0.2)
    return item * 2


def main():
    items = [1, 2, 3, 4, 5]
    start = time.perf_counter()

    with ThreadPoolExecutor(max_workers=3) as ex…

14 0 Open

Browse by section

Each section groups closely related Python snippets.