Reference library

Concurrency & performance

asyncio, threading, multiprocessing, and profiling-friendly performance patterns.

6 matches

How to Parse JSON Files in Parallel with Python ThreadPoolExecutor

Load and transform JSON records from multiple files concurrently using ThreadPoolExecutor for faster I/O-bound parsing.

import time
from concurrent.futures import ThreadPoolExecutor
import json

def load_json_file(path):
    with open(path, 'r') as f:
        return json.load(f)

def transform_record(record):
    record['full_name'] = f"{record.pop('first_name', '')} {record.pop('last_name', '')}".strip()
    record['score'] = int(reco…

17 0 Open

Concurrency & performance medium

How to Run Coroutines Concurrently with asyncio.gather in Python

Run multiple async coroutines concurrently and collect their results in the order they were passed.

asyncio concurrency gather

Python

import asyncio


async def fetch_data(name: str, delay: float) -> str:
    """Simulate an async operation (e.g., API call) with a delay."""
    await asyncio.sleep(delay)
    return f"{name} data (after {delay}s)"


async def main() -> None:
    """Run multiple coroutines concurrently with asyncio.gather."""
    resul…

15 0 Open

Concurrency & performance medium

How to Share Memory Between Processes in Python with multiprocessing.Value and Array

Share a numeric value and a list-like array across multiple Python processes using multiprocessing.Value and multiprocessing.Array, with each process modifying the same memory.

multiprocessing shared-memory concurrency

Python

import multiprocessing

def worker(shared_value, shared_array, index):
    shared_value.value += 10
    shared_array[index] = shared_array[index] * 2

if __name__ == "__main__":
    shared_value = multiprocessing.Value("i", 5)
    shared_array = multiprocessing.Array("i", [1, 2, 3, 4, 5])

    processes = []
    for i…

14 0 Open

Concurrency & performance medium

How to Use ProcessPoolExecutor for CPU Parallel Map in Python

Run a function over a sequence of inputs in parallel across multiple CPU cores with ProcessPoolExecutor.map.

concurrency processpoolexecutor parallelism

Python

from concurrent.futures import ProcessPoolExecutor
import math

def compute_square(num):
    return num * num

def is_prime(n):
    if n < 2:
        return False
    for i in range(2, int(math.sqrt(n)) + 1):
        if n % i == 0:
            return False
    return True

if __name__ == "__main__":
    numbers = rang…

11 0 Open

Concurrency & performance medium

How to Use Thread Pool Executor map for IO-Bound Tasks in Python

Run multiple I/O-bound tasks concurrently with ThreadPoolExecutor map and collect their results in order.

threadpool concurrency io-bound

Python

import time
from concurrent.futures import ThreadPoolExecutor

def io_bound_task(task_id: int) -> str:
    time.sleep(0.2)  # mock I/O wait
    return f"Task {task_id} completed"

def main() -> None:
    task_ids = [1, 2, 3, 4, 5]
    with ThreadPoolExecutor(max_workers=3) as executor:
        results = list(executor.…

12 0 Open

Concurrency & performance medium

How to Use multiprocessing Pool map and starmap in Python

Parallelize functions over iterables with Pool.map, and unpack multiple arguments via Pool.starmap.

multiprocessing parallelism pool

Python

from multiprocessing import Pool


def square(x):
    return x * x


def add_and_multiply(a, b, c):
    return (a + b) * c


if __name__ == "__main__":
    numbers = [1, 2, 3, 4, 5]
    with Pool(processes=2) as pool:
        squares = pool.map(square, numbers)
        print(f"squares: {squares}")

        starmap_arg…

14 0 Open

Browse by section

Each section groups closely related Python snippets.