How to parallel map embeddings with a thread pool in Python

Run embedding computations in parallel using ThreadPoolExecutor, collect results into a dict keyed by the original item.

Medium Python 3.10+ Aug 9, 2026 AI & LLM integration patterns 15 views 0 copies

concurrency threadpool embeddings llm parallel

Python code

24 lines

Python 3.10+

import threading
from concurrent.futures import ThreadPoolExecutor
import time


def compute_embedding(item: int) -> tuple[int, int]:
    time.sleep(0.05)  # Simulate embedding work
    return item, item * 10


def parallel_map_embed(items, max_workers=3):
    results = {}
    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        futures = {executor.submit(compute_embedding, x): x for x in items}
        for future, original in futures.items():
            idx, embedding = future.result()
            results[idx] = embedding
    return results


if __name__ == "__main__":
    data = [1, 2, 3, 4]
    output = parallel_map_embed(data)
    print({k: v for k, v in sorted(output.items())})

Output

stdout

{1: 10, 2: 20, 3: 30, 4: 40}

How it works

ThreadPoolExecutor creates a pool of worker threads that run compute_embedding concurrently. Using executor.submit schedules each call and returns a Future object. The with block ensures all threads shut down cleanly, and future.result() blocks until that task finishes, retrieving its return value. Dict unpacking preserves the mapping so results stay keyed by original item even when threads finish out of order.

Common mistakes

Calling executor.map instead of submit when you need to capture the original input with the result
Forgetting to call result() before the executor context exits, which can lose results
Using too many workers for CPU-bound work, causing thread overhead without speedup

Variations

Use executor.map for simple ordered results when you don't need to keep track of original keys
Switch to ProcessPoolExecutor for CPU-heavy embedding functions that release the GIL

Real-world use cases

Batch-embedding text chunks in an LLM RAG pipeline while keeping a mapping back to source documents.
Running multiple embedding API calls concurrently inside a feature store to avoid sequential latency.
Parallelizing embedding generation for a large dataset before inserting into a vector database.

How to parallel map embeddings with a thread pool in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from AI & LLM integration patterns

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from AI & LLM integration patterns

Keep learning

Tutorials

Quizzes