How to Share Memory Between Processes in Python with multiprocessing.Value and Array
Share a numeric value and a list-like array across multiple Python processes using multiprocessing.Value and multiprocessing.Array, with each process modifying the same memory.
Python code
21 linesimport multiprocessing
def worker(shared_value, shared_array, index):
shared_value.value += 10
shared_array[index] = shared_array[index] * 2
if __name__ == "__main__":
shared_value = multiprocessing.Value("i", 5)
shared_array = multiprocessing.Array("i", [1, 2, 3, 4, 5])
processes = []
for i in range(5):
p = multiprocessing.Process(target=worker, args=(shared_value, shared_array, i))
processes.append(p)
p.start()
for p in processes:
p.join()
print(f"Shared value: {shared_value.value}")
print(f"Shared array: {list(shared_array)}")Output
Shared value: 55
Shared array: [2, 4, 6, 8, 10]How it works
multiprocessing.Value and multiprocessing.Array create shared memory objects that multiple processes can read and write. Each process receives the same shared objects via the args tuple, so changes are immediately visible to all processes. The join() calls ensure the main process waits for all workers to finish before reading the final values. The 'i' type code indicates a signed C integer, and array indices map to shared memory locations directly. There is no need for locks here because each process writes to a different array index, avoiding race conditions.
Common mistakes
- Forgetting to call `join()` and reading shared values before all processes finish
- Using regular Python lists or ints instead of `multiprocessing.Value`/`Array`, which are not shared between processes
- Choosing the wrong type code (e.g., `'f'` for float, `'d'` for double) that doesn't match your data
- Assuming concurrent writes to the same array index are safe — use a lock for shared-write scenarios
Variations
- Use `multiprocessing.Manager().Value()` and `.list()` for more flexible but slower shared objects
- Pass a `multiprocessing.Lock` to protect simultaneous updates to the same array index or value
Real-world use cases
- A parallel data-processing pipeline where each worker updates a shared progress counter or accumulator.
- A multi-process simulation where each process computes partial results stored in a shared array for final aggregation.
- A worker pool that aggregates logs or metrics into a shared structure without a central database.
Sponsored
More from Concurrency & performance
- Benchmark list.append vs deque.append in Python medium
- Build a Python Performance Profiler That Generates Readable Reports medium
- Graceful Shutdown Executor Context Manager in Python medium
- How to Build a Producer-Consumer Pattern with asyncio.Queue in Python medium
- How to Cancel an asyncio Task with Graceful Cleanup in Python medium
- How to Convert Data in Parallel with ThreadPoolExecutor in Python easy
Keep learning
Related tutorials and quizzes for this topic.