Skip to content

Multiprocessing

In one line

multiprocessing runs separate processes with independent memory, enabling true multi-core CPU parallelism.

In simple words

The multiprocessing module creates separate OS processes, each with its own Python interpreter and memory space. Since each process has its own GIL, you get true parallelism for CPU-bound work.

Processes don't share memory by default, so you use Queue, Pipe, or Value/Array (shared memory) to communicate. There's overhead in starting processes and serializing data, so it's only worth it for substantial CPU work.

concurrent.futures.ProcessPoolExecutor provides a clean high-level API similar to ThreadPoolExecutor but with processes.

Example

python
from multiprocessing import Pool
import math

def is_prime(n):
    if n < 2:
        return False
    for i in range(2, int(math.sqrt(n)) + 1):
        if n % i == 0:
            return False
    return True

if __name__ == "__main__":
    numbers = range(10_000, 10_100)
    with Pool(4) as pool:  # 4 processes
        results = pool.map(is_prime, numbers)
    primes = [n for n, p in zip(numbers, results) if p]
    print(f"Found {len(primes)} primes")

Pool(4) creates 4 worker processes. pool.map distributes work across them — true parallel CPU execution, bypassing the GIL.

Common confusions

  • Confused with: threading

    The difference: multiprocessing uses separate processes (true parallelism, no GIL, separate memory). threading uses threads within one process (GIL limits CPU parallelism, shared memory). Use multiprocessing for CPU-bound, threading for I/O-bound.

Related terms

Related Resources

Related Lessons

Decode error messages

Plain-English explanations of common errors — what they mean, why they happen, and how to fix them.

View errors

← Back to glossary