Python concurrent.futures and Parallel Work

The essentials

Quick reference

One focused task per row. Jump to the related section for complete, working examples.

Use	Syntax	Examples
Create a thread pool	`with ThreadPoolExecutor(max_workers=8) as executor: results = list(executor.map(fetch, urls))`	View examples
Create a process pool	`with ProcessPoolExecutor(max_workers=4) as executor: results = list(executor.map(cpu_task, values))`	View examples
Create an interpreter pool	`with InterpreterPoolExecutor(max_workers=4) as executor: results = list(executor.map(cpu_task, values))`	View examples
Submit one call	`future = executor.submit(function, argument)`	View examples
Retrieve a result	`value = future.result(timeout=5)`	View examples
Add a completion callback	`future.add_done_callback(record_completion)`	View examples
Map in input order	`results = executor.map(function, items)`	View examples
Bound submitted map work	`results = executor.map(function, items, buffersize=16)`	View examples
Batch process-pool items	`results = executor.map(function, items, chunksize=64)`	View examples
Consume completed futures	`results = [future.result() for future in as_completed(futures)]`	View examples
Wait for a condition	`done, pending = wait(futures, timeout=5, return_when=FIRST_EXCEPTION)`	View examples
Limit a result wait	`future.result(timeout=2)`	View examples
Cancel pending work	`cancelled = future.cancel()`	View examples
Shut down an executor	`executor.shutdown(wait=True, cancel_futures=True)`	View examples
Guard process entry	`if __name__ == '__main__': main()`	View examples
Choose a process context	`ProcessPoolExecutor(mp_context=multiprocessing.get_context('spawn'))`	View examples

concurrent.futures gives threads, processes, and—on Python 3.14 and later—isolated interpreters one submission-and-result model, but those workers do not have the same cost or sharing rules. Choose the executor from measured workload behavior: threads commonly overlap blocking I/O, while processes or interpreters can run suitable CPU-bound Python on multiple cores. Bound both worker count and outstanding work, preserve input identity when completion order differs, and design timeout, cancellation, exception, and shutdown behavior before increasing concurrency.

Step by step

Detailed examples

Choose workers by blocking, CPU, and isolation behavior

ThreadPoolExecutor workers share memory and are inexpensive, making them a common fit for independent blocking I/O. On ordinary GIL-enabled CPython builds, threads do not make pure Python bytecode CPU-parallel, though native code that releases the GIL and free-threaded builds can differ. ProcessPoolExecutor provides separate memory and multi-core execution at serialization and startup cost. InterpreterPoolExecutor, added in Python 3.14, also provides separate GILs but isolates runtime state and serializes submitted callables, arguments, and results. Benchmark a realistic workload instead of assuming more workers are faster.

Reuse a bounded thread pool for independent calls

from concurrent.futures import ThreadPoolExecutor

def normalize(value: str) -> str:
    return value.strip().casefold()

with ThreadPoolExecutor(max_workers=2, thread_name_prefix='normalize') as executor:
    results = list(executor.map(normalize, [' READY ', ' Done ']))

print(results)

Output

['ready', 'done']

Back to quick reference ↑

Treat each Future as a value-or-exception boundary

submit() returns immediately with a Future. result() returns the callable's value or re-raises its exception in the waiting thread, so retrieving every submitted future is essential for observing failures. Keep the mapping from Future to input when reporting errors. Callbacks should be small and nonblocking; use them for notification or bookkeeping, not a second hidden workflow. Never make a pool task wait on another task from the same saturated pool because that can deadlock.

Retrieve success and surface a task exception

from concurrent.futures import ThreadPoolExecutor

def divide(pair: tuple[int, int]) -> float:
    left, right = pair
    return left / right

with ThreadPoolExecutor(max_workers=2) as executor:
    good = executor.submit(divide, (6, 3))
    bad = executor.submit(divide, (1, 0))
    print(good.result())
    try:
        bad.result()
    except ZeroDivisionError as error:
        print(type(error).__name__)

Output

2.0
ZeroDivisionError

Back to quick reference ↑

Use map when output order should match input order

Executor.map() may run calls simultaneously but yields results in the original iterable order; a slow early item can therefore delay later completed results. Process pools can use chunksize to reduce per-item transport overhead, while it has no effect for thread or interpreter pools. Python 3.14 adds buffersize to limit submitted results that have not yet been yielded. On older versions, implement bounded submission explicitly instead of passing that keyword.

Preserve order despite different completion times

import time
from concurrent.futures import ThreadPoolExecutor

def delayed_square(item: tuple[int, float]) -> int:
    value, delay = item
    time.sleep(delay)
    return value * value

items = [(3, 0.03), (2, 0.02), (1, 0.01)]
with ThreadPoolExecutor(max_workers=3) as executor:
    print(list(executor.map(delayed_square, items)))

Output

[9, 4, 1]

Back to quick reference ↑

Use as_completed for responsive per-item handling

as_completed() yields each Future after it finishes, which supports progress reporting and early handling of independent results. Completion order is intentionally nondeterministic and must not be used as a business ordering guarantee. Associate futures with stable input keys, handle exceptions individually, and sort or otherwise reconcile collected results when downstream output must be reproducible.

Retain input identity and canonicalize final output

from concurrent.futures import ThreadPoolExecutor, as_completed

def measure(label: str) -> tuple[str, int]:
    return label, len(label)

with ThreadPoolExecutor(max_workers=3) as executor:
    futures = {executor.submit(measure, label): label for label in ['pear', 'fig', 'apple']}
    completed = [future.result() for future in as_completed(futures)]

print(sorted(completed))

Output

[('apple', 5), ('fig', 3), ('pear', 4)]

Back to quick reference ↑

Separate waiting deadlines from work cancellation

A timeout on result(), wait(), or as_completed() limits how long the caller waits; it does not terminate a running callable. Future.cancel() succeeds only before execution starts. Cooperative thread cancellation therefore needs application-owned signals that tasks check. The executor context manager waits for submitted work on exit. shutdown(cancel_futures=True) can discard queued tasks but still allows already-running tasks to finish, so design bounded operations inside each task as well.

Cancel a task while the only worker is occupied

from concurrent.futures import ThreadPoolExecutor
from threading import Event

started = Event()
release = Event()

def blocking_task() -> str:
    started.set()
    release.wait()
    return 'first'

with ThreadPoolExecutor(max_workers=1) as executor:
    running = executor.submit(blocking_task)
    started.wait(timeout=1)
    pending = executor.submit(lambda: 'second')
    print(pending.cancel())
    release.set()
    print(running.result(timeout=1))

Output

True
first

Back to quick reference ↑

Design process tasks as importable, serializable units

ProcessPoolExecutor transports callables, arguments, and return values through multiprocessing, so they must be picklable; lambdas, nested functions, open handles, and many live runtime objects are poor boundaries. Worker subprocesses must be able to import __main__, which makes a main guard mandatory and means interactive definitions should not be expected to work. Start-method defaults vary by platform and release; Python 3.14 changed the default away from fork. Select an explicit multiprocessing context only when the application has a tested reason, and never submit executor or Future operations from a process-pool task because the documented result is deadlock.

Run a top-level CPU function behind a main guard

from concurrent.futures import ProcessPoolExecutor

def square(value: int) -> int:
    return value * value

def main() -> None:
    with ProcessPoolExecutor(max_workers=2) as executor:
        print(list(executor.map(square, [2, 3, 4])))

if __name__ == '__main__':
    main()

Output

[4, 9, 16]

Back to quick reference ↑

Local code tester

Coordinate deterministic thread-pool results

Submit small in-memory tasks, preserve their input identity, and print a stable result without processes or external services.

Runs in your browser

Python

from concurrent.futures import ThreadPoolExecutor, as_completed

def normalize(item: tuple[str, str]) -> tuple[str, str]:
    key, value = item
    return key, value.strip().casefold()

items = [('primary', ' READY '), ('secondary', ' Done ')]
with ThreadPoolExecutor(max_workers=2) as executor:
    futures = [executor.submit(normalize, item) for item in items]
    results = dict(future.result() for future in as_completed(futures))

for key in sorted(results):
    print(f'{key}: {results[key]}')

Output

Press Run to load Python locally.

Sources and further reading

References

Authoritative documentation used to verify and expand this cheat sheet.

Help us improve

Found a typo or missing example?

Tell us what would make this cheat sheet clearer, more complete, or more useful.

Share feedback

Quick reference

Detailed examples

Choose workers by blocking, CPU, and isolation behavior

Treat each Future as a value-or-exception boundary

Use map when output order should match input order

Use as_completed for responsive per-item handling

Separate waiting deadlines from work cancellation

Design process tasks as importable, serializable units

Coordinate deterministic thread-pool results

More Python cheat sheets

Python asyncio and await Cheat Sheet

Python Hashing, HMAC, and Secrets Cheat Sheet

Python Memory, Garbage Collection, and Weak References Cheat Sheet

References

Found a typo or missing example?