ProcessPoolExecutor is a convenient way to spread CPU-bound Python work across multiple processes. Most of the time, its normal shutdown behavior is exactly what an application wants: stop accepting work, let running tasks finish, and clean up worker processes.

Some failures do not fit that model.

A worker can become stuck in native code, wait indefinitely on an external resource, or execute a task whose runtime has exceeded the application’s operational deadline. At that point, cancelling a Future may not stop work that is already running, and waiting for an orderly pool shutdown may take too long.

Python 3.14 adds two explicit escape hatches to ProcessPoolExecutor:

executor.terminate_workers()
executor.kill_workers()

Both stop the executor’s living workers and perform executor shutdown. The important difference is how forcefully the operating system is asked to stop those processes.

These APIs make hard-stop behavior much easier to express, but they do not make abrupt process termination safe. Correct use requires understanding what can be abandoned when a worker disappears.

Start with normal shutdown

A process pool should normally be allowed to finish its work.

from concurrent.futures import ProcessPoolExecutor


def transform(value):
    return value * value


with ProcessPoolExecutor() as executor:
    results = list(executor.map(transform, range(100)))

Leaving the with block shuts the executor down cleanly. That is preferable because workers get a chance to complete their current tasks and exit through the normal lifecycle.

Hard termination belongs on an exceptional path. If every request routinely ends with terminate_workers(), the application probably needs a different task model rather than a stronger shutdown primitive.

Cancellation is not process termination

A common mistake is to treat Future.cancel() as a way to interrupt arbitrary running work.

future = executor.submit(expensive_job, payload)
future.cancel()

Cancellation succeeds only if the future can still be cancelled. Once its call is running, cancellation does not provide a general mechanism for asynchronously stopping the Python code executing in another process.

That distinction matters when implementing deadlines:

from concurrent.futures import ProcessPoolExecutor, TimeoutError

executor = ProcessPoolExecutor(max_workers=4)
future = executor.submit(expensive_job, payload)

try:
    result = future.result(timeout=10)
except TimeoutError:
    ...

The timeout limits how long the caller waits for the result. It does not mean the worker stopped after ten seconds.

If the application must stop the process that is still executing the task, Python 3.14’s new worker-control methods provide a pool-level mechanism for doing so.

Terminate the pool with terminate_workers()

terminate_workers() attempts to terminate every living worker in the executor:

from concurrent.futures import ProcessPoolExecutor, TimeoutError

executor = ProcessPoolExecutor(max_workers=4)
future = executor.submit(expensive_job, payload)

try:
    result = future.result(timeout=10)
except TimeoutError:
    executor.terminate_workers()
    raise
else:
    executor.shutdown()

Internally, the executor calls Process.terminate() on its workers and also performs executor shutdown to release the pool’s other resources.

On POSIX systems, multiprocessing.Process.terminate() uses SIGTERM. On Windows, it uses TerminateProcess().

Those platform semantics are not equivalent to a cooperative Python cancellation protocol. In particular, Python’s multiprocessing documentation warns that exit handlers and finally clauses are not executed when a process is terminated this way.

That means code such as this cannot be relied on for cleanup after forced termination:

def job(path):
    handle = open(path, "w")
    try:
        write_output(handle)
    finally:
        handle.close()

The operating system will reclaim process-owned resources such as memory and file descriptors, but application-level cleanup logic may never run.

Escalate with kill_workers() when necessary

kill_workers() is the stronger sibling:

executor.kill_workers()

It calls Process.kill() for living workers and then shuts down the executor.

On POSIX, Process.kill() uses SIGKILL. A process cannot catch or ignore SIGKILL, so this is useful when a worker does not respond to ordinary termination.

That strength also removes any opportunity for cooperative cleanup inside the target process.

A useful operational model is:

normal completion
    -> cooperative application cancellation
        -> terminate_workers()
            -> kill_workers()

Not every system needs every stage, but escalation should generally move from the least destructive mechanism that can satisfy the deadline to the most forceful one.

The executor is finished after either call

Neither method resets the pool.

After calling terminate_workers() or kill_workers(), do not submit new tasks to that executor:

executor.terminate_workers()

# Wrong: this executor has been shut down.
executor.submit(expensive_job, another_payload)

If the application needs to continue processing work, construct a new executor after deciding that doing so is safe:

executor.terminate_workers()
executor = ProcessPoolExecutor(max_workers=4)

That recreation should happen at an architectural boundary. A helper function silently replacing a dead pool can hide failures and make it unclear which tasks were lost.

A timeout does not identify the guilty worker

The new methods operate on the entire pool, not on the process executing one particular Future.

Suppose four jobs are running:

worker 1: task A -- stuck
worker 2: task B -- healthy
worker 3: task C -- healthy
worker 4: task D -- healthy

Calling terminate_workers() because task A exceeded its deadline attempts to terminate all four living workers. Healthy tasks B, C, and D can be interrupted too.

This is one of the most important design constraints of the API.

If tasks need independent hard deadlines without collateral termination, a shared long-lived process pool may be the wrong isolation boundary. Consider separate worker processes, smaller dedicated pools, or an external job system where each unit of work has an explicit process lifecycle.

Treat interrupted output as uncommitted

Abrupt termination can occur at any instruction boundary from the application’s point of view.

A worker might have written half a file:

def render_report(target, data):
    with open(target, "wb") as f:
        f.write(build_header(data))
        f.write(build_body(data))

If the process dies between writes, the target can exist but be invalid.

A safer pattern is to write to a temporary location and publish atomically only after success:

import os
from pathlib import Path


def render_report(target, data):
    target = Path(target)
    temporary = target.with_suffix(target.suffix + ".tmp")

    with temporary.open("wb") as f:
        f.write(build_header(data))
        f.write(build_body(data))
        f.flush()
        os.fsync(f.fileno())

    os.replace(temporary, target)

If a worker is killed before os.replace(), consumers do not mistake the partial temporary file for a completed report.

The same principle applies beyond files: design worker side effects so incomplete execution is distinguishable from committed execution.

Database transactions need explicit boundaries

A database transaction can provide a useful failure boundary when the database correctly detects a dead connection and rolls the transaction back.

def update_account(dsn, account_id):
    connection = connect(dsn)
    try:
        with connection:
            apply_changes(connection, account_id)
    finally:
        connection.close()

But forced process termination can prevent the Python finally block from running. Correctness therefore cannot depend solely on application cleanup.

The database and driver must provide the desired behavior when the worker connection vanishes unexpectedly.

Also consider external effects that are not covered by the transaction. A worker that updates a row and then sends an HTTP request can be terminated between those operations. Retrying the task may duplicate one side effect while repeating another.

Use established patterns such as idempotency keys, transactional outboxes, or durable job state when tasks can be retried after uncertain completion.

Shared queues and locks raise additional risk

The multiprocessing documentation gives a particularly important warning about terminating processes that use pipes, queues, locks, or semaphores.

A process terminated while using a pipe or queue can leave that communication mechanism corrupted. A process terminated while holding a lock or semaphore can cause other processes to deadlock.

This is not just an implementation detail. It affects how worker functions should be designed when hard termination is part of the failure model.

Prefer tasks whose important state lives outside ad-hoc process-shared synchronization primitives. If workers must use shared locks or queues, test forced termination while those resources are actively being used rather than assuming cleanup will restore them.

Child processes are a separate lifecycle

A pool worker can itself launch another process. Terminating that worker does not imply that all of its descendants will also be terminated.

The multiprocessing documentation explicitly notes that descendants can become orphaned when their parent is terminated.

For example:

def run_tool(command):
    return subprocess.run(command, check=True)

If a pool worker running this function is forcibly stopped, the lifecycle of the external command needs separate consideration.

Applications that launch process trees should use operating-system facilities appropriate to their platform, such as process groups, job objects, containers, or a supervisor that owns the complete tree. Do not assume terminate_workers() recursively cleans up arbitrary descendants.

Separate task deadlines from pool recovery

A robust design treats a task timeout and recovery of the execution environment as two related but distinct decisions.

from concurrent.futures import ProcessPoolExecutor, TimeoutError


def execute_with_deadline(job, timeout):
    executor = ProcessPoolExecutor(max_workers=1)
    future = executor.submit(job)

    try:
        return future.result(timeout=timeout)
    except TimeoutError:
        executor.terminate_workers()
        raise
    finally:
        executor.shutdown(wait=False, cancel_futures=True)

A one-worker pool limits collateral damage, but creating a new process pool per tiny task can be expensive. The right granularity depends on task duration, startup cost, isolation needs, and failure frequency.

For a service with many jobs, a supervisor may instead own a pool generation. If one task forces termination, the supervisor marks all in-flight work from that generation as uncertain, creates a fresh pool, and retries only jobs whose side effects are safe to repeat.

That makes recovery policy explicit rather than burying it inside the executor wrapper.

Do not confuse terminate with graceful SIGTERM handling

On POSIX, it is tempting to hear “SIGTERM” and assume normal application shutdown handlers will run.

multiprocessing.Process.terminate() is documented as abrupt termination: exit handlers and finally clauses are not guaranteed to execute. Code should therefore not depend on a worker’s normal Python teardown path when terminate_workers() is used.

If an application requires cooperative cleanup, implement cooperative cancellation separately. For example, tasks can periodically check a cancellation token represented by an IPC primitive designed for that purpose:

def scan(cancel_event, items):
    output = []

    for item in items:
        if cancel_event.is_set():
            return output
        output.append(process(item))

    return output

Cooperative cancellation works only when the task reaches the check. It cannot rescue a worker stuck indefinitely in a native call, which is why a hard-stop fallback can still be valuable.

Make escalation observable

Forced termination should leave a clear operational trail.

Log facts such as:

  • which job exceeded its deadline;
  • how long it had been running;
  • whether cooperative cancellation was attempted;
  • whether the pool was terminated or killed;
  • which other jobs were in flight;
  • whether those jobs are eligible for retry;
  • when a replacement pool was created.

Do not log entire task payloads by default. Work items can contain credentials, personal data, or proprietary inputs.

Metrics are useful too. A rising rate of pool termination is usually a symptom worth investigating rather than a normal concurrency statistic.

Test termination as a failure injection

The hard-stop path deserves dedicated tests because ordinary unit tests rarely exercise it.

Create a task that deliberately waits:

import time


def stuck_job():
    while True:
        time.sleep(1)

Submit it, trigger the deadline, and verify that the application does not remain blocked indefinitely after calling terminate_workers().

Then test the dangerous boundaries:

  1. terminate during a file write and verify partial output is not published;
  2. terminate during a database transaction and verify the database reaches an understood state;
  3. terminate while other tasks are running and verify they are marked uncertain;
  4. terminate while shared IPC is active if the architecture uses it;
  5. launch a descendant process and verify the supervisor handles it correctly;
  6. recreate the executor and verify new tasks do not inherit stale application assumptions.

If the system has a kill_workers() escalation path, test that separately. A path reserved for emergencies is precisely the kind of path that can rot unnoticed.

Remember the Python 3.14 compatibility boundary

terminate_workers() and kill_workers() were added in Python 3.14. Code that supports older Python versions cannot call them unconditionally.

A version-aware application can keep the hard-stop policy behind a small abstraction, but avoid emulating the feature by reaching into private ProcessPoolExecutor internals. Private worker-process attributes are implementation details and can change between Python releases.

If Python 3.13 or earlier must be supported, choose a public process-management design appropriate to those versions instead of depending on undocumented executor state.

Python 3.14 also changed the default process start method for ProcessPoolExecutor away from fork on POSIX. If an application is upgrading specifically to use the new termination methods, test startup behavior and worker initialization as part of the same migration rather than assuming only shutdown semantics changed.

Choose the failure boundary before production

The new methods solve a concrete gap in the high-level futures API: an application can now explicitly terminate or kill all living workers in a ProcessPoolExecutor without reaching into private implementation details.

They are most useful when the pool itself is the intended failure boundary.

That boundary should be chosen deliberately. Terminating the pool can abandon healthy tasks, skip Python cleanup, damage process-shared communication state, and leave descendant processes alive. Retrying interrupted jobs can also duplicate external side effects unless those effects were designed for uncertain execution.

Use normal shutdown for the normal path. Prefer cooperative cancellation when tasks can participate in it. Use terminate_workers() when the pool must stop promptly, and reserve kill_workers() for cases where stronger termination is actually required.

The API is simple. The engineering work is making sure that killing a worker is an expected failure mode rather than an unexpected corruption mode.

Python Concurrency ProcessPoolExecutor Multiprocessing Python 3.14