Subprocess

Avoid Subprocess Pipe Deadlocks in Python

Launching a command from Python is easy. Capturing its output is also easy. The trouble starts when a parent process waits for a child while the child is waiting for the parent to read from a pipe. That circular wait is a deadlock: neither process can make progress even though neither has crashed. This problem is especially confusing because the same code may work during testing and hang only when a command produces more output. Small output fits in an operating-system pipe buffer. Larger output can fill that buffer and expose the incorrect coordination.