Python, non-blocking pipe, flushing, and missing stdout/stderr

Question

I have two python processes connected by a pipe. The pipe was created with:

read_file_descriptor, write_file_descriptor = os.pipe()
os.set_blocking(read_file_descriptor, False)
os.set_inheritable(read_file_descriptor, True)

The parent process forks off a child, and the child reads from the read file descriptor using code that in essence boils down to this:

lines = []
read_handle = os.fdopen(read_file_descriptor)
while True:
    line = read_handle.readline()
    if _TERMINATION_CHAR in line:
        # trigger final upload
        line = line[: line.index(_TERMINATION_CHAR)]
        received_stream_termination = True
    elif len(line) == 0:
        # The line would at least have the newline char if it was a blank.

        # no more to read right now; just keep looping and trying to read
        # until the timeout or the termination character tell us to stop
        time.sleep(0.01)
        continue

    fp.write(line)
    fp.flush()
    if received_stream_termination:
        break
# handle lines...

The parent process, meanwhile, redirects its stdout and stderr to point at the write_file_descriptor . When the parent is done, it does:

logger.info("Cleaning up")
print(_TERMINATION_CHAR)  # tell the reader that the stream is done
sys.stdout.flush()
sys.stderr.flush()

The process is running with PYTHONUNBUFFERED=1 set in the environment. I am stressing this code by having the parent write ~20k lines (10k each to stdout and stderr, interleaved):

    for i in range(10000):
        time.sleep(0.01)
        print(f"From stdout: {i}")
        print(f"From stderr: {i}", file=sys.stderr)

    return a + b

With this, the lines the handler sees are:

# ... there are more lines before this. Nothing seems to be missing up to this point
From stdout: 9012
From stderr: 9012
From stdout: 9013
From stderr: 9013
From stdout: 9014
From stderr: 9014
From stdout: 9015
From stderr: 9015
2022-11-15 23:01:28,536 - INFO : Cleaning up

So... a lot of lines at the end are missing. But we still see the log message. Any ideas why this may be happening?

Answer 1

The problem turned out to be a race condition between two different mechanisms for indicating completion. One mechanism was the termination character, the other was a sigterm handler. The sigterm was sent between when the print statements completed their execution and when the terminating character was written. Simplifying to always use the termination character only solved the issue. So nothing funky with i/o, just your run-of-the-mill race condition!