Docker Container prints the output only while exiting

Question

I wrote a python program and its Dockerfile:

import time
print("Begin")
time.sleep(100);
print("End")

The image for it was created,and it was run using docker run <image-id> and the behaviour that surprises me is, after giving the run command in the console, it waits for sleep(100) seconds and prints "Begin" and "End" together.

Why are we not getting the intermediate results while running it?

Also how can I write a streaming app (in kafka or so), in this manner if it wont send the data immediately after it produces?

Answer 1

When you run your python script from the console, it displays Begin on stdout right away because it is a tty (interactive) and flushes at the end of each line . But if you redirect stdout and stdin like so python /tmp/a.py < /dev/null | cat python /tmp/a.py < /dev/null | cat , the python script will not notice it is run from a tty and will only flush when it completes.

If you run the same script from a docker container , it does not have a tty by default, you have to explicitly ask for one with --tty , -t Allocate a pseudo-TTY :

docker run -t yourimage

Alternatively, if you do no want the container to run with a tty, you can force the flush to happen regardless by setting the PYTHONUNBUFFERED environment variable, by adding the -u option to the python interpreter or by modifying your script like so:

import sys
import time
print("Begin")
sys.stdout.flush()
time.sleep(100);
print("End")

or with the flush argument (python3 only):

import time
print("Begin", flush=True)
time.sleep(100);
print("End")

Answer 2

When printing to stdout the OS does not guarantee it will be written immediately. What is guaranteed is that when the file descriptor will be closed the OS will flush the write buffer (this is why when the docker exits you get the output).

In order to ensure OS will flush, add the following code after any important print:

import sys
sys.stdout.flush()