带有 Docker 日志的 tqdm 进度条

Question

. Answers to this question are eligible for a +200 reputation bounty. Mikko Ohtamaa wants to to this question.

I am using tqdm to display various progress bars for my Python console application. For the production deployment of the applications, I use Docker.

The progress bars work fine when running a Python application in a terminal. However, when Dockerized and the terminal output is accessed through docker logs the progress bar does not function because as far as I understand it is not an interactive terminal. Although it looks like the progress gets rendered if docker logs is dumped after the progress bar have completed, but not sure if there are some other conditions for this to happen (output buffering?).

I would like to modify my tqdm behavior so that

• It detects when it is run in non-interactive Dockerised environment
• Instead of displaying interactive progress bar, it will log completion statements (10% done, X iterations/s) regularly

This way the progress durations and such would be more accessible when running the application in production.

What would be the way to attach such a custom behavior to tqdm ?

Answer 1

You can recognize if your process is running in non-interactive docker environment according to absence of TERM environment variable.

# interactive environment
$ docker run --rm -it centos:7 env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=8c810e490671
TERM=xterm
HOME=/root

# non-interactive environment
$ docker run --rm  centos:7 env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=e267f2ba0f8f
HOME=/root

Consider this fact in your script and in case of non-interactive environment use total parameter of tqdm or disable it:

total: int or float, optional

The number of expected iterations. If unspecified, len(iterable) is used if possible. If float("inf") or as a last resort, only basic progress statistics are displayed (no ETA, no progressbar). If gui is True and this parameter needs subsequent updating, specify an initial arbitrary large positive number, eg 9e9.

disable: bool, optional

Whether to disable the entire progressbar wrapper [default: False]. If set to None, disable on non-TTY.

https://github.com/tqdm/tqdm/#parameters

Update:

It looks like TERM env variable in non-interactive environment has the value "dumb"

docker run --rm  centos:7 bash -c 'echo "${TERM}"'
dumb