StreamHandler - 不记录到 docker 容器中的 sys.stdout - Python

Question

I have a RotatingFileHandler and StreamHandler in my Python app. This is running in an apache server in a docker container. So I have symlinked both the apache access and error logs to the /dev/stdout seen as:

RUN ln -sf /dev/stdout /project/Project_Service/log/access.log && \
    ln -sf /dev/stdout /project/Project_Service/log/error.log

I have set up the handlers as:

app = Flask(__name__)
app.logger.setLevel(logging.DEBUG)
formatter = logging.Formatter(
    "%(asctime)s - %(name)s - %(levelname)s - %(message)s")

file_handler = RotatingFileHandler(app_error,
                                   maxBytes=1024 * 1024 * 100, backupCount=20)
file_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(formatter)

stream_handler = logging.StreamHandler(stream=sys.stdout)
stream_handler.setLevel(logging.DEBUG)
stream_handler.setFormatter(formatter)

app.logger.addHandler(file_handler)
app.logger.addHandler(stream_handler)
app.logger.debug('DEBUG LOG')
app.logger.info('INFO LOG')
app.logger.warning('WARNING LOG')
app.logger.error('ERROR LOG')
app.logger.critical('CRITICAL LOG')

If i look at the app_error file, i can see output correctly:

2018-12-06 19:37:46,863 - app - DEBUG - DEBUG LOG
2018-12-06 19:37:46,865 - app - INFO - INFO LOG
2018-12-06 19:37:46,865 - app - WARNING - WARNING LOG
2018-12-06 19:37:46,866 - app - ERROR - ERROR LOG
2018-12-06 19:37:46,868 - app - CRITICAL - CRITICAL LOG

And this is seen in the docker logs:

[Thu Dec 06 19:37:46.864764 2018] [:error] [pid 8] 2018-12-06 19:37:46,863 - app - DEBUG - DEBUG LOG
[Thu Dec 06 19:37:46.865872 2018] [:error] [pid 8] 2018-12-06 19:37:46,865 - app - INFO - INFO LOG
[Thu Dec 06 19:37:46.866116 2018] [:error] [pid 8] 2018-12-06 19:37:46,865 - app - WARNING - WARNING LOG
[Thu Dec 06 19:37:46.866370 2018] [:error] [pid 8] [2018-12-06 19:37:46,866] ERROR in app: ERROR LOG
[Thu Dec 06 19:37:46.868450 2018] [:error] [pid 8] 2018-12-06 19:37:46,866 - app - ERROR - ERROR LOG
[Thu Dec 06 19:37:46.870184 2018] [:error] [pid 8] [2018-12-06 19:37:46,868] CRITICAL in app: CRITICAL LOG
[Thu Dec 06 19:37:46.870448 2018] [:error] [pid 8] 2018-12-06 19:37:46,868 - app - CRITICAL - CRITICAL LOG

Which makes sense, as i have both the error log, which i'm assuming has log levels of ERROR and CRITICAL, and the streamhandler. However, if i take out the symlink to the error log, both logs disappear. It's as if the StreamHandler is outputting to dev/stderr, even though i have declared it as sys.stdout in the StreamHandler. I've also tried removing the "stream=", but that did not fix anything. I've been pouring over all the documentation, and can't seem to figure out how to fix this.

I've also set PYTHONUNBUFFERED=0 in the environment variables.

Any help or direction you can provide is appreciated.

Answer 1

Flask attaches a "default" StreamHandler to the app logger already. Thus after you attach your 2 loggers there's actually 3 loggers:

print(app.logger.handlers)

[<logging.StreamHandler object at 0x7ff30592b358>,
    <logging.handlers.RotatingFileHandler object at 0x7ff3058e2e80>,
    <logging.StreamHandler object at 0x7ff3058f33c8>]

This also explains why the 2 lines have a different format -- they come from the default StreamHandler which has its own formatter:

2018-12-06 19:37:46,863 - app - DEBUG - DEBUG LOG
2018-12-06 19:37:46,865 - app - INFO - INFO LOG
2018-12-06 19:37:46,865 - app - WARNING - WARNING LOG
[2018-12-06 19:37:46,866] ERROR in app: ERROR LOG        # <----
2018-12-06 19:37:46,866 - app - ERROR - ERROR LOG
[2018-12-06 19:37:46,868] CRITICAL in app: CRITICAL LOG  # <----
2018-12-06 19:37:46,868 - app - CRITICAL - CRITICAL LOG

The fix is to simply remove all handlers before you add yours :

app.logger.handlers = []