Python logging in modules not working as expected

Question

I am sure I am just misunderstanding how this works but I cannot seem to have log formatting propagate to logging in modules unless the module "knows" the name of the module that called it.

The following code demonstrates the issue. The code below works but I have to hard code the name of the module that called it for the formatting to apply.

app.py

import logging
import mymodule

logger = logging.getLogger('app')
log_formatter = logging.Formatter(f'%(asctime)s %(name)s[%(process)d]: %(levelname)s - %(message)s\n')
log_handler_console = logging.StreamHandler()
log_handler_console.setFormatter(log_formatter)
log_handler_console.terminator = ''
logger.addHandler(log_handler_console)

logger.error('testing 1')
mymodule.dosomething()

module.py

import logging
logger = logging.getLogger('app.module')
def dosomething():
    logger.error('testing 2')

As per code above I get this:

2022-01-09 19:01:53,986 app[46541]: ERROR - testing 1
2022-01-09 19:01:53,987 app.module[46541]: ERROR - testing 2

but if I change the name of the logger to eg 'test' I get:

2022-01-09 19:21:07,590 app[46580]: ERROR - testing 1
testing 2

If I change the name of the logger in line 2 of module.py to anything other than 'app' or 'app.something' then the formatting no longer applies. I get how the inheritance works (I think) but how can I know the calling module name without hard coding it? I did see a few examples here on SO but with cautions that its a bad idea.

Essentially what I am trying to do is define a log format in the main app and have any module that the main app calls use the same format. I am I crazy? Seems like an obvious thing to want?

Answer 1

What you expect, "Essentially what I am trying to do is define a log format in the main app and have any module that the main app calls use the same format.", is totally reasonable.

"how can I know the calling module name without hard coding it?" You can use the below code to set up logger name hierarchy analogous to your Python package hierarchy.

logger = logging.getLogger(__name__)

With propagation enabled by default on all children loggers, all filtered messages will be propagated up through the hierarchical structure, and finally to the root logger, and then handled by its handler in its specified format.

For more detailed explanation of those practices of using Python logging, you may refer to Good Practices of using Python Logging

Answer 2

Actually, while I was still trying to figure this out I figured the answer for myself. Rather than delete the question I thought I would answer it for anyone else who comes across this issue.

Replacing both instances (ie, in the main app and in the module) of

logger = logging.getLogger('something')

with

logger = logging.getLogger()

A 'root' logger is created and everything works as I was expecting.