Is it better to use root logger or named logger in Python

Question

I am trying to figure out the best practice for using logging in python across multiple modules. I see here: http://docs.python.org/2/howto/logging#logging-from-multiple-modules on how to use the root logger to log across multiple modules. As the link points out though, you can't tell where in your application your messages come from as they all show name "root."

It seems to me there are two options (this assumes my modules are NOT in a package structure, but are just a bunch of modules in the same folder):

1) Use the root logger as in the example, but change the log formatting to include the filename:

# myapp.py
import logging
import mylib

def main():
    logging.basicConfig(format='%(asctime)s %(filename)s %(levelname)s: %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.INFO)  #filename='myapp.log', 
    logging.info('Started')
    mylib.do_something()
    logging.info('Finished')

if __name__ == '__main__':
    main()

#mylib.py
import logging

def do_something():
    logging.info('Do something')




In [4]: import myapp

In [5]: myapp.main()
03/06/2014 12:22:07 PM myapp.py INFO: Started
03/06/2014 12:22:07 PM mylib.py INFO: Do something
03/06/2014 12:22:07 PM myapp.py INFO: Finished

2) Use a root logger in the main app but a named logger in the submodules, and use 'name' instead of 'filename' in the log format:

# myapp.py
import logging
import mylib

def main():
    #logging.basicConfig(format='%(asctime)s %(filename)s %(levelname)s: %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.INFO)  #filename='myapp.log', 
    logging.basicConfig(format='%(asctime)s %(name)s %(levelname)s: %(message)s', datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.INFO)  #filename='myapp.log', 
    logging.info('Started')
    mylib.do_something()
    logging.info('Finished')

if __name__ == '__main__':
    main()

#mylib.py
import logging

def do_something():
    #logging.info('Do something')
    logger = logging.getLogger(__name__)
    logger.info('Do something')



In [3]: import myapp

In [4]: myapp.main()
03/06/2014 12:27:29 PM root INFO: Started
03/06/2014 12:27:29 PM mylib INFO: Do something
03/06/2014 12:27:29 PM root INFO: Finished

Answer 1

Vinay Sajip (maintainer of the logging module) makes a recommendation here which is similar to your option #2, except that you could use a named logger everywhere, even in myapp:

import logging
import mylib
logger = logging.getLogger(__name__)

def main():
    logging.basicConfig(format='%(asctime)s %(module)s %(levelname)s: %(message)s',
                        datefmt='%m/%d/%Y %I:%M:%S %p', level=logging.INFO)  
    logger.info('Started')
    mylib.do_something()
    logger.info('Finished')

if __name__ == '__main__':
    main()

which yields

03/06/2014 12:59:25 PM myapp INFO: Started
03/06/2014 12:59:25 PM mylib INFO: Do something
03/06/2014 12:59:25 PM myapp INFO: Finished

Note that if you use %(module)s instead of %(name)s , then you get myapp where before you got root or myapp.py , or __main__ .

This symmetry -- of always using logger -- may be especially useful if myapp is sometimes called as a script and sometimes imported as a module.

Answer 2

It seems that the main aim of the OP is to have an indication of where messages are coming from, which IMO is an essential part of any logging setup.

And I found that the logging module is currently in need of a bit of tweaking to accomplish something practical: because although one of the LogRecord attributes is lineno , identifying the file is a bit more difficult: you have module , filename and pathname . But the latter is the whole path, when what you are really likely to want is the path from the CWD.

I therefore suggest this:

cwd_path = pathlib.Path.cwd()

class TweakedFormatter(logging.Formatter):
    def formatMessage(self, record):
        # tweak: curtail the pathname to show only the path elements from the CWD
        path = pathlib.Path(record.pathname)
        # NB following evaluates to False if pathname of log event is not under CWD (rare occurrence)
        if path.is_relative_to(cwd_path):
            record.pathname = str(path.relative_to(cwd_path))
        return self._style.format(record) # copied from source

def get_logger():
    logger = logging.Logger(project_name)
    console_handler = logging.StreamHandler(sys.stdout)
    console_formatter = TweakedFormatter(
        fmt='%(asctime)s %(levelname)s [%(pathname)s %(lineno)d]: %(message)s', datefmt='%a %H:%M:%S')
    console_handler.setFormatter(console_formatter)
    logger.addHandler(console_handler)
    return logger

I illustrate the case of a typical console logger: where you don't particularly need the full date printing out, and where screen space is at a premium (anti-clutter).

NB I in fact only call get_logger once, and import the result to all files. I've never really found a use for a hierarchy of loggers (though I'm sure that idea would never have come about if there hadn't been a need for it...).