Python needs to log each script into its own log

Question

I have scripts parent.py and child.py (many childs) and I need to have logs for each, so any logging within parent.py should be in parent.log and child.py should be in child.log

I have the below in each script but I get empty logs... why??

#main.py
import child

handler = logging.FileHandler('logs/main.log')
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - % 
(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)

child.child_func()
logger.info('testing parent...')


#child.py

handler = logging.FileHandler('logs/child.log')
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - % 
(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)

def child_func():
    logger.info('testing child...')

What I need to have is

#parent.log
{format} testing parent...

#child.log
{format} testing child...

Answer 1

The folks above are right about the default level on the loggers. Also, instead of spreading your configuration around everywhere, I find it more manageable to consolidate logging configuration to be early on in the application. See the example below.

Note: I don't expect this to be selected as an answer. I just wanted to point out what I believe is a better way of organizing the code.

main.py

import logging
import child


logger = logging.getLogger(__name__)


def setup_logging():
    main_handler = logging.FileHandler('logs/main.log')
    child_handler = logging.FileHandler('logs/child.log')

    # Note that you can re-use the same formatter for the handlers.
    formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")

    main_handler.setFormatter(formatter)
    child_handler.setFormatter(formatter)

    # By default, loggers inherit the level from their parent, so we only need
    # to set the level on the root logger if you want to have only one knob to
    # control the level.
    root_logger = logging.getLogger()
    root_logger.setLevel(logging.DEBUG)

    main_logger = logging.getLogger(__name__)
    child_logger = logging.getLogger('child')
    child_logger.propagate = False

    main_logger.addHandler(main_handler)
    child_logger.addHandler(child_handler)


def main():
    setup_logging()

    child.child_func()
    logger.info('testing parent...')


if __name__ == '__main__':
    main()

child.py

import logging


logger = logging.getLogger(__name__)


def child_func():
    logger.info('testing child...')

Setting up a root logger and a child logger (no main)

Here's an example of setting up the root logger to log to logs/main.log , and the child logger to go to logs/child.log

def setup_logging():
    root_handler = logging.FileHandler('logs/main.log')
    child_handler = logging.FileHandler('logs/child.log')

    # Note that you can re-use the same formatter for the handlers.
    formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")

    root_handler.setFormatter(formatter)
    child_handler.setFormatter(formatter)

    # By default, loggers inherit the level from their parent, so we only need
    # to set the level on the root logger if you want to have only one knob to
    # control the level.
    root_logger = logging.getLogger()
    root_logger.setLevel(logging.DEBUG)
    root_logger.addHandler(root_handler)

    child_logger = logging.getLogger('child')
    child_logger.propagate = False
    child_logger.addHandler(child_handler)

Answer 2

You can set severity-level on both handlers and loggers - I believe the logger is set to logging.WARNING by default, so you would only get warning-logs using your code.

You can read more in this thread: What is the point of setLevel in a python logging handler?

import logging
import child

handler = logging.FileHandler('logs/main.log')
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)       # <-- changed 
child.child_func()
logger.info('testing parent...')
logger.warning('testing parent...')
logger.debug('testing parent...')

#child.py
import logging

handler = logging.FileHandler('logs/child.log')
formatter = logging.Formatter("%(asctime)s [%(filename)s:%(lineno)s - %(funcName)10s()] %(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)      # <-- changed
def child_func():
    logger.info('testing child...')
    logger.warning('testing child...')
    logger.debug('testing child...')