Inherit logger name in Python

Question

I am trying to log the uncaught exception in python. For that, I am setting the sys.excepthook in my __init__.py as follows.

import sys
import logging
import traceback


def log_except_hook(*exc_info):
    logger = logging.getLogger(__name__)
    text = "".join(traceback.format_exception(*exc_info))
    logger.critical(f"Unhandled exception:\n{text}")


sys.excepthook = log_except_hook

My problem is that when I run a process and an exception occurs, the module name in the logged exception is the name of the folder containing the code ( src in this case), instead of the name of the function in which it occurred ( foo_produce_an_error in this example). Please find an example below:

2019-10-09 18:55:48,638 src CRITICAL: Unhandled exception:
Traceback (most recent call last):
  File "/Users/ivallesp/Projects/Project_Folder/main.py", line 109, in <module>
    foo_function(7)
  File "/Users/ivallesp/Projects/Project_Folder/src/new_module.py", line 8, in foo_function
    foo_produce_an_error(x)
  File "/Users/ivallesp/Projects/Project_Folder/src/new_module.py", line 12, in foo_produce_an_error
    x / 0
ZeroDivisionError: division by zero

How can I make logging to show the module and name of the function where the error ocurred in the first logging line?

Answer 1

You haven't provided enough information to answer the question - for example, how you've configured logging (specifically, the format string/formatter used). I can illustrate how to achieve the desired result in general, with an example. Suppose you have a failing function in a module failfunc :

# failfunc.py
def the_failing_func():
    1 / 0

Then, your main script might be:

# logtest_ue.py
import logging
import sys

from failfunc import the_failing_func

def log_except_hook(*exc_info):
    logger = logging.getLogger(__name__)
    tb = exc_info[-1]
    # get the bottom-most traceback entry
    while tb.tb_next:
        tb = tb.tb_next
    modname = tb.tb_frame.f_globals.get('__name__')
    funcname = tb.tb_frame.f_code.co_name
    logger.critical('Unhandled in module %r, function %r: %s',
                    modname, funcname, exc_info[1], exc_info=exc_info)

sys.excepthook = log_except_hook

def main():
    the_failing_func()

if __name__ == '__main__':
    logging.basicConfig(format='%(levelname)s %(message)s')
    sys.exit(main())

when this is run, it prints

CRITICAL Unhandled in module 'failfunc', function 'the_failing_func': division by zero
Traceback (most recent call last):
  File "logtest_ue.py", line 23, in <module>
    sys.exit(main())
  File "logtest_ue.py", line 19, in main
    the_failing_func()
  File "/home/vinay/projects/scratch/failfunc.py", line 2, in the_failing_func
    1 / 0
ZeroDivisionError: division by zero

Note the slightly simpler way of getting a traceback into the log, using the exc_info keyword parameter. Also note that in this case, the normal module and function name (which could be displayed using %(module)s and %(funcName)s in a format string) would be those where the sys.excepthook points to, not the values where the exception actually occurred. For those, you would have to use the traceback object in the way I've illustrated to get the bottom-most frame (where the exception actually occurred) and get the module and function names from that frame.

Answer 2

The module is hard to get but you can have filename and line number. They are contained in exc_info which you don't need to format yourself. You can just pass them to the log function and use a formatter to display it all. Here's an example code to give you an idea how this is done:

import sys
import logging
import traceback


def log_except_hook(*exc_info):
    logger = logging.getLogger(__name__)
    handler = logging.StreamHandler()
    handler.setFormatter(logging.Formatter('%(asctime)s %(fname)s:%(lnum)s %(message)s %(exc_info)s'))
    logger.addHandler(handler)

    frame = traceback.extract_tb(exc_info[2])[-1]
    fname = frame.filename
    lnum = frame.lineno
    logger.critical("Unhandled exception:", exc_info=exc_info, extra={'fname':fname, 'lnum':lnum})

sys.excepthook = log_except_hook