Exceptions: logging the traceback only once

Question

I am scratching my head about what is the best-practice to get the traceback in the logfile only once . Please note that in general I know how to get the traceback into the log.

Let's assume I have a big program consisting of various modules and functions that are imported, so that it can have quite some depth and the logger is set up properly.

Whenever an exception may occur I do the following:

try:
    do_something()
except MyError as err:
    log.error("The error MyError occurred", exc_info=err)
    raise

Note that the traceback is written to the log via the option exc_info=err .

My Problem is now that when everything gets a bit more complex and nested I loose control about how often this traceback is written to the log and it gets quite messy.

An example of the situation with my current solution for this problem is as follows:

from other_module import other_f

def main():

    try:
        # do something
        val = other_f()

    except (AlreadyLoggedError1, AlreadyLoggedError2, AlreadyLoggedError3):
        # The error was caught within other_f() or deeper and 
        # already logged with traceback info where it occurred
        # After logging it was raised like in the above example
        # I do not want to log it again, so it is just raised
        raise
    except BroaderException as err:
        # I cannot expect to have thought of all exceptions
        # So in case something unexpected happened 
        # I want to have the traceback logged here
        # since the error is not logged yet
        log.error("An unecpected error occured", exc_info=err)
        raise

The problem with this solution is, that I need to to keep track of all Exceptions that are already logged by myself and the line except (AlreadyLoggedError1, AlreadyLoggedError2, ...) gets arbitrary long and has to be put at any level between main() and the position the error actually occured.

So my question is: Is there some better (pythonic) way handling this? To be more specific: I want to raise the information that the exception was already logged together with the exception so that I do not have to account for that via an extra except block like in my above example.

Answer 1

The solution normally used for larger applications is for the low-level code to not actually do error handling itself if it's just going to be logged, but to put exception logging/handling at the highest level in the code possible, since exceptions will bubble up as far as needed. For example, libraries that send errors to a service like New Relic and Sentry don't need you to instrument each small part of your code that might throw an error, they are set up to just catch any exception and send it to a remote service for aggregation and tracking.