Log unexpected errors to file via Python logging module

Question

I've been searching the interwebs, Python's official docs, and StackOverflow for quite some time now but cannot seem to find a proper answer.

I'm following Python's official HOWTO on logging - my logger looks like this:

import os, logging
os.chdir("C:/Users/kerfuffle/Desktop/logtest")

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

fh = logging.FileHandler("logging.log")
fh.setLevel(logging.INFO)
sh = logging.StreamHandler()
sh.setLevel(logging.DEBUG)

formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
sh.setFormatter(formatter)

logger.addHandler(fh)
logger.addHandler(sh)

logger.debug("debug!")
logger.info("info!")
logger.warning("warning!")

The last 3 lines of the code work fine, we also have a proper output via the StreamHandler.

logging.log - file:

2020-05-27 15:41:59,586 - INFO - info!
2020-05-27 15:41:59,586 - WARNING - warning!

console output:

2020-05-27 15:41:59,584 - DEBUG - debug!
2020-05-27 15:41:59,586 - INFO - info!
2020-05-27 15:41:59,586 - WARNING - warning!

Now what I do not understand is the following: If I deliberately create an error I expect it to be logged to a file. So I add the yet unknown variable "var" to my script:

var += 1

I will receive the standard error output in the console next to the StreamHandler() logging as seen above:

2020-05-27 15:45:37,444 - DEBUG - debug!
2020-05-27 15:45:37,445 - INFO - info!
2020-05-27 15:45:37,451 - WARNING - warning!
Traceback (most recent call last):
  File "c:/Users/kerfuffle/Desktop/logtest/file.py", line 23, in <module>
    var += 1
NameError: name 'var' is not defined

My logfile only shows:

2020-05-27 15:45:37,445 - INFO - info!
2020-05-27 15:45:37,445 - WARNING - warning!

I know from the tutorials that I can try to escape everything via:

try:
    var += 1
except Exception:
    logger.exception("You moron! Don't do that") #or
    logger.error('What a stupid error!')

But how can I catch errors and write them into my log file which are not predictable or primarily not expected by the user? (=> so no try/except)

You sometimes just don't know about errors before they are happening. For example when I have a script running via a cronjob.

Thank you for your replies: :-)

Answer 1

I believe you are looking for the following flag: exc_info

try:
    var += 1
except Exception as e:
    logging.critical(e, exc_info=True)

This will capture the generated error message and propagates it to logging. Also see https://stackoverflow.com/a/29556251/8140659