Making Python loggers log all stdout and stderr messages

Question

Using the python logging package, and writing a class Log, I'd like to tee stdout and stderr to a log file :

log = Log("log.txt")
print "line1"
print "line2"
print >>sys.stderr, "err1"
del log
print "line to screen only"

The output log file would contain :

16/11/2017 09:51:58 INFO - line1
16/11/2017 09:51:58 INFO - line2
16/11/2017 09:51:58 INFO - err1

Any idea how to write this Log class, keeping the advantages of the "logging" package (time-stamps, ...) ?

Answer 1

The right way to achieve what you ask is to use Logger object. It gives you much more flexability. This object can be bound to multiple handlers; You need a streamhandler to log message to sys.stdout and a file handler to log it to a file. You then both print to the screen and log to a file in a single command.

import logging

# create logger 
logger = logging.getLogger('example')
logger.setLevel(logging.INFO)

# create file handler which logs messages
fh = logging.FileHandler('fh.log')
fh.setLevel(logging.DEBUG)

# create console handler to print to screen
ch = logging.StreamHandler()
ch.setLevel(logging.INFO)

# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
ch.setFormatter(formatter)

# add the handlers to the logger
logger.addHandler(fh)
logger.addHandler(ch)

Now every call to logger.info(msg) will be printed both to the screen, and written to fh.log file.

---

There is another way, where you can replace sys.stdout and sys.stderr stream objects. Create a class and customize it ( original answer here ):

import sys

class writer(object):
    _fh = None
    _orig_stdout = sys.stdout

   def __init__(self):
       _fh = open('logging.log', 'w')

    def write(self, data):
        fp.write(data)
        _orig_stdout.write(data)

    def flush():
        _orig_stdout.flush()

logger = writer()

sys.stdout = logger
sys.stderr = logger

Answer 2

I made this package to do exactly what you said.

pip install stdlogging

After installed from pip, you can capture any output by logging. But be careful to capture stdout because it's very fragile.

import stdlogging
stdlogging.enable(stdout=False, stderror=True)