将所有stdout / stderr全局重定向到记录器

Question

Background

I have a very large python application that launches command-line utilities to get pieces of data it needs. I currently just redirect the python launcher script to a log file, which gives me all of the print() output, plus the output of the command-line utilities, ie:

python -m launcher.py &> /root/out.log

---

Problem

I've since implemented a proper logger via logging , which lets me format the logging statements more precisely, lets me limit log file size, etc. I've swapped out most of my print() statements with calls to my logger. However, I have a problem: none of the output from the command-line applications is appearing in my log. It instead gets dumped to the console. Also, the programs aren't all launched the same way: some are launched via popen() , some by exec() , some by os.system() , etc.

---

Question

Is there a way to globally redirect all stdout / stderr text to my logging function, without having to re-write/modify the code that launches these command-line tools? I tried setting setting the following which I found in another question:

sys.stderr.write = lambda s: logger.error(s)

However it fails with " sys.stderr.write is read-only".

Answer 1

While this is not a full answer, it may show you a redirect to adapt to your particular case. This is how I did it a while back. Although I cannot remember why I did it this way, or what the limitation was I was trying to circumvent, the following is redirecting stdout and stderr to a class for print() statements. The class subsequently writes to screen and to file:

import os
import sys
import datetime

class DebugLogger():

    def __init__(self, filename):
        timestamp = datetime.datetime.strftime(datetime.datetime.utcnow(), 
                                               '%Y-%m-%d-%H-%M-%S-%f')
        #build up full path to filename
        logfile = os.path.join(os.path.dirname(sys.executable), 
                               filename + timestamp)
        self.terminal = sys.stdout
        self.log = open(logfile, 'a')

    def write(self, message):
        timestamp = datetime.datetime.strftime(datetime.datetime.utcnow(), 
                                               ' %Y-%m-%d-%H:%M:%S.%f')
        #write to screen
        self.terminal.write(message)
        #write to file
        self.log.write(timestamp + ' - ' + message)      
        self.flush()

    def flush(self):
        self.terminal.flush()
        self.log.flush()
        os.fsync(self.log.fileno())

    def close(self):
        self.log.close()


def main(debug = False):
    if debug:
        filename = 'blabla'
        sys.stdout = DebugLogger(filename)
        sys.stderr = sys.stdout
    print('test')

if __name__ == '__main__':
    main(debug = True)

Answer 2

import sys
import io

class MyStream(io.IOBase):
    def write(self, s):
        logger.error(s)

sys.stderr = MyStream()

print('This is an error', stream=sys.stderr)

This make all call to sys.stderr go to the logger. The original one is always in sys.__stderr__