How to write custom python logging handler?

Question

How to write custom console log function to output only on the console window log messages on a single line (not append) until the first regular log record.

progress = ProgressConsoleHandler()
console  = logging.StreamHandler()  

logger = logging.getLogger('test')
logger.setLevel(logging.DEBUG) 
logger.addHandler(console)  
logger.addHandler(progress)

logger.info('test1')
for i in range(3):
    logger.progress('remaining %d seconds' % i)
    time.sleep(1)   
logger.info('test2')

So that the console output is only three lines:

INFO: test1
remaining 0 seconds... 
INFO: test2

Any suggestions on the best way on how to implement this?

Answer 1

import logging
class ProgressConsoleHandler(logging.StreamHandler):
    """
    A handler class which allows the cursor to stay on
    one line for selected messages
    """
    on_same_line = False
    def emit(self, record):
        try:
            msg = self.format(record)
            stream = self.stream
            same_line = hasattr(record, 'same_line')
            if self.on_same_line and not same_line:
                stream.write(self.terminator)
            stream.write(msg)
            if same_line:
                stream.write('... ')
                self.on_same_line = True
            else:
                stream.write(self.terminator)
                self.on_same_line = False
            self.flush()
        except (KeyboardInterrupt, SystemExit):
            raise
        except:
            self.handleError(record)
if __name__ == '__main__':
    import time
    progress = ProgressConsoleHandler()
    console  = logging.StreamHandler()  

    logger = logging.getLogger('test')
    logger.setLevel(logging.DEBUG) 
    logger.addHandler(progress)

    logger.info('test1')
    for i in range(3):
        logger.info('remaining %d seconds', i, extra={'same_line':True})
        time.sleep(1)   
    logger.info('test2')

Notice that only one handler is being registered, and the extra keyword argument to let the handler know it should stay on one line. There is more logic in the emit() method to handle changes between messages that should stay on one line and messages that need to have their own line.

Answer 2

Disclaimer 1

I have not thoroughly tested this solution, but it seems to work for basic logging functionality. Also, the way of implementing it is definitely not best practice. For these reasons, I don't recommend using this solution in production... at least without further testing

Disclaimer 2

Part of the reason this approach is not recommended/not best practice is because you can very easily accidentally override a method of the logging.Logger class in your wrapper class. If this happens, it can lead to strange, hard to explain bugs.

If using this approach, take care to check that any attribute/method names you plan to use in your own custom methods/attributes do not already exist in the logging.Logger class.

Refer to the documentation for the logging module for more information

Without further ado: The answer

Answer

I created a base class for custom loggers as such:

import logging

class CustomLoggerBC():
    def __init__(self, name:str):
         self._logger = logging.getLogger(name)
         newdict = {k: getattr(self._logger, k) for k in dir(self._logger) if k not in dir(self)}
         self.__dict__.update(newdict)

You then can write your own custom logger that inherits from the base class CustomLoggerBC as such:

class MyCustomLogger(CustomLoggerBC):
    def __init__(name:str, ...rest of your arguments here):
        super().__init__(name)
        ... rest of your custom constructor here

In your class constructor or any method in your class you can make any desired changes you want or would normally make to the logging.Logger instance stored in self._logger such as adding a handler or setting the log level:

class MyCustomLogger(CustomLoggerBC):
    def __init__(name:str, ...rest of your arguments here):
        super().__init__(name)
        ... Define your handler here
        self._logger.addHandler(self._customHandler)
        self._logger.setLevel(logging.INFO)
        ... rest of your custom constructor here

Examples

Code like this:

cl = MyCustomLogger("MyCustomLogger")
cl.info("This is a test")
cl.info("This is also a test")

Could easily produce something like this

[ INFO ] [05/19/2022 10:03:45] This is a test

[ INFO ] [05/19/2022 10:03:45] This is also a test

Additionally, you can even override the methods of the logging.Logger class ( although I don't recommend it )

class MyCustomLogger(CustomLoggerBC):
    def __init__(name:str, ...rest of your arguments here):
        super().__init__(name)
        ... rest of your custom constructor here
    def info(self, msg):
        print("My custom implementation of logging.Logger's info method!")

Code like this:

cl = MyCustomLogger("MyCustomLogger")
cl.info("This is a test")
cl.info("This is also a test")

Now instead produces something like this

My custom implementation of logging.Logger's info method!

My custom implementation of logging.Logger's info method!

Why Would You Want to Do This?

You can preconfigure your loggers ahead of time. For instance, if you know the logger for part A of your application will have a certain format, a certain handler, and a certain level. It is much easier to do something like this in part A

myPartALogger = PartACustomLogger()

instead of doing all the initializing work for the logger in part A.

It also makes loggers more reusable if part A and part B need separate loggers but would have the same configuration (say for example same level and formatter but a different handle) you can create two instances of your PartACustomLogger for each part with different names and pass different handlers to each instance.

Why it Works

Essentially you are wrapping the logging.Logger class with MyCustomLogger . The base class CustomLoggerBC updates the subclass's implementation instance dictionary with all the methods and attributes of the logging.Logger class so that the implementation essentially functions as the logging.Logger object. Any attribute/method requests to your custom logger class MyCustomLogger appear to be forwarded to the logging.Logger instance to be resolved by the logging.Logger instance, but in reality your own class is resolving the requests. This makes so that it seems you are subclassing the logging.Logger class but in reality you aren't.