Can Python's logging format be modified depending on the message log level?

Question

I'm using Python's logging mechanism to print output to the screen. I could do this with print statements, but I want to allow a finer-tuned granularity for the user to disable certain types of output. I like the format printed for errors, but would prefer a simpler format when the output level is "info."

For example:

  logger.error("Running cmd failed")
  logger.info("Running cmd passed")

In this example, I would like the format of the error to be printed differently:

 # error Aug 27, 2009 - ERROR: Running cmd failed # info Running cmd passed

Is it possible to have different formats for different log levels without having multiple logging objects? I'd prefer to do this without modifying the logger once it's created since there are a high number of if/else statements to determine how the output should be logged.

Answer 1

I just ran into this issue and had trouble filling in the "holes" left in the above example. Here's a more complete, working version that I used. Hopefully this helps someone:

# Custom formatter
class MyFormatter(logging.Formatter):

    err_fmt  = "ERROR: %(msg)s"
    dbg_fmt  = "DBG: %(module)s: %(lineno)d: %(msg)s"
    info_fmt = "%(msg)s"


    def __init__(self, fmt="%(levelno)s: %(msg)s"):
        logging.Formatter.__init__(self, fmt)


    def format(self, record):

        # Save the original format configured by the user
        # when the logger formatter was instantiated
        format_orig = self._fmt

        # Replace the original format with one customized by logging level
        if record.levelno == logging.DEBUG:
            self._fmt = MyFormatter.dbg_fmt

        elif record.levelno == logging.INFO:
            self._fmt = MyFormatter.info_fmt

        elif record.levelno == logging.ERROR:
            self._fmt = MyFormatter.err_fmt

        # Call the original formatter class to do the grunt work
        result = logging.Formatter.format(self, record)

        # Restore the original format configured by the user
        self._fmt = format_orig

        return result

Answer 2

Yes, you can do this by having a custom Formatter<\/code> class:

class MyFormatter(logging.Formatter):
    def format(self, record):
        #compute s according to record.levelno
        #for example, by setting self._fmt
        #according to the levelno, then calling
        #the superclass to do the actual formatting
        return s

Answer 3

And again like JS answer but more compact.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG :"DBG: %(module)s: %(lineno)d: %(message)s",
               logging.ERROR : "ERROR: %(message)s",
               logging.INFO : "%(message)s",
               'DEFAULT' : "%(levelname)s: %(message)s"}

    def format(self, record):
        self._fmt = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)

hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)

Answer 4

ONE WAY OF DOING THIS

Define a class

import logging

class CustomFormatter(logging.Formatter):
    """Logging Formatter to add colors and count warning / errors"""

    grey = "\x1b[38;21m"
    yellow = "\x1b[33;21m"
    red = "\x1b[31;21m"
    bold_red = "\x1b[31;1m"
    reset = "\x1b[0m"
    format = "%(asctime)s - %(name)s - %(levelname)s - %(message)s (%(filename)s:%(lineno)d)"

    FORMATS = {
        logging.DEBUG: grey + format + reset,
        logging.INFO: grey + format + reset,
        logging.WARNING: yellow + format + reset,
        logging.ERROR: red + format + reset,
        logging.CRITICAL: bold_red + format + reset
    }

    def format(self, record):
        log_fmt = self.FORMATS.get(record.levelno)
        formatter = logging.Formatter(log_fmt)
        return formatter.format(record)

Instantiate logger

# create logger with 'spam_application'
logger = logging.getLogger("My_app")
logger.setLevel(logging.DEBUG)

# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)

ch.setFormatter(CustomFormatter())

logger.addHandler(ch)

And use!

logger.debug("debug message")
logger.info("info message")
logger.warning("warning message")
logger.error("error message")
logger.critical("critical message")

Result

Answer 5

This is an adaptation of estani's answer<\/a> to the new implementation of logging.Formatter<\/code> which now relies on formatting styles. Mine relies on '{'<\/code> style format, but it can be adapted. Could be refined to be more general and allow selection of formatting style and custom messages as arguments to __init__<\/code> , too.

class SpecialFormatter(logging.Formatter):
    FORMATS = {logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),
           logging.ERROR : logging._STYLES['{']("{module} ERROR: {message}"),
           logging.INFO : logging._STYLES['{']("{module}: {message}"),
           'DEFAULT' : logging._STYLES['{']("{module}: {message}")}

    def format(self, record):
        # Ugly. Should be better
        self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
        return logging.Formatter.format(self, record)

hdlr = logging.StreamHandler(sys.stderr)
hdlr.setFormatter(SpecialFormatter())
logging.root.addHandler(hdlr)
logging.root.setLevel(logging.INFO)

Answer 6

Instead of relying on styles or internal fields, you could also create a Formatter that delegates to other formatters depending on record.levelno (or other criteria). This is a slightly cleaner solution in my humble opinion. Code below should work for any python version >= 2.7:

class MyFormatter(logging.Formatter):

    default_fmt = logging.Formatter('%(levelname)s in %(name)s: %(message)s')
    info_fmt = logging.Formatter('%(message)s')

    def format(self, record):
        if record.levelno == logging.INFO:
            return self.info_fmt.format(record)
        else:
            return self.default_fmt.format(record)

Answer 7

The above solution works with 3.3.3 release. However with 3.3.4 you get the following error.

FORMATS = { logging.DEBUG : logging._STYLES['{']("{module} DEBUG: {lineno}: {message}"),

TypeError: 'tuple' object is not callable

After some searching around in the logging class Lib\\logging__init__.py I found that a data structure has changed from 3.3.3 to 3.3.4 that causes the issue

3.3.3

_STYLES = {
    '%': PercentStyle,
    '{': StrFormatStyle,
    '$': StringTemplateStyle
}

3.3.4

_STYLES = {
   '%': (PercentStyle, BASIC_FORMAT),
   '{': (StrFormatStyle, '{levelname}:{name}:{message} AA'),
    '$': (StringTemplateStyle, '${levelname}:${name}:${message} BB'),
}

The updated solution is therefore

class SpecialFormatter(logging.Formatter):
     FORMATS = {logging.DEBUG : logging._STYLES['{'][0]("{module} DEBUG: {lineno}: {message}"),
       logging.ERROR : logging._STYLES['{'][0]("{module} ERROR: {message}"),
       logging.INFO : logging._STYLES['{'][0]("{module}: {message}"),
       'DEFAULT' : logging._STYLES['{'][0]("{module}: {message}")}

 def format(self, record):
    # Ugly. Should be better
    self._style = self.FORMATS.get(record.levelno, self.FORMATS['DEFAULT'])
    return logging.Formatter.format(self, record)

Answer 8

If you are just looking to skip formatting certain levels, you can do something simpler than the other answers like the following:

class FormatterNotFormattingInfo(logging.Formatter):
    def __init__(self, fmt = '%(levelname)s:%(message)s'):
        logging.Formatter.__init__(self, fmt)

    def format(self, record):
        if record.levelno == logging.INFO:
            return record.getMessage()
        return logging.Formatter.format(self, record)