Python 记录器/处理程序配置错误？

Question

Though I have been working with python/ipython for some time now, I consider myself a newb. There are still many things, especially about the logging support, I thought I understood from the documentation, but apparently are more difficult to configure than I previously had hoped. I am using ipython 5.5.0 / python 2.7.17 on Xubuntu 18.04.04 LTS with the colorlogs. My logging configuration module is below.

import coloredlogs
import datetime
import logging
import logging.config
import os
import yaml

def setup_logging( default_path='../Config/logging.yaml',
                   default_level=logging.DEBUG,
                   env_key='LOG_CFG'):

  path = os.path.join(os.path.dirname(os.path.realpath(__file__)), default_path)
  value = os.getenv(env_key, None)

  # If the envvar is set, use it's value
  if value:
    path = value

  _dt = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
  print("%s Using Logging Configuration: %s" % (_dt, path) )

  #
  # If the configuration file path is there, read it
  #
  if os.path.exists(path):
    with open(path, 'rt') as f:
      try:
        config = yaml.safe_load(f.read())
        logging.config.dictConfig(config)
        coloredlogs.install(level=default_level)
      except Exception as err:
        print(err)
        print('Error in Logging Configuration. Using default configs')
        logging.basicConfig(level=default_level)
        coloredlogs.install(level=default_level)

  # Otherwise, continue without a configuration
  else:
    logging.basicConfig(level=logging.DEBUG)
    coloredlogs.install(level=logging.DEBUG)
    print('Failed to load configuration file. Using default configs')

The configuration is held in a yaml file with the following definitions.

version: 1
disable_existing_loggers: False

formatters:
    basic:
        format: "%(name)s - %(message)s"
    standard:
        format: "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
    error:
        format: "%(levelname)s <PID %(process)d:%(processName)s> %(name)s.%(funcName)s(): %(message)s"

handlers:
    console_basic:
        class: logging.StreamHandler
        level: DEBUG
        formatter: basic
        stream: ext://sys.stdout

    console_out:
        class: logging.StreamHandler
        level: DEBUG
        formatter: standard
        stream: ext://sys.stdout

    console_err:
        class: logging.StreamHandler
        level: DEBUG
        formatter: standard
        stream: ext://sys.stderr

    debug_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: DEBUG
        formatter: standard
        filename: /tmp/debug.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    info_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: INFO
        formatter: standard
        filename: /tmp/info.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    warn_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: WARN
        formatter: standard
        filename: /tmp/warn.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    error_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: ERROR
        formatter: error
        filename: /tmp/errors.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

    critical_file_handler:
        class: logging.handlers.RotatingFileHandler
        level: CRITICAL
        formatter: standard
        filename: /tmp/critical.log
        maxBytes: 10485760 # 10MB
        backupCount: 20
        encoding: utf8

root:
    level: CRITICAL
    handlers: [console_err]
    propogate: no

loggers:
  test:
    level: DEBUG
    handlers: [console_basic]
    propogate: no

  Utils.paragraph_processing:
    level: DEBUG
    handlers: [info_file_handler, debug_file_handler, warn_file_handler, error_file_handler, critical_file_handler]
    propogate: no

  Utils.graphing_functions:
    level: DEBUG
    handlers: [info_file_handler, debug_file_handler, warn_file_handler, error_file_handler, critical_file_handler]
    propogate: no

The following snippet of my test.py module follows.

import coloredlogs
from copy import deepcopy
import cv2
import imutils
import logging
import logging.config
import os
import yaml

import matplotlib.pyplot as PLT
import matplotlib.image as MPI
import numpy as np

import Tests.filtering_tests as FT
import Tests.morphology_tests as MT

import Utils.global_defs as GL
import Utils.graphing_functions as GF
import Utils.paragraph_processing as PP
import Utils.logging_functions as LF

.
.
.
def phony_main():
  LF.setup_logging()
  # create logger
  LOG = logging.getLogger(__name__)

  LOG.critical("Logging Started...")


# -----------------------------------------------------------------------------
#
# Main
#
img = None

if __name__ == "__main__":
    # execute only if run as a script
    phony_main()

My questions are, when I change the configuration as I did from [console_out] to [console_basic], I expected the messages to conform, but they do not. Leading me to believe that some other logger, root(?), is handling the call? But, if I change that to use [console_basic] the messages are still the same. That is, one would expect the time and levelname to no longer be there, but they are!

Again, I do not pretend to understand what's going on, but where I thought the documentation showed simple inheritance I am beginning to wonder it's a bit more complicated than that. What am I doing wrong?

When I fix my spelling mistake and remove the logger for test, I still get the same behavior. Turning propagation on so that console logs will got to root logger, which has [console_basic] still shows the messages using the old format.

Making the following changes to my yaml, seems to fix the issues, as pointed out by @blues.

root:
    level: NOTSET
    handlers: [console_basic]
    propagate: no

loggers:
  __main__:
    level: DEBUG
    handlers: [console_basic]
    propagate: no

  Utils.paragraph_processing:
    level: DEBUG
    handlers: [info_file_handler, debug_file_handler, warn_file_handler, error_file_handler, critical_file_handler]
    propagate: no

  Utils.graphing_functions:
    level: DEBUG
    handlers: [info_file_handler, debug_file_handler, warn_file_handler, error_file_handler, critical_file_handler]
    propagate: no

Answer 1

There is two things going on here. First of all there is a misspelling of propagate in the config. It is wrongly spelled prop o gate: notice the "o" where an "a" should be. That means all the loggers do in fact propagate their logs up the hierarchy.

The second thing is that when propagation is on, the level of the ancestor loggers, in this case the root logger, is ignored and only the level of the handlers is taken into consideration. Since the console_err handler that is added to root has level DEBUG and all logs propagate to root this handler will log every log.

The relevant piece of information can be found the python documentation here :

Messages are passed directly to the ancestor loggers' handlers - neither the level nor filters of the ancestor loggers in question are considered.