python RotatingFileHandler名称中的变量

Question

I am trying to create a logging system that sorts a specific log into a directory pattern that resembles: logs/<NAME HERE>/<LEVEL>.log . So far it has not been working out. I wanted to see if I could implement this feature in the logging configuration file instead of programmatically making one for each case.

{
    "version": 1,
    "disable_existing_loggers": false,
    "formatters": {
        "simple": {
            "format": "%(asctime)s - %(name)s <%(levelname)s>: %(message)s"
        }
    },

    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "level": "DEBUG",
            "formatter": "simple",
            "stream": "ext://sys.stdout"
        },

        "info_file_handler": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "INFO",
            "formatter": "simple",
            "filename": "logs/%(name)s/info.log",
            "maxBytes": "10485760",
            "backupCount": "20",
            "encoding": "utf8"
        },

        "error_file_handler": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "ERROR",
            "formatter": "simple",
            "filename": "logs/%(name)s/errors.log",
            "maxBytes": "10485760",
            "backupCount": "20",
            "encoding": "utf8"
        }
    },

    "loggers": {
        "my_module": {
            "level": "ERROR",
            "handlers": ["console"],
            "propagate": "no"
        }
    },

    "root": {
        "level": "INFO",
        "handlers": ["console", "info_file_handler", "error_file_handler"]
    }
}

With the configuration shown above, I get ValueError: Unable to configure handler u'error_file_handler': [Errno 2] No such file or directory: u'.../logs/%(name)s/errors.log' . So, the approach I took does not support variable names as part of the filename? Am I confined to programmatically creating a logging object or is there a way to do this in Python's default logging library? I know that I can create specific loggers for each individual class, but I would really appreciate not requiring myself to manually write out each logger for each case and keep the logging config to unique cases. I would also like to use the config file if I could.

Answer 1

The logging configuration system allows you to specify handlers that you want to create, but it isn't designed to allow you to create a template for handlers, which seems to be what you want.

I don't know if "name" refers to the logger name, and it doesn't seem like your scheme makes a lot of sense, given how logging levels, loggers and handlers work. Even if you used a constant name for the log file (getting you around your current problem) you would have both ERROR and INFO level messages in info.log (as well as WARNING messages).

I wouldn't advise trying to set up separate log files per logger and per level - it's not an especially useful mode of operation, from my experience.