Cannot change oslo_log log format

Question

I was trying to configure the log format for oslo_log.log to let the log context contains the file name and line number.

It seems that the log configuration has been changed but the output on the console remains the same.

from oslo_config import cfg
from oslo_log import log 
import oslo_log

CONF = cfg.CONF
DOMAIN = "demo"
log.register_options(CONF)

LOG = log.getLogger(__name__)

_DEFAULT_LOGGING_CONTEXT_FORMAT = ('%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s')

log.set_defaults(_DEFAULT_LOGGING_CONTEXT_FORMAT, _DEFAULT_LOG_LEVELS)

log.setup(CONF, DOMAIN)

LOG.info("Welcome to Oslo Logging")
LOG.debug("A debugging message")
LOG.warning("A warning occurred")
LOG.error("An error occurred")

print(CONF.logging_context_format_string)

output without log.set_defaults():

2021-10-05 10:48:02.438 28234 INFO __main__ [-] Welcome to Oslo Logging
2021-10-05 10:48:02.439 28234 WARNING __main__ [-] A warning occurred
2021-10-05 10:48:02.439 28234 ERROR __main__ [-] An error occurred
%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s

output with log.set_defaults():

2021-10-05 10:48:31.376 28246 INFO __main__ [-] Welcome to Oslo Logging
2021-10-05 10:48:31.377 28246 WARNING __main__ [-] A warning occurred
2021-10-05 10:48:31.378 28246 ERROR __main__ [-] An error occurred
%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s

Why the object LOG behaves the same as when the config did not change?

Answer 1

You are changing the log format for default context logs (aka logging_context_format_string ), which is used for log lines where there is a oslo.context object attached. Log lines with a context can log more detailed information; such as the project or user id performing the request. This does not affect regular log lines like this where there is no context attached.

2021-10-05 10:48:31.378 28246 ERROR main [-] An error occurred

If you are trying to change regular log line (aka logging_default_format_string ), you need to either override it using oslo.configs , set_override .

eg

CONF.set_override('logging_default_format_string', '%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s')

Or you can set it in your configuration file, eg

[DEFAULT]
log_date_format = %Y-%m-%d %H:%M:%S
logging_context_format_string = %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s
logging_default_format_string = %(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s

You can load a configuration file like this for your project.

cfg.CONF(sys.argv[1:], default_config_files=['test.conf'])

The reason you cannot do this using log.set_defaults is because that function only allows for you to modify the context based log lines. You can see the implementation here .

As a bonus this is a complete example.

from oslo_config import cfg
from oslo_log import log

_DEFAULT_LOGGING_CONTEXT_FORMAT = (
    '%(asctime)s.%(msecs)03d %(process)d %(levelname)s '
    '%(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s'
)

CONF = cfg.CONF
log.register_options(CONF)
LOG = log.getLogger(__name__)
log.setup(CONF, 'test')
CONF.set_override(
    'logging_default_format_string', '%(asctime)s,%(msecs)d %(levelname)-8s '
                                     '[%(filename)s:%(lineno)d] %(message)s'
)

log.set_defaults(
    logging_context_format_string=_DEFAULT_LOGGING_CONTEXT_FORMAT,
)

# Log something
LOG.info("Welcome to Oslo Logging")
LOG.debug("A debugging message")
LOG.warning("A warning occurred")
LOG.error("An error occurred")

print('logging_default_format_string:', CONF.logging_default_format_string)
print('logging_context_format_string:', CONF.logging_context_format_string)