How to get multiple independent loggers using structlog

Question

I am looking desperately for a solution for the following problem: For a project, I need multiple independent loggers let's say one for logging readable progress for the user to console (stderr) and one for collecting some stats into a json output file. I am using structlog and my example below shows the problem

import structlog

logger1 = structlog.get_logger()
logger1.info('make something that needs to be logged', x=1, y=2, z='abc')

logger2 = structlog.get_logger()
structlog.configure(processors=[structlog.processors.JSONRenderer()])
logger2.info('structured_output', a=[1, 2, 3], b={'a': 1})

logger1.info('now this one is also json but it shouldn\'t')

gives the output

2022-06-08 08:30.34 [info     ] make something that needs to be logged x=1 y=2 z=abc
{"a": [1, 2, 3], "b": {"a": 1}, "event": "structured_output"}
{"event": "now this one is also json but it shouldn't"}

logger1 works fine until I apply the configuration for json rendering.

What I would like to see is

2022-06-08 08:30.34 [info     ] make something that needs to be logged x=1 y=2 z=abc
{"a": [1, 2, 3], "b": {"a": 1}, "event": "structured_output"}
2022-06-08 08:39.41 [info     ] still no json output, great!

How can I keep both loggers separate so that the configuration for logger1 is unchanged?

Thanks! Max

Answer 1

This is a bit more involved, but it's possible using structlog's most powerful standard library integration strategy , where every log entry goes through the standard library and that allows you to use the standard library for:

1. multiplexing
2. routing the log entries by logger name.

Multiplexing is demonstrated at the bottom of the linked above, with an example of how to have colorful logs in the terminal and plain ones into a file.

What's missing now is to dispatch between them based on the name of the logger. You'd do something like structlog.[stdlib.]get_logger("json-only") to set it to a well-known name that is used a logger name and then can be referred to in standard library logging config.

It's not as straight-forward as I'd like it to be, but the example in the docs should provide a good place to start.

Answer 2

nevermind, I was overcomplicating things... I leave the structlog logging to console as it is and write a small custom json-report class that does the trick. Thanks for your help though!