在python中创建分层记录器

Question

The Python logging module allows users to create loggers in a hierarchical fashion. Though in understand it at a high level i have difficulty in implementing such hierarchies when i code is distributed across multiple classes and the actual function flow will create a complex mesh..

I am able to do a simple hierarchy with the below code:

import logging 

logFormatter = logging.Formatter("%(asctime)s [%(levelname)-5.5s] %(childname)s  %(message)s") 

rootLogger = logging.getLogger() 
consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(logFormatter)
rootLogger.addHandler(consoleHandler)  
rootLogger.setLevel(logging.INFO) 
rootLogger.info("Hi..",extra={'childname':' '})
cL1= rootLogger.getChild("Child1") 
cL1.info("Hello..",extra={'childname': cL1.name})
cL2 = cL1.getChild('Child2') 
cL2.info("Hi .. hello..",extra={'childname':cL2.name})

The output of the above code block will be

In [8]: rootLogger.info("Hi..",extra={'childname':' '})
2017-08-23 22:52:47,703 [INFO ]    Hi..

In [10]: cL1.info("Hello..",extra={'childname': cL1.name})
2017-08-23 22:52:47,999 [INFO ] Child1  Hello..

In [12]: cL2.info("Hi .. hello..",extra={'childname':cL2.name})
2017-08-23 22:52:48,296 [INFO ] Child1.Child2  Hi .. hello..

So my with the extra parameter i am able to dump the whole hierarchy in the log.

Now how to achieve this when the code is split across multiple classes, across multiple files?

moduleA.py

class A:
    def __init__(self,name):
        self.name=name
    def doSomethingForA_1(self,**kwargs):
        self.logger(msg="Yes i did A_1",level=logging.INFO)
    def doSomethingForA_2(self,**kwargs):
        self.logger(msg="Yes i did A_2",level=logging.INFO)
    def logger(msg,level):
        '''How do i get the parent logger to create the child logger here?'''
        logger=parentLogger.getLogger(self.name)
        logger.log(msg,level,extra={'childname':self.name})

moduleAggregationType1.py

class AggregatorType1:
    def __init__(self,name):
        self.name=name
        self.objs = []
    def doAggregationType1(self,**kwargs):
        self.logger(msg="Starting aggregation Operation-1",level=logging.INFO)
        for obj in self.objs:
            obj.doSomethingForA_1()
    def logger(msg,level):
        '''How do i get the parent logger to create the child logger here?'''
        logger=parentLogger.getLogger(self.name)
        logger.log(msg,level,extra={'childname':self.name})

moduleAggregationType2.py
class AggregatorType2:
    def __init__(self,name):
        self.name=name
        self.objs = []
    def doAggregationType2(self,**kwargs):
        self.logger(msg="Starting aggregation Operation-2",level=logging.INFO)
        for obj in self.objs:
            obj.doSomethingForA_2()
    def logger(msg,level):
        '''How do i get the parent logger to create the child logger here?'''
        logger=parentLogger.getLogger(self.name)
        logger.log(msg,level,extra={'childname':self.name})

With these modules say i write a script by importing them.

agg_1 = AggregatorType1(name='agg_1')

agg_1.objs = [A(name='aType1_1'),A(name='aType1_2'),A(name='aType1_3')]

agg_2 = AggregatorType1(name='agg_1')

agg_2.objs = [A(name='aType2_1'),A(name='aType2_2'),A(name='aType2_3')]

In this case my logging should be:

2017-08-23 22:52:47,999 [INFO ] agg_1 Starting aggregation Operation-1
2017-08-23 22:52:47,999 [INFO ] agg_1.aType1_1 Yes i did A_1
2017-08-23 22:52:47,999 [INFO ] agg_1.aType1_2 Yes i did A_1
2017-08-23 22:52:47,999 [INFO ] agg_1.aType1_3 Yes i did A_1
2017-08-23 22:52:47,999 [INFO ] agg_2 Starting aggregation Operation-2
2017-08-23 22:52:47,999 [INFO ] agg_2.aType2_1 Yes i did A_2
2017-08-23 22:52:47,999 [INFO ] agg_2.aType2_2 Yes i did A_2
2017-08-23 22:52:47,999 [INFO ] agg_2.aType2_3 Yes i did A_2

So at the time of execution of each method i should be determining what is my parent logger to which my log should be propagated..

Now we can have a decorator for all the callable methods and we can determine the parent logger in that and create a child logger and delete it at the end of the decorator. But still the question remains..

HOW TO DETERMINE THE PARENT LOGGER AT RUN TIME AT EACH METHOD LEVEL?

Answer 1

Plain answer you can not. There is no such thing as "parent logger". What python logging system promises is that you can reference by name the same logger from anywhere in your interpreter process.

So your options are (as I see it):

1) Either statically declare where to log when constructing variable (which is ugly and error prone)

2) use built-in %(module)s and %(funcName)s modifiers. For that you will have to stop using your own logger method, as it will mask off actual name of method you are inside of

3) use traceback module for getting your stack trace in runtime

I have modified a little bit your code to reflect all those changes (and dropped your optional parameter on the way)

class A():
    def __init__(self, name):
        logger = logging.getLogger(name)
        self.log = logger.log
    def doSomethingForA_1(self,**kwargs):
        self.log(msg="Yes i did A_1",level=logging.INFO)
    def doSomethingForA_2(self,**kwargs):
        stack = traceback.extract_stack(limit=2)
        self.log(msg='Caller=%s:%d>%s>'%(stack[0][0:3]), level=logging.INFO)

in main:

logFormatter = logging.Formatter("[%(levelname)s] %(name)s <%(module)s.%(funcName)s>  %(message)s")


agg_1 = AggregatorType1(name='agg_1')
agg_2 = AggregatorType2(name='agg_2')

agg_1.objs = [A(name='agg_1.aType1_1'),A(name='agg_1.aType1_2'),A(name='agg_1.aType1_3')]
agg_2.objs = [A(name='agg_2.aType2_1'),A(name='agg_2.aType2_2')]

agg_1.doAggregationType1()
agg_2.doAggregationType2()

This will provide the following output:

[INFO] root <main.<module>>  Hi..
[INFO] agg_1 <moduleAggregationType1.doAggregationType1>  Starting aggregation Operation-1
[INFO] agg_1.aType1_1 <moduleA.doSomethingForA_1>  Yes i did A_1
[INFO] agg_1.aType1_2 <moduleA.doSomethingForA_1>  Yes i did A_1
[INFO] agg_1.aType1_3 <moduleA.doSomethingForA_1>  Yes i did A_1
[INFO] agg_2 <moduleAggregationType2.logger>  Starting aggregation Operation-2
[INFO] agg_2.aType2_1 <moduleA.doSomethingForA_2>  Caller=<moduleAggregationType2.py:11.doAggregationType2>
[INFO] agg_2.aType2_2 <moduleA.doSomethingForA_2>  Caller=<moduleAggregationType2.py:11.doAggregationType2>