Python使用参数中的函数记录开销

Question

I am using python logging module (python 3.4.3). Let's say my logging level is set to INFO and I would like to use a function as an argument of my logging message.

Now I would expect when log level is decreased (eg WARNING) that the argument functions are not executed.

Is this true?

Example (console):

>>> import math
>>> import logging
>>> logging.basicConfig(level=logging.INFO)
>>> logging.debug('Factorial of 100000=',math.factorial(100000))
>>> */empty output but slightly delayed, small CPU peak/*
>>> logging.debug('Factorial of 10=',math.factorial(10))
>>> */empty output but instant, no noticeable CPU usage/*

The function is probably not executed, but still there is noticable delay. Could someone explain me please the possible cause?

Answer 1

Function arguments evaluations before call

Well you see the arguments you pass to logging.debug('Factorial of 100000=', math.factorial(100000)) need to be evaluated before logging.debug() gets called , and evaluating math.factorial(100000) takes time

It doesn't matter if ultimately the function does nothing in its call, the arguments will still get evaluated

Conceptually you can think of it as doing

value = math.factorial(100000)           # This gets evaluated
logging.debug('Factorial is %d', value)  # Even if this is a "no op" call, 
                                         # the arg evaluation still takes place

Workaround

import math
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()

if logger.getEffectiveLevel() <= logging.DEBUG:    # Or use == if you want DEBUG only
    logger.debug('Factorial is %d', math.factorial(100000))

Using Python 3's logger.getEffectiveLevel()

Given that the levels are ordered as

CRITICAL    50
ERROR       40
WARNING     30
INFO        20
DEBUG       10
NOTSET      0

You can see that our if block (with <= logging.DEBUG ) will only run on DEBUG and below (so DEBUG and NOTSET )

Answer 2

The problem is that you evaluate the function before you call. One way of dealing with this is using logger.getEffectiveLevel() as described by bakkal. This can be a bit verbose though. Another way to get around this is to create an intermediary object to pass to the logger that does the work in it's __str__ function. This delays the function call until the logging module is sure the message will be emitted.

For example:

import logging
import math

class CallableMsg:
    def __init__(self, func, *args, **kwargs):
        self.func = func
        self.args = args
        self.kwargs = kwargs

    def __str__(self):
        return str(self.func(*self.args, **self.kwargs))

logging.basicConfig(level=logging.INFO)

# single arg log
logging.info(CallableMsg(math.factorial, 10)) 
# using the CallableMsg in a format string
logging.info("10! = %s", CallableMsg(math.factorial, 10))
# factorial function never gets evaluated for the debug call
logging.debug(CallableMsg(math.factorial, 1000000))

The logging module is also kind enough to preserve the line on which the log call was actually made if there is ever an error.

logging.info(CallableMsg(math.factorial, -1))

Outputs:

--- Logging error ---
Traceback (most recent call last):
  File "C:\Python34\lib\logging\__init__.py", line 971, in emit
    msg = self.format(record)
  File "C:\Python34\lib\logging\__init__.py", line 821, in format
    return fmt.format(record)
  File "C:\Python34\lib\logging\__init__.py", line 558, in format
    record.message = record.getMessage()
  File "C:\Python34\lib\logging\__init__.py", line 319, in getMessage
    msg = str(self.msg)
  File "C:python\run.py", line 12, in __str__
    return str(self.func(*self.args, **self.kwargs))
ValueError: factorial() not defined for negative values
Call stack:
  File "<string>", line 1, in <module>
  File "C:\Python34\lib\idlelib\run.py", line 126, in main
    ret = method(*args, **kwargs)
  File "C:\Python34\lib\idlelib\run.py", line 353, in runcode
    exec(code, self.locals)
  File "C:\python\run.py", line 18, in <module>
    logging.info(CallableMsg(math.factorial, -1))
Message: <__main__.CallableMsg object at 0x02ECF970>
Arguments: ()