Python：讓迭代器遍歷不同線程上的日志消息

Question

我有2個線程在運行：

# Thread 1: In model
logging.getLogger('ui').info("Sit still, I'm computing...")
more_stuff = compute_stuff(stuff)
logging.info("Ok, I'm done.")

...

# Thread 2: In view, streaming messages to client
for message in log_iterator('ui'):
    send_to_client(message)

我希望log_iterator掛起，直到出現日志消息，然后在出現日志消息時，對其進行處理。 問題是，您如何制作log_iterator？ 還是有更好的方法實現這一目標？

Answer 1

您可以通過1個線程將日志寫入文件。 第二個線程可以讀取文件並顯示日志。

這是日志記錄配置：

import logging
logger = logging.getLogger('simple_example') # set logger
logger.setLevel(logging.INFO) # set logger level

fh = logging.FileHandler('path-to-log-file') #set Handler
fh.setLevel(logging.INFO) #set Handler level

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') # set log format
fh.setFormatter(formatter) # add format to Handler

logger.addHandler(fh) # add Handler to logger

在我的示例函數中， 將日志重寫為另一個文件：

def print_message():
    for i in range(0,1000):     
        f = open('path-to-log-file', 'r')
        s = f.read()
        f.close()
        f2 = open('path-to-log-file2', 'r+')
        f2.write(s)
        f2.close()
        time.sleep(.2)

import threading
t = threading.Thread(target=print_message)
t.start()
logger.error('error message1')
logger.error('error message2')
logger.error('error message3')
t.join()

因此，記錄器通過第一個線程將日志寫入文件，第二個線程從文件中讀取日志。 我認為這就是您想要的。

同樣，如果您希望避免將日志寫入文件，則可以創建自己的Handler，該Handler將日志准確發送到第二個線程。 我認為這可能很復雜，但是我不太了解線程。

這是日志記錄文檔： https : //docs.python.org/2/howto/logging.html

Answer 2

好吧，經過一番摸索，這是我想出的解決方案：

def log_iterator(logger_names):
    '''
    Get an iterator that returns LogRecord objects whenever a log message is made.

    logger_names can be:
        a string identifying which logger you want to get messages iterate from
        a list identifying all the loggers you want to get messages from
        a dict<name: level> where level identifies the minimum loggling level 
            you want (default is INFO)
    '''
    if isinstance(logger_names, str):
        logger_names = {logger_names: logging.INFO}
    elif isinstance(logger_names, (list, tuple)):
        logger_names = {name: logging.INFO for name in logger_names}

    log_message_event = threading.Event()
    for logger_name, log_level in logger_names.iteritems():
        logger = logging.getLogger(logger_name)
        logger.addHandler(FireOnLog(log_message_event, log_level))

    while True:
        log_message_event.wait()
        record = log_message_event.record
        log_message_event.clear()
        yield record


class FireOnLog(logging.StreamHandler):
    '''
    Custom loghandler that fires an event every time a log message 
    comes in.  To use, you need to add this handler to a logger.
    '''

    def __init__(self, event, log_level = logging.INFO):
        '''
        Event is a threading.Event object
        log_level is the minimum logging level you want
        '''
        logging.StreamHandler.__init__(self)
        self._event = event
        self._log_level = log_level

    def emit(self, record):
        if record.levelno >= self._log_level:
            self._event.record = record
            self._event.set()

使用以下代碼測試時，它的行為正確：

def _print_messages():
    print 'Started Printing loop'
    for log_record in log_iterator(['testlog', 'anotherlog']):
        print '%s:%s:%s' % (log_record.levelname, log_record.name, log_record.message)


def test_log_handling():

    t = threading.Thread(target = _print_messages)
    t.start()

    time.sleep(.1)
    logging.getLogger('testlog').critical('Message!')
    logging.getLogger('anotherlog').critical('Another Message right away')
    logging.getLogger('badlog').critical("Someething useless that you don't want to hear")
    time.sleep(.1)
    logging.getLogger('testlog').critical('Another Message after some wait')

似乎這是python具有內置功能的非常通用的功能，因此，如果有人知道如何使用內置功能執行此操作，請做出回應，我會給您一點意見！