Python 具有独立进程的多进程日志记录

Question

目前，Python 的标准文档推荐使用带有 QueueHandler/QueueListener 的多进程队列来实现多进程日志记录。 但是，由于多进程队列是使用未命名管道实现的，因此独立进程无法共享同一个队列。

我对其他可行的解决方案感兴趣； 我个人的想法是，可以改为实现基于命名管道的队列，这应该允许队列在多个独立进程之间共享。

Answer 1

正常记录对我来说很好：

#!/usr/bin/env python

import uuid
import time
import multiprocessing
import logging

logging.basicConfig(filename='multiprocess.log', level=logging.DEBUG)


def act(p):
    start_time = time.monotonic()
    work()
    logging.info(f"{p} took {time.monotonic() - start_time:.3f} seconds")


def work(start_of_uuid='00000'):
    while True:
        uid = uuid.uuid4().hex
        if uid.startswith(start_of_uuid):
            return uid


if __name__ == '__main__':
    start_time = time.monotonic()
    with multiprocessing.Pool(multiprocessing.cpu_count() - 1) as pool:
        pool.map(act, range(24))
    logging.info(f"Elapsed time overall was {time.monotonic() - start_time:.3f} seconds")

在 multiprocess.log 中生成：

INFO:root:5 took 0.048 seconds
INFO:root:9 took 2.191 seconds
INFO:root:6 took 3.581 seconds
INFO:root:11 took 4.800 seconds
INFO:root:14 took 0.295 seconds
INFO:root:8 took 5.361 seconds
INFO:root:1 took 5.577 seconds
INFO:root:3 took 10.739 seconds
INFO:root:15 took 5.605 seconds
INFO:root:2 took 16.118 seconds
INFO:root:4 took 16.931 seconds
INFO:root:20 took 3.577 seconds
INFO:root:16 took 15.270 seconds
INFO:root:13 took 17.115 seconds
INFO:root:21 took 4.529 seconds
INFO:root:22 took 4.715 seconds
INFO:root:23 took 4.897 seconds
INFO:root:10 took 25.925 seconds
INFO:root:12 took 24.482 seconds
INFO:root:18 took 16.629 seconds
INFO:root:0 took 28.958 seconds
INFO:root:19 took 20.696 seconds
INFO:root:7 took 34.055 seconds
INFO:root:17 took 36.142 seconds
INFO:root:Elapsed time overall was 41.827 seconds