简体   繁体   English

如何在Python中将HTTPHandler与RotatingFileHandler链接?

[英]How should I chain HTTPHandler with RotatingFileHandler in Python?

I needed to create a system, where log messages generated in an embedded system are logged remotely on a server and stored in the rotated log files. 我需要创建一个系统,在该系统中,嵌入式系统中生成的日志消息将远程记录在服务器上,并存储在轮换的日志文件中。 Due to restrictions in network communication, the log messages had to be transported via HTTP protocol. 由于网络通信的限制,必须通过HTTP协议传输日志消息。 The server runs already a Flask based HTTP server, and therefore I wanted to integrate the logging system with Flask-based web applicattion. 该服务器已经在运行基于Flask的HTTP服务器,因此我想将日志记录系统与基于Flask的Web应用程序集成在一起。

The Python logging module offers the dedicated HTTPhandler class for that purpose. Python logging模块为此提供了专用的HTTPhandler类。 Basing on the logging documentation, I have created the first implementation. 基于logging文档,我创建了第一个实现。

The server part: 服务器部分:

from flask import Flask
from flask import Response
from flask import request
import logging
import logging.handlers
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'logging server'

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       record = logging.makeLogRecord(rd)
       log1.handle(record)
       return "OK"

log1=logging.getLogger('MTEST')
log1.setLevel(logging.DEBUG)
fh=logging.handlers.RotatingFileHandler('logs','a',maxBytes=10000000,backupCount=10)
formatter = logging.Formatter('%(asctime)s %(name)-15s %(levelname)-8s %(message)s')
rfh.setFormatter(formatter)
log1.addHandler(rfh)
log1.error("First error generated locally")
app.run()

The client part: 客户端部分:

import logging, logging.handlers
myLogger = logging.getLogger('MTEST')
myLogger.setLevel(logging.DEBUG)
httpHandler = logging.handlers.HTTPHandler('localhost:5000',url='/log',method="POST")
myLogger.addHandler(httpHandler)

myLogger.info('Small info message')
myLogger.debug('Small debug message')
myLogger.erro('Small error message')

However, with that implementation the server reported the following errors (I copied one of four almost identical error messages) and only the locally generated error message was logged. 但是,通过该实现,服务器报告了以下错误(我复制了四个几乎相同的错误消息之一),并且仅记录了本地生成的错误消息。

Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/handlers.py", line 76, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python2.7/logging/handlers.py", line 156, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 741, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 465, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 329, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting

I have added printing of the form data transmitted to the server and the attributes of the created LogRecord object. 我添加了打印传输到服务器的表单数据和创建的LogRecord对象的属性的信息。 It appeared, that the args attribute is transmitted as empty tupple and not created. 似乎args属性是作为空的tupple传输的,而不是创建的。 To cure that I have modified the handle_log routine: 为了解决这个问题,我修改了handle_log例程:

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       rd['args']=""
       record = logging.makeLogRecord(rd)
       log1.handle(record)
       return "OK"

Logging still wasn't working, but the errors produced by the server have changed: 日志仍然无法正常工作,但是服务器产生的错误已更改:

Logged from file test1.py, line 9
127.0.0.1 - - [10/Jul/2018 23:52:51] "POST /log HTTP/1.0" 200 -
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/handlers.py", line 76, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python2.7/logging/handlers.py", line 156, in shouldRollover
    msg = "%s\n" % self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 741, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 467, in format
    record.asctime = self.formatTime(record, self.datefmt)
  File "/usr/lib/python2.7/logging/__init__.py", line 423, in formatTime
    ct = self.converter(record.created)
TypeError: a float is required

To check, what is wrong, I have added the filter printing the created LogRecord object: 要检查什么地方不对,我添加了打印创建的LogRecord对象的过滤器:

from flask import Flask
from flask import Response
from flask import request
import logging
import logging.handlers
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'logging server'

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       rd['args']=""
       record = logging.makeLogRecord(rd)
       log1.handle(record)
       return "OK"

class myflt(logging.Filter):
  def filter(self,rec):
    print(rec.__dict__)
    return 1

log1=logging.getLogger('MTEST')
log1.setLevel(logging.DEBUG)
rfh=logging.handlers.RotatingFileHandler('logs','a',maxBytes=10000000,backupCount=10)
formatter = logging.Formatter('%(asctime)s %(name)-15s %(levelname)-8s %(message)s')
rfh.setFormatter(formatter)
log1.addHandler(rfh)
log1.addFilter(myflt())
log1.error("First error generated locally")
app.run()

After that, I could compare the LogRecord objects created for the locally generated message: 之后,我可以比较为本地生成的消息创建的LogRecord对象:

{'threadName': 'MainThread', 'name': 'MTEST', 'thread': 139678016341824, 'created': 1531259894.112536, 'process': 5595, 'processName': 'MainProcess', 'args': (), 'module': 'srv1', 'filename': 'srv1.py', 'levelno': 40, 'exc_text': None, 'pathname': 'srv1.py', 'lineno': 37, 'msg': 'First error generated locally', 'exc_info': None, 'funcName': '<module>', 'relativeCreated': 44.27504539489746, 'levelname': 'ERROR', 'msecs': 112.53595352172852}

And for the remotely generated message: 对于远程生成的消息:

{'relativeCreated': u'12.3879909515', 'process': u'5597', 'module': u'test1', 'funcName': u'<module>', 'filename': u'test1.py', 'levelno': u'10', 'processName': u'MainProcess', 'lineno': u'9', 'msg': u'Small debug message', 'args': '', 'exc_text': u'None', 'name': u'MTEST', 'thread': u'140221336438592', 'created': u'1531259898.51', 'threadName': u'MainThread', 'msecs': u'511.312007904', 'pathname': u'test1.py', 'exc_info': u'None', 'levelname': u'DEBUG'}

It appears, that the transmission of the log record via HTTPHandler converts all numbers into strings, and therefore the makeLogRecord function is not able to correctly recreate the LogRecord object in the server. 看来,通过HTTPHandler传输日志记录会将所有数字转换为字符串,因此makeLogRecord函数无法正确地在服务器中重新创建LogRecord对象。

What is the proper way to transmit the log messages via HTTPHandler so that they may be properly handled by eg, RotatingFileHandler in a remote machine? 通过HTTPHandler传输日志消息的正确方法是什么,以便可以由远程计算机中的RotatingFileHandler正确处理它们?

I have found something, that seems to be a viable workaround. 我发现了一些似乎可行的解决方法。 The dictionary with atributes of the LogRecord object created in the client is converted to JSON before passing to the server. 在客户端中创建的带有LogRecord对象属性的字典在传递到服务器之前会转换为JSON。 The server then decodes it and recreates the original LogRecord . 然后,服务器对其进行解码并重新创建原始LogRecord

The server part: 服务器部分:

from flask import Flask
from flask import Response
from flask import request
import logging
import logging.handlers
import json
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'logging server'

@app.route('/log', methods=['POST'])
def handle_log():
    if request.method == 'POST':
       rd=request.form.to_dict()
       rec=json.loads(rd['record'])
       record = logging.makeLogRecord(rec)
       log1.handle(record)
       return "OK"

class myflt(logging.Filter):
  def filter(self,rec):
    print(rec.__dict__)
    return 1

mfl=myflt()

log1=logging.getLogger('MTEST')
log1.setLevel(logging.DEBUG)
rfh=logging.handlers.RotatingFileHandler('logs','a',maxBytes=10000000,backupCount=10)
formatter = logging.Formatter('%(asctime)s %(name)-15s %(levelname)-8s %(message)s')
rfh.setFormatter(formatter)
log1.addHandler(rfh)
log1.addFilter(mfl)
log1.error("First error generated locally")
app.run()

The client part: 客户端部分:

import logging, logging.handlers
import json

class myHTTPHandler(logging.handlers.HTTPHandler):
  def mapLogRecord(self,record):
    trec={'record':json.dumps(record.__dict__)}
    return trec

myLogger = logging.getLogger('MTEST')
myLogger.setLevel(logging.DEBUG)
httpHandler = myHTTPHandler('localhost:5000',url='/log',method="POST")
myLogger.addHandler(httpHandler)

myLogger.info('Small info message')
myLogger.debug('Small debug message')
myLogger.error('Small error message')

The above implementation works correctly, however it seems to be unnecessarily complicated. 上面的实现可以正常工作,但是似乎不必要地复杂。 Is there any simpler way to use HTTPHandler to communicate with the remote RotatingFileHandler via HTTP protocol? 有没有更简单的方法可以使用HTTPHandler通过HTTP协议与远程RotatingFileHandler通信?

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM