Use Python Logging with twisted
Question
I know how to make twisted use python logging ( docs )
But normal python logging is still swallowed. Print-statements are visible, but logger.warn('...') is not.
I use this logging set up in my library which I want to use from twisted:
import logging
logger=logging.getLogger(os.path.basename(sys.argv[0]))
class Foo:
def foo(self):
logger.warn('...')
I don't want to change my library to use twisted logging, since the library is already used in a lot of projects which don't use twisted.
If I google for this problem, I only find solutions which pipe the twisted logs in to the python logging.
How can I view see the logging of my library (without changing it)?
2 answers
solution1
0 2013-11-27 18:48:37
The thing is that Twisted is asynchronous, and avoids doing blocking I/O wherever possible. However, stdlib logging is not asynchronous, and so it does blocking I/O, and the two can't easily mix because of this. You may be able to achieve some measure of cooperation between them if you eg use a QueueHandler (introduced in the stdlib in Python 3.2 and mentioned here , but available to earlier versions through the logutils project). You can use this handler (and this handler only) to deal with events sent using stdlib logging, and your corresponding QueueListener can dispatch the events received using Twisted (non-blocking) I/O. It should work, as the queue handler shouldn't block if created with no finite capacity, and assuming that the I/O sinks can get rid of the events quickly enough (otherwise, memory would fill up).
solution2
0 2015-10-31 14:28:33
utils/log.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
import logging.handlers
from logging.config import dictConfig
from twisted.python.failure import Failure
from twisted.python import log as twisted_log
logger = logging.getLogger(__name__)
DEFAULT_LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'loggers': {
'twisted':{
'level': 'ERROR',
}
}
}
def failure_to_exc_info(failure):
"""Extract exc_info from Failure instances"""
if isinstance(failure, Failure):
return (failure.type, failure.value, failure.getTracebackObject())
def configure_logging(logfile_path):
"""
Initialize logging defaults for Project.
:param logfile_path: logfile used to the logfile
:type logfile_path: string
This function does:
- Assign INFO and DEBUG level to logger file handler and console handler
- Route warnings and twisted logging through Python standard logging
"""
observer = twisted_log.PythonLoggingObserver('twisted')
observer.start()
dictConfig(DEFAULT_LOGGING)
default_formatter = logging.Formatter(
"[%(asctime)s] [%(levelname)s] [%(name)s] [%(funcName)s():%(lineno)s] [PID:%(process)d TID:%(thread)d] %(message)s",
"%d/%m/%Y %H:%M:%S")
file_handler = logging.handlers.RotatingFileHandler(logfile_path, maxBytes=10485760,backupCount=300, encoding='utf-8')
file_handler.setLevel(logging.INFO)
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.DEBUG)
file_handler.setFormatter(default_formatter)
console_handler.setFormatter(default_formatter)
logging.root.setLevel(logging.DEBUG)
logging.root.addHandler(file_handler)
logging.root.addHandler(console_handler)
hello.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
logger = logging.getLogger(__name__)
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.resource import Resource
class BasicPage(Resource):
isLeaf = True
def render_GET(self, request):
logger.info("<html><body><h1>Basic Test</h1><p>This is a basic test page.</p></body></html>")
return "<html><body><h1>Basic Test</h1><p>This is a basic test page.</p></body></html>"
def hello():
logger.info("Basic web server started. Visit http://localhost:8000.")
root = BasicPage()
factory = Site(root)
reactor.listenTCP(8000, factory)
reactor.run()
exit()
main.py
def main():
configure_logging('logfilepath')
hello.hello()
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.