Python 2.7：记录类线程RotatingFileHandler缓冲区溢出？ 错误32

Question

I am receiving the following Error when logging goes to rotate the log files:

Traceback (most recent call last):
  File "C:\Python27\Lib\logging\handlers.py", line 72, in emit
    self.doRollover()
  File "C:\Python27\Lib\logging\handlers.py", line 174, in doRollover
    self.rotate(self.baseFilename, dfn)
  File "C:\Python27\Lib\logging\handlers.py", line 113, in rotate
    os.rename(source, dest)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process
Logged from file logger_example.py, line 37

PSMON results (full picture) :

I saw in that in python 3.2 there is an implementation of a QueuedHandler for threaded logging, but I don't think that is what is going on here, as I have tried to turn logging off while threading. I even implemented a Queued Version based off of the handler.py included in python3 (like this) , but received the same error. I am obviously doing something incorrectly, but I am unable to figure it out. As it is writing to the same file from multiple classes.

• https://docs.python.org/3.5/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes
• https://docs.python.org/3.5/library/logging.handlers.html#queuehandler

logger_example.py:

class log_this(object):
    def __init__(self,module_name, MainFrame):
        super(log_this, self).__init__()
        self.MainFrame = MainFrame
        self.module_name = module_name
        LOG_FILENAME = os.path.join('logs','test.log')
        LOG_LEVEL = logging.DEBUG
        logging.getLogger(self.module_name)
        logging.basicConfig(level=LOG_LEVEL,
                            format='\n--%(asctime)s %(funcName)s %(name)-12s %(levelname)-8s %(message)s',
                            datefmt='%m-%d %H:%M:%S',
                            filename=LOG_FILENAME,
                            filemode='w')
        #console = logging.StreamHandler()
        self.handler = logging.handlers.RotatingFileHandler(
              LOG_FILENAME, maxBytes=1000000, backupCount=5)
        #console.setLevel(logging.INFO)
        self.count = 0


    def log_info(self,name,msg):
        print('In: log_info')
        try:
            log_name = logging.getLogger(".".join([self.module_name,name]))
            log_name.addHandler(self.handler)
            if self.MainFrame.threading is False:
                log_name.info("\n%s" % (msg))
        except Exception,e:
            print(traceback.format_exc())
            print(e)
            exit()
        return

    def log_debug(self,func_name,msg,debug_info):
        try:
            log_name = logging.getLogger(".".join([self.module_name,func_name]))
            log_name.addHandler(self.handler)
            #called_frame_trace = "\n    ".join(traceback.format_stack()).replace("\n\n","\n")
            outer_frames = inspect.getouterframes(inspect.currentframe().f_back.f_back)
            call_fn = outer_frames[3][1]
            call_ln = outer_frames[3][2]
            call_func = outer_frames[3][3]
            caller = outer_frames[3][4]
            # a string with args and kwargs from log_debug
            args_kwargs_str = "\n"+str(debug_info).replace(", '","\n, '")
            if self.MainFrame.threading is False:
                results = log_name.debug("%s\nARGS_KWARGS_STR:%s\ncall_fn: %s\ncall_ln: %i\ncall_func: %s\ncaller: %s\nException:" % (
                                            msg,
                                            args_kwargs_str,
                                            call_fn,
                                            call_ln,
                                            call_func,
                                            caller
                                            ),exc_info=1)
        except Exception, e:
            print(traceback.format_exc())
            print(e)
            exit()
        return

Example usage that is being used in multiple classes across several directories/modules:

from Logs.logger_example import log_this
class Ssh(object):

    def __init__(self, MainFrame):
        super(Ssh, self).__init__()
        self.MainFrame = MainFrame
        self.logger = log_this(__name__,self.MainFrame)
    def infoLogger(self,msg=None):
        this_function_name = sys._getframe().f_back.f_code.co_name
        self.logger.log_info(this_function_name,str(msg)+" "+this_function_name)
        return
    def debugLogger(self,msg=None,*args,**kwargs):
        debug_info = {'ARGS':args, 'KWARGS':kwargs}
        this_function_name = sys._getframe().f_back.f_code.co_name
        self.logger.log_debug(this_function_name,str(msg)+this_function_name,debug_info)
        return
    #-------------------------------------------------
    # Example Usage:
    # It used like this in other classes as well
    #-------------------------------------------------
    def someMethod(self):
        self.infoLogger('some INFO message:%s' % (some_var))
        self.debugLogger('Some DEBUG message:',var1,var2,var3=something)

Example Output:

--06-05 12:35:34 log_debug display_image.EventsHandlers.MainFrameEventHandler.csvFileBrowse DEBUG    Inside:  csvFileBrowse
ARGS_KWARGS_STR:
{'ARGS': ()
, 'KWARGS': {}}
call_fn: C:\Python27\lib\site-packages\wx-3.0-msw\wx\_core.py
call_ln: 8660
call_func: MainLoop
caller: ['        wx.PyApp.MainLoop(self)\n']
Exception:
None

--06-05 12:35:34 log_info display_image.EventsHandlers.MainFrameEventHandler.csvFileBrowse INFO     
Inside:  csvFileBrowse

--06-05 12:35:41 log_debug display_image.sky_scraper.fetchPage DEBUG    retailer_code,jNumberfetchPage
ARGS_KWARGS_STR:
{'ARGS': ('1', u'J176344')
, 'KWARGS': {}}
call_fn: C:\Users\User\Desktop\SGIS\SGIS_6-2-14\SGIS-wxpython-master\display_image\EventsHandlers\MainFrameEventHandler.py
call_ln: 956
call_func: onScanNumberText
caller: ['            results = self.updateMainFrameCurrentItemInfo(retailer_code)\n']
Exception:
None

Any suggestions would be greatly appreciated.

Answer 1

Error 32 is what it says - something else has the file open, so it can't be renamed by logging. Check that you only have one handler opening that file, it's not open in an editor or being scanned by an antivirus tool, etc.

Answer 2

I got the same error while developing a flask application. To solve the problem, I had to change the environmental variable from "FLASK_DEBUG=1" to "FLASK_DEBUG=0". The reason being that turning debugging on leads to threading errors. I got the solution after reading this blog