Windows下Python记录器`RotatingFileHandler`失败

Question

I am trying to use the RotatingFileHandler handler to manage potentially long logfiles for a Python project running under Windows. I have found that the log file rotation works fine when run using the IDLE interpreter (on a Windows PC) but fails with a WindowsError exception when run from the Windows command prompt.

Here is an example code that shows the problem -

import logging, logging.handlers
import datetime
import time

mainlogfile = 'fred.log'
logging_level = logging.DEBUG
logging_rotate_time = datetime.timedelta(minutes=1)

logger = logging.getLogger('Main_Logger')
logger.setLevel(logging.DEBUG)

handler = logging.handlers.RotatingFileHandler(mainlogfile, backupCount=7)
handler.setLevel(logging_level)
formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s')
handler.setFormatter(formatter)      
logger.addHandler(handler)

next_rotate = datetime.datetime.now() + logging_rotate_time

while True:
    time.sleep(5)
    logger.info('Tick ')
    n = datetime.datetime.now()
    if n>next_rotate:
        logger.info('Rotating logfile')
        handler.doRollover()
        logger.info('Succesfully rotated logfile')
        next_rotate = next_rotate + logging_rotate_time
        logger.info('Next logfile rotate at '+ str(next_rotate))

When I run this in the IDLE interpreter, it works fine and the log file fred.log is rotated every minute producing fred.log.1, fred.log.2 etc. with contents such as -

2018-12-10 12:24:40,269 INFO Succesfully rotated logfile
2018-12-10 12:24:40,269 INFO Next logfile rotate at 2018-12-10 12:25:40.182000
2018-12-10 12:24:45,269 INFO Tick 
2018-12-10 12:24:50,269 INFO Tick 
2018-12-10 12:24:55,269 INFO Tick 
2018-12-10 12:25:00,267 INFO Tick 
2018-12-10 12:25:05,267 INFO Tick 
2018-12-10 12:25:10,266 INFO Tick 
2018-12-10 12:25:15,266 INFO Tick 
2018-12-10 12:25:20,266 INFO Tick 
2018-12-10 12:25:25,266 INFO Tick 
2018-12-10 12:25:30,265 INFO Tick 
2018-12-10 12:25:35,265 INFO Tick 
2018-12-10 12:25:40,263 INFO Tick 
2018-12-10 12:25:40,263 INFO Rotating logfile

However, when run in a command prompt it fails on the first rotate attempt -

C:\> python try10.py
Traceback (most recent call last):
  File "try10.py", line 29, in <module>
    handler.doRollover()
  File "C:\Python27\lib\logging\handlers.py", line 142, in doRollover
    os.rename(self.baseFilename, dfn)
WindowsError: [Error 32] The process cannot access the file because it is being used by another process

I have searched and found that there are problems under Windows when trying to close or rename log files that are in use by another Python process but I can't see how that is relevant to my simple example. I have tried calling handler.flush() and handler.close() before handler.doRollover() but this did not change the behavior.

My questions are -

• Why is the exception raised when run at the command prompt but not under IDLE?
• Is there any changes I can make to the code to allow it to run at the Windows command prompt

Answer 1

Looking at the code of the doRollover() method in the logging.handlers module, it can be seen that the exception above is being raised when doRollover tries to rename the log-file fred.log to fred.log.1 using os.rename() . Trying to rename an open file like this will cause the exception under Windows, although earlier in the doRollover() method the open logfile should have been closed. Since my program as stands always fails in Windows console it looks like the logfile is not being properly closed. In the IDLE interpreter, it seems that file i/o is done differently than the console (via sockets as described in the documentation ) so maybe that this is what allows the code to run there.

Given that in order to allow doRollover() to rename the logfile, it has to be closed first and I discovered that this can be forced by temporarily removing the handler from the logger while doing the rollover - like this -

while True:
    time.sleep(5)
    logger.info('Tick ')
    n = datetime.datetime.now()
    if n>next_rotate:
        logger.info('Rotating logfile')

        logger.removeHandler(handler)      
        handler.doRollover()        
        logger.addHandler(handler)

        logger.info('Succesfully rotated logfile')
        next_rotate = next_rotate + logging_rotate_time
        logger.info('Next logfile rotate at '+ str(next_rotate))

With these mods the code works fine and the log file fred.log is rotated every minute producing fred.log.1 , fred.log.2 etc. as expected