PermissionError when using python 3.3.4 and RotatingFileHandler

Question

I am trying to get a rotating log file for a GUI application I am writing with python 3.3.4 and PyQt4.

import logging
import resources

logger = logging.getLogger('main.test')

def main():
    logger.setLevel(logging.DEBUG)

    fh = RotatingFileHandler(resources.LOG_FILE_PATH, maxBytes=500, backupCount=5)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
    fh.setFormatter(formatter)

    logger.addHandler(fh)
    logger.info('main')

Answer 1

Instead of adding handler to the logger object you can directly specify handler in basicConfig(). If you add RotatingFileHandler to the logger object then one object might open the log file and another at the same time might try to rename it which throws the PermissionError.

Below code seems to work pretty well.

import logging
import resources
from logging.handlers import RotatingFileHandler

logging.basicConfig(format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', handlers=[RotatingFileHandler(filename=resources.LOG_FILE_PATH, maxBytes=500, backupCount=5)])
logger = logging.getLogger('main.test')

def main():
    logger.setLevel(logging.DEBUG)
    logger.info('main')

Answer 2

Spent half a day on this as non previous answer resolved my issue.

My working solution is to use https://pypi.org/project/concurrent-log-handler/ instead of RotatingFileHandler. In multiple thread scenarios like Flask app, PermissionError will be raised when we rotate the log file that reaches maximum size.

Install pypiwin32 to get rid of No Module name win32con error.

Thanks go to https://www.programmersought.com/article/43941158027/

Answer 3

You cannot specify the same filename in both basicConfig() and RotatingFileHandler(). I had this same issue and I removed the filename parameter from basicConfig() and it now works.

Answer 4

Check that the file isn't being kept open by eg Windows file indexing, anti-virus or other software. Files that are open can't be renamed.

Answer 5

I changed the application to use dictConfig and created a separate file that holds the dictionary configuration. At the top of my main application I have:

from log.logger import LOGGING

logging.config.dictConfig(LOGGING)
logger = logging.getLogger('testlogging')

Then in log.logger I have:

import logging
import sys
import resources

LOGGING = {
        "version":1,
        "handlers":{
                    "fileHandler":{
                        "class":"logging.handlers.RotatingFileHandler",
                        "formatter":"myFormatter",
                        "filename":resources.LOG_FILE_PATH,
                        "maxBytes":100000,
                        "backupCount":5
                        },
                    "console":{
                        "class":"logging.StreamHandler",
                        "formatter":"myFormatter"
                        }
                    },        
        "loggers":{
            "aoconnect":{
                "handlers":["fileHandler", "console"],
                "level":"DEBUG",
                }
            },

        "formatters":{
            "myFormatter":{
                "format":"%(asctime)s - %(name)s - %(levelname)s - %(message)s"
                }
            }
        }

This all seems to work pretty well.

Answer 6

In My case file size is full, after removing server.log file it worked

LOGS_DIR = os.path.join(BASE_DIR, 'logs')
LOGGING = {
    'version': 1,
    'handlers': {
    'log_file': {
        'level': 'INFO',
        'class': 'logging.handlers.RotatingFileHandler',
        'filename': os.path.join(LOGS_DIR, 'server.log'),
        'backupCount': 10, # keep at most 10 log files
        'maxBytes': 5*1024*1024 # 5242880 bytes (5MB)
        },
    },
    'loggers': {
        'django': {
            'handlers':['log_file'],
            'propagate': True,
            'level':'INFO',
        },
    },
}

Answer 7

In my case (Windows Server 2016 + IIS + FastCGI + Flask) finally I fix it by turning off files indexing in the folder. how-to

Source: https://stackoverflow.com/a/22467917/9199668

Btw, it was working for months correctly... I have no idea why...

Answer 8

In my case it happens only in Windows. To solve it I changed the delay<\/strong> parameter to True for my TimedRotatingFileHandler log handler.