用于登录Django的文件描述符不会更新

Question

We use Python(2.7)/Django(1.8.1) and Gunicorn(19.4.5) for our web application and supervisor(3.0) to monitor it. I have recently encountered 2 issues in logging:

1. Django was logging into previous day logs(We have log rotation enabled)
2. Django was not logging anything at all.

The first scenario is understandable where the log rotation changed the file but Django was not updated.

The second scenario fixed when I restarted the supervisor process. Which led me to believe again the file descriptor was not updated in the django process.

I came by this SO thread which states:

Each child is an independent process, and file handles in the parent may be closed in the child after a fork (assuming POSIX). In any case, logging to the same file from multiple processes is not supported.

So I have few questions:

1. My gunicorn has 4 child processes and if one of them fails while writing to a log file will the other child process won't be able to use it? and how to debug these kind of scenarios?

2. Personally I found debugging errors in python logging module to be difficult. Can some one point how to debug errors such as this or is there any way I can monkey patch logging to not fail silently? *(Kindly read update section)*

3. I have seen Django LogRotation causes the Issue type 1 as explained above and not some script scheduled via cron. So what is preferable?

Note: The logging config is not a problem. I have already spent fair amount of time trying to figure that out. Also if the config is the issue Django will not write log files after a process restart.

Update :

For my second question I see that logging modules provides an option to raiseExceptions on failure although this is discourages in production environment. Documentation here . So now my question becomes how do I set this in Django?

Answer 1

I felt like closing this question. Bit awkward and seems stupid after 2 months. But I guess being stupid is part of the learning and want this to be as a reference for people who stumble across this.

Scenario 1: Django on using TimedRotatingFileHandler seems not to update the file descriptor some times and hence writes to old log files unless we restart the supervisor. We are yet to find the reason for this behaviour and update the reason if found. For now we are using WatchedFileHandler and then using logrotate utility to rotate the logs.

Scenario 2: This is the stupid question. When I was logging with some string formatting I forgot to give enough variables which is why the logger was erring. But this didn't get propagated. But locally when I was testing I found that logging module was actually throwing that error but silently and any logs after it in the module were not getting printed. Lessons learns from this scenario were:

1. If there is a problem in logging find out if the string formatting does not err
2. Using log.debug('example: {msg}'.format(msg=msg)) of python instead of log.debug('example: %s', msg) .