Intermitent 500 HTTP errors with Flask and WSGI

Question

I'm trying to setup a Flask application on a machine running Apache and mod_wsgi. My application runs 'randomly' well, meaning that sometimes it works and sometimes I refresh it and it throws an Internal Server Error. It seems quite random.. I have cleared the cache of my browser, tried a different browser, tried incognito mode, asked a friend to try from his laptop. It always shows this intermitent 500 behaviour.

Does anyone have any ideas where I can look for the cause? Or maybe you had this problem before?

All the data I can think of about this is below, let me know if you need anything else.

Thanks!

---

The Apache error_log shows the following when the refreshing fails:

[Wed Aug 14 16:42:52 2013] [error] [client 171.65.95.100] mod_wsgi (pid=1160): Target WSGI script '/home/server/servers/flaskapp.wsgi' cannot be loaded as Python module.
[Wed Aug 14 16:42:52 2013] [error] [client 171.65.95.100] mod_wsgi (pid=1160): Exception occurred processing WSGI script '/home/server/servers/flaskapp.wsgi'.
[Wed Aug 14 16:42:52 2013] [error] [client 171.65.95.100] Traceback (most recent call last):
[Wed Aug 14 16:42:52 2013] [error] [client 171.65.95.100]   File "/home/server/servers/flaskapp.wsgi", line 5, in <module>
[Wed Aug 14 16:42:52 2013] [error] [client 171.65.95.100]     from flaskapp.frontend import app
[Wed Aug 14 16:42:52 2013] [error] [client 171.65.95.100] ImportError: cannot import name app

The application is organized like this:

flaskapp.wsgi
flaskapp/
    __init__.py (empty)
    settings.py
    frontend/
        __init__.py (app is defined here)
        static/
            style.css
        templates/
            index.html
        views.py

The init .py contains the following:

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config.from_object('flaskapp.settings')
db = SQLAlchemy(app)

import flaskapp.views

The configuration file in the Apache httpd.conf file related to this application is:

<VirtualHost *:80>
        ServerName <redacted>

        WSGIDaemonProcess flaskapp user=server group=server
        WSGIScriptAlias /flaskapp /home/server/servers/flaskapp.wsgi

        <Directory /home/server/servers/flaskapp/>
            WSGIProcessGroup flaskapp
            WSGIApplicationGroup %{GLOBAL}
            Order allow,deny
            Allow from all
        </Directory>

</VirtualHost>

Answer 1

I had the same problem on Apache+wsgi+Django. I have tried setting up wsgi in Daemon mode , as recommended by Django manual, and that seems to solve the problem. I have now done 1000 pageloads with no 500 responses.

The same solution should probably work using a Flask setup.

Answer 2

Miguel's answer makes sense and indeed restarting the server fixes this and other problems (changes not taking effect).

My uneducated guess is that the different processes running under Apache load at some point the modules of the application and when a refresh is issued they won't 'bother' to update this information.

Restarting the Apache server, thus killing all these processes and re-spawning new ones, solves the issue.

Answer 3

I have a solution that's more of a hack than a solution really but it works.

First the background, according to Graham Dumpleton in this rticle https://github.com/GrahamDumpleton/mod_wsgi/issues/198

When touching a WSGI file in daemon mode the whole process is always reloaded but that relies on the WSGI file having already been loaded successfully. To combat problem where a WSGI file fails to load, but where code it loads leaves state in memory, as can occur with Django, then you need to use the startup-timeout option of the WSGIDaemonProcess directive to specify a time period, after which a forced process restart will occur if the WSGI file still can't be loaded even after subsequent attempts.

There are suggestions for solving this problem including

• using startup-timeout in WSGIDaemonProcess
• using request-timeout in WSGIDaemonProcess
• restarting apache

If like me for some reasons, you are not comfortable with all these, a hack you can try is to trigger another error - the kind of error that will force wsgi to reload.

In my case, I commented out the line

from {app name} import app as application

in my wsgi file. This raised another error but this time forces a reload when I uncomment and then solves the initial problem (which was caused by wsgi not reloading)