Django on Apache with FCGI - 500 internal server error under high load

Question

I'm using Windows Server 2008, Apache 2.2 Python 2.7 and Django as FCGI server

FCGI server is started as s service with SrvAny:


python c:\\amebas_site\\amoeba\\manage.py runfcgi host=127.0.0.1 port=8881 daemonize=True method=threaded outlog=c:\\amebas_site\\logs\\access.log errlog=c:\\amebas_site\\logs\\error.log maxchildren=100 maxspare=90 minspare=50 maxrequests=10 debug=True

I tried different maxspare, minspare, maxrequests and maxchildren values, but nothing helped. I always get "500 server error" under high load.

Apache config:

FastCGIExternalServer c:\\amebas_site\\fcgi.hook -host 127.0.0.1:8881

  RewriteEngine On ... RewriteCond ${REQUEST_FILENAME} !-f RewriteRule ^/(.*)$ /fcgi.hook/$1 [QSA,L] 

My django website works good when there are 5-6 users online, there are no errors. But when I do stress-test (50 virtual users via loadimpact.com), the site shows "500 internal server error" page and I get an error message in my apache logs:

[Thu Dec 06 16:29:48 2012] [error] [client 54.246.71.7] (OS 10054)An existing connection was forcibly closed by the remote host. : FastCGI: comm with server "C:/amebas_site/fcgi.hook" aborted: read failed [Thu Dec 06 16:29:48 2012] [error] [client 54.246.71.7] FastCGI: incomplete headers (0 bytes) received from server "C:/amebas_site/fcgi.hook"

But there is noting in Django logs. How can I debug this problem and get rid of it?

Answer 1

Look, you have maxchildren=100 1. If your request is more then 1-2seconds (possible in debug), then your server just couldn't handle all 50 connections 2. If you have static file and it's served by django - then it also uses additional connections

Solution: you could analyse logs and calculate how many requests you have during 5 seconds for example