Why do Gunicorn sync workers hang when spawning threads?

Question

I have a Python log handler that spawns a new thread for each log entry , and within the new thread the log is sent to another server. However, I'm finding that the request times out intermittently. If I disable the handler, the problem goes away.

I have tried other WSGI servers (WSGIUtils, WSGIRef) and I cannot reproduce this issue.

Any ideas?

I'm running Gunicorn 19.3 with sync workers and Django 1.6 on Debian.

Answer 1

I think this is another case of GIL - Global Interpreter Lock. Assume this happens:

• Request comes in
• Worker starts
• Worker logs -> thread is started
• Worker is done
• Response is sent
• Thread logs event

Looks good, right. But what if the thread does its work faster? Then you get:

• Request comes in
• Worker starts
• Worker logs -> thread is started
• Worker is done
• Thread logs event
• Response is sent

and depending on which locks the thread holds or which OS functions it invokes, the thread might not release the GIL for some time which would block the worker or at least prevent the response from being delivered completely.

When looking at the code, I see that you're creating an SSL connection for each log message. SSL is very, very, very expensive. Don't do that. Instead, create a worker thread which opens the SSL socket once and keeps it open. The python library should release the GIL when you write to the socket so other code can work. But I'm not sure whether Python releases the lock when you open a socket.