在python中挂起http POST请求该怎么办？

Question

Sample code:

socket.setdefaulttimeout(150)

MechBrowser = mechanize.Browser()
Header = {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8 GTB7.1 (.NET CLR 3.5.30729)'}
Url = "http://example.com"
Data = "Justatest=whatever&letstry=doit"
Request = urllib2.Request(Url, Data, Header)
Response = MechBrowser.open(Request)
Response.close()

I don't think there's anything wrong with above codes, but every now and then I'll get hanging http POST request which prevents the whole script from continuously running. I already used socket.setdefaulttimeout(150) how come it's not working? What is the reason causing this problem? And what should I do to get rid of this?

Answer 1

Found the problem.

I've been using requests a lot recently and realized that the timeout you set in both mechanize and requests are "NOT a time limit on the entire response download" , which means if the connection is really slow and there are still data transferring, it will not timeout, which could hang the connection for quite a long time.

So what I've done is wrap those requests with threads and set timeout for those threads, and this way timeout is more accurate, just make sure you clean up/garbage collect those hanging connections.

Answer 2

Could be many reasons - the server that you are posting to could be busy, network congestion, etc.

However, you can eliminate problems from your end by making sure you send complete headers (you are missing Content-type header).

Answer 3

i think it's because you didn't set content-length header. if a request is posted to the server, the data are submitted in the request's body. due to tcp connection 's streaming based characteristics, there's no way for the server to find out the length of request, unless you tell it explicitly in header. without knowing when your request is end , the server has to wait indefinitely.

and the timeout attribute doesn't work here because your socket isn't blocked on any recv/send operation. you've done writing data through socket, but the server thinks you have more to send.