无缓冲的urllib2.urlopen

Question

I have client for web interface to long running process. I'd like to have output from that process to be displayed as it comes. Works great with urllib.urlopen() , but it doesn't have timeout parameter. On the other hand with urllib2.urlopen() the output is buffered. Is there a easy way to disable that buffer?

Answer 1

urllib2 is buffered when you just call read()

you could define a size to read and therefore disable buffering.

for example:

import urllib2

CHUNKSIZE = 80

r = urllib2.urlopen('http://www.python.org')
while True:
    chunk = r.read(CHUNKSIZE)
    if not chunk:
        break
    print(chunk)

this would print the response after each chunk is read from the socket, not buffer until the entire response is received.

Answer 2

A quick hack that has occurred to me is to use urllib.urlopen() with threading.Timer() to emulate timeout. But that's only quick and dirty hack.