python subprocess32 超时，溢出错误

Question

this is my log

File "/opt/ibm/db2-governor/helpers/utils.py", line 10, in run_cmd
output = proc.communicate(timeout = timeout)[0]
  File "/opt/ibm/dynamite/python/lib/python2.7/site-packages/subprocess32.py", line 927, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/opt/ibm/dynamite/python/lib/python2.7/site-packages/subprocess32.py", line 1713, in _communicate
    orig_timeout)
  File "/opt/ibm/dynamite/python/lib/python2.7/site-packages/subprocess32.py", line 1786, in _communicate_with_poll
    ready = poller.poll(self._remaining_time(endtime))
OverflowError: Python int too large to convert to C lon

so the code that triggers this is

output = proc.communicate(timeout = timeout)[0]

timeout is set to 20, this happens intermitently (almost never but it happens), im using python 2.7.11 with subprocess32 library, is this a python bug?

ok, i checked subprocess32.py, the line goes like this

endtime = time.time() + timeout

ready = poller.poll(self._remaining_time(endtime))

so basically timestamp is too large to convert into ac int, is there anything i can do to resolve this?

Answer 1

Sounds like a bug all right.

If you're interested, here's a workaround proposal: instead of communicate , read from process stdout in a thread and check if process is over by either nothing more to read or return code yield through poll .

Since you control the loop, you can wait 1 second in main thread and countdown for the timeout (not extra accurate, since sleep can drift, but that would be good enough & simple). Also kill the process when reaches 0.

import threading

output = ""

def subp(p):
    global output

    while True:
        # read blocks but since we're in a thread it doesn't matter
        data = proc.stdout.read()
        if not data or proc.poll() != None:
            break
        output += data

# here create the process    
proc = subprocess...

# create a thread, pass the process handle
t = threading.Thread(target=subp,args=(proc,))

while True:
    if proc.poll() != None:
      # exit: OK
      break
    timeout -= 1
    if timeout < 0:
        # took too long: kill
        proc.terminate()
        break
    time.sleep(1)

t.join()