[英]python communicate() hangs after process terminate()
The below code tries to run multiple commands in parallel and each command has a timeout.下面的代码尝试并行运行多个命令,每个命令都有一个超时。 If processing is not completed by timeout it is to be stopped (I am using terminate()).
如果超时未完成处理,则将停止(我正在使用终止())。
The issue is after termination (returncode is set to -ve) the communicate() method hangs and when forced exit (Ctrl+C) then following error is displayed.问题是在终止后(返回码设置为 -ve),communication() 方法挂起,当强制退出 (Ctrl+C) 时,会显示以下错误。
(stdout, stderr) = proc.communicate()
File "python3.7/subprocess.py", line 926, in communicate
stdout = self.stdout.read()
procList = []
for app in appList:
try:
p = subprocess.Popen(app['command'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
procList.append((app, p))
except Exception as e:
print(e)
start = time.time()
while len(procList):
time.sleep(30)
try:
for app, proc in procList:
if (time.time() - start > app['timeoutSec']):
proc.terminate()
if proc.poll() is not None and app['mailSent'] == 0:
(stdout, stderr) = proc.communicate() #Hangs here is the process is terminated
send_results_mail('Execution Completed or Terminated')
app['mailSent'] = 1
except subprocess.SubprocessError as e:
print(e)
procList = [(app, proc) for (app, proc) in procList if app['mailSent'] == 0]
EDIT: Here is a working example, using kill(), with a misbehaving child (that won't respond to terminate()) but I'm not sure of the nature of your child process.编辑:这是一个工作示例,使用 kill(),有一个行为不端的孩子(不会响应 terminate()),但我不确定您的子进程的性质。 Hope this helps get closer to a solution for you!
希望这有助于您更接近解决方案!
Simple child program, Python 3:简单的子程序,Python 3:
import signal
import time
def sighandler(signal, _stack):
print(f"Ignoring signal {signal}")
signal.signal(signal.SIGTERM, sighandler)
signal.signal(signal.SIGINT, sighandler)
secs = 10
print(f"Sleeping {secs} seconds...")
time.sleep(secs)
print("Exiting...")
Revised parent program, Python 3:修订后的父程序,Python 3:
import subprocess
import time
start = time.time()
appList = [
{'command': 'python ./child.py', 'timeoutSec': 2, 'mailSent': 0},
{'command': 'python ./child.py', 'timeoutSec': 2, 'mailSent': 0},
{'command': 'python ./child.py', 'timeoutSec': 2, 'mailSent': 0},
]
def logmsg(msg):
elap = time.time()-start
print(f"{elap:2.1f} secs: {msg}")
def send_results_mail(result):
logmsg(f"Result: {result}")
procList = []
for app in appList:
try:
p = subprocess.Popen(app['command'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
procList.append((app, p))
except Exception as e:
logmsg(e)
logmsg(f"Launching child... {p.pid}")
start = time.time()
while len(procList):
time.sleep(1)
try:
for app, proc in procList:
if (time.time() - start > app['timeoutSec']):
logmsg(f"Trying to terminate()...{proc.pid}")
proc.terminate()
proc.kill()
if proc.poll() is not None and app['mailSent'] == 0:
proc.kill()
logmsg(f"Trying to communicate()...{proc.pid}")
(stdout, stderr) = proc.communicate()
send_results_mail('Execution Completed or Terminated')
app['mailSent'] = 1
except subprocess.SubprocessError as e:
logmsg(e)
procList = [(app, proc) for (app, proc) in procList if app['mailSent'] == 0]
Output:输出:
$ python parent.py
0.0 secs: Launching child... 537567
0.0 secs: Launching child... 537568
0.0 secs: Launching child... 537569
2.0 secs: Trying to terminate()...537567
2.0 secs: Trying to terminate()...537568
2.0 secs: Trying to terminate()...537569
3.0 secs: Trying to terminate()...537567
3.0 secs: Trying to communicate()...537567
3.0 secs: Result: Execution Completed or Terminated
3.0 secs: Trying to terminate()...537568
3.0 secs: Trying to communicate()...537568
3.0 secs: Result: Execution Completed or Terminated
3.0 secs: Trying to terminate()...537569
3.0 secs: Trying to communicate()...537569
3.0 secs: Result: Execution Completed or Terminated
Thanks for all the suggestions and responses, but nothing solved the problem as for me the processes I was spanning were creating multiple levels of sub-processes.感谢所有的建议和响应,但没有什么能解决问题,因为我所跨越的流程正在创建多个级别的子流程。
So I had to terminate the processes recursively and for that I used the solution using psutil
所以我不得不递归地终止进程,为此我使用了使用
psutil
的解决方案
Pls see the post below for details详情请看下面的帖子
https://stackoverflow.com/a/27034438/2393961 https://stackoverflow.com/a/27034438/2393961
once all the child and grandchild processes are killed communicate()
works fine.一旦所有的子进程和孙进程都被杀死,
communicate()
可以正常工作。
Another, piece I learnt is even though htop
can show you the tree structure of the process but each process is independent, killing a parent does not automatically kill its decedents.我学到的另一件事是,即使
htop
可以向您展示进程的树结构,但每个进程都是独立的,杀死父进程并不会自动杀死其死者。 Thanks to my friend Noga for pointing that out感谢我的朋友Noga指出这一点
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.