Python subprocess.run 超时在 Windows 与 Linux 上表现不同

Question

以下代码片段在python 3.6.8上运行似乎在 Windows 与 Linux 上的行为非常不同

from subprocess import run, PIPE
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime as dt
import threading

def run_command(sleep, timeout):
    try:
        command = f"ping 127.0.0.1 -n {sleep + 1}"
        print(f"{dt.now()} - {threading.get_ident()} - {command}")
        out = run(command, timeout=timeout, stdout=PIPE, universal_newlines=True, shell=True)
        print(out)
    except Exception as e:
        print(f"{dt.now()} - {threading.get_ident()} - ", e)

if __name__ ==  "__main__":
    with ThreadPoolExecutor(max_workers=2) as e:
        e.submit(run_command, 10, 1)
        e.submit(run_command, 10, 5)

我一直期望得到 Linux 行为，即打印：

2021-03-04 15:15:40.276109 - 140510989964864 - ping 127.0.0.1 -n 11
2021-03-04 15:15:40.276998 - 140510981572160 - ping 127.0.0.1 -n 11
2021-03-04 15:15:41.279731 - 140510989964864 -  Command 'ping 127.0.0.1 -n 11' timed out after 1 seconds
2021-03-04 15:15:45.284004 - 140510981572160 -  Command 'ping 127.0.0.1 -n 11' timed out after 5 seconds

但是当我在 Windows 上运行时，似乎subprocess.run超时“一起”，请注意超时时间。 我不知道问题是否也可能出在 ThreadPoolExecutor 上

2021-03-04 15:14:49.671291 - 376104 - ping 127.0.0.1 -n 11
2021-03-04 15:14:49.671291 - 366200 - ping 127.0.0.1 -n 11
2021-03-04 15:15:00.179961 - 376104 -  Command 'ping 127.0.0.1 -n 11' timed out after 1 seconds
2021-03-04 15:15:00.179961 - 366200 -  Command 'ping 127.0.0.1 -n 11' timed out after 5 seconds

从 Windows 运行它时，我必须做什么才能获得相同的行为？

Answer 1

尝试使用shell=False调用run 。 您将必须使用command = command.spit()拆分命令，以便您现在传递以run列表：

from subprocess import run, PIPE
from concurrent.futures import ThreadPoolExecutor
from datetime import datetime as dt
import threading

def run_command(sleep, timeout):
    try:
        command = f"ping 127.0.0.1 -n {sleep + 1}"
        command = command.split()
        print(f"{dt.now()} - {threading.get_ident()} - {command}")
        out = run(command, timeout=timeout, stdout=PIPE, universal_newlines=True, shell=False)
        print(out)
    except Exception as e:
        print(f"{dt.now()} - {threading.get_ident()} - ", e)

if __name__ ==  "__main__":
    with ThreadPoolExecutor(max_workers=2) as e:
        e.submit(run_command, 10, 1)
        e.submit(run_command, 10, 5)

印刷：

2021-03-06 16:45:04.011326 - 8516 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:45:04.012325 - 28136 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:45:05.022674 - 8516 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 1 seconds
2021-03-06 16:45:09.023190 - 28136 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 5 seconds

更新

我切换到 Python 3.6.4 并看到了 OP 的问题。 只是为了它，我转而使用使用multiprocessing.pool.Pool的等效实现：

from subprocess import run, PIPE
from multiprocessing.pool import ThreadPool
from datetime import datetime as dt
import threading

def run_command(sleep, timeout):
    try:
        command = f"ping 127.0.0.1 -n {sleep + 1}"
        command = command.split()
        print(f"{dt.now()} - {threading.get_ident()} - {command}")
        out = run(command, timeout=timeout, stdout=PIPE, universal_newlines=True, shell=False)
        print(out)
    except Exception as e:
        print(f"{dt.now()} - {threading.get_ident()} - ", e)

if __name__ ==  "__main__":
    p = ThreadPool(2)
    p.apply_async(run_command, (10, 1))
    p.apply_async(run_command, (10, 5))
    p.close()
    p.join()

然后我跑了几次，一次（第三次）结果是正确的（见下文）。 所以我用ThreadPoolExecutor回到原来的版本，在 Python 3.6.4 下运行了大约 10 次，但没有成功产生正确的答案。

C:\Program Files\Python36>python -V
Python 3.6.4

C:\Program Files\Python36>python \booboo\test\test.py
2021-03-06 16:53:52.051412 - 21732 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:53:52.052379 - 11744 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:53:57.058553 - 21732 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 1 seconds
2021-03-06 16:53:57.060555 - 11744 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 5 seconds

C:\Program Files\Python36>python \booboo\test\test.py
2021-03-06 16:58:58.173625 - 17672 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:58:58.173625 - 22920 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:59:03.181457 - 17672 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 1 seconds
2021-03-06 16:59:03.186461 - 22920 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 5 seconds

C:\Program Files\Python36>python \booboo\test\test.py
2021-03-06 16:59:19.643543 - 22272 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:59:19.643543 - 30580 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:59:20.654799 - 22272 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 1 seconds
2021-03-06 16:59:24.651398 - 30580 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 5 seconds

C:\Program Files\Python36>python \booboo\test\test.py
2021-03-06 16:59:46.907140 - 27496 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:59:46.907140 - 30460 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 16:59:51.915997 - 27496 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 1 seconds
2021-03-06 16:59:51.922999 - 30460 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 5 seconds

C:\Program Files\Python36>

更新 2

使用多处理，它可以工作：

from subprocess import run, PIPE
from concurrent.futures import ProcessPoolExecutor
from datetime import datetime as dt
import threading

def run_command(sleep, timeout):
    try:
        command = f"ping 127.0.0.1 -n {sleep + 1}"
        command = command.split()
        print(f"{dt.now()} - {threading.get_ident()} - {command}")
        out = run(command, timeout=timeout, stdout=PIPE, universal_newlines=True, shell=False)
        print(out)
    except Exception as e:
        print(f"{dt.now()} - {threading.get_ident()} - ", e)

if __name__ ==  "__main__":
    with ProcessPoolExecutor(max_workers=2) as e:
        e.submit(run_command, 10, 1)
        e.submit(run_command, 10, 5)

C:\Program Files\Python36>python \booboo\test\test.py
2021-03-06 17:11:32.402874 - 7724 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 17:11:32.403878 - 26884 - ['ping', '127.0.0.1', '-n', '11']
2021-03-06 17:11:33.416621 - 7724 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 1 seconds
2021-03-06 17:11:37.414169 - 26884 -  Command '['ping', '127.0.0.1', '-n', '11']' timed out after 5 seconds