从PHP并行执行脚本并等待

Question

I have to start many scripts from PHP and finish them as fast as possible. These scripts are written in a different language, though. I've started them somehow but I don't know how to wait for them. My code:

foreach ($commands as $command) {
    exec($command.' &');
}

The PHP script stop execution but there is still a lot of work in the background. How to wait in PHP for these tasks?

Similar questions:
Parallel processing in PHP - How do you do it? – waiting/synchronisation ignored
Executing multiple PHP scripts in parallel, and being notified when finished – limited to PHP scripts

Answer 1

If the scripts to execute are written in different languages, the best approach is probably GNU parallel :

Note the documentation:

If command is given, GNU parallel solve the same tasks as xargs. If command is not given GNU parallel will behave similar to cat | sh.

So you can easily build, in PHP, a list of task to do separated by newlines, and feed them to parallel like this:

echo $'ls\necho 42\npython3 -c "print(43)"' | parallel 
1
2
3
42
43

Feeding the list of tasks to parallel 's stdin is not that hard using proc_open in PHP.

Then you'll just have to wait for parallel to complete, when parallel has terminated, all subtasks are terminated too, and you can easily configure how many tasks are executed in parallel, if you have too many, executing them all in parallel like you're trying to do will be slower than starting them 5 by 5 like parallel can typically do.