[英]Bash script to read ids from file and spawn process in a controlled manner
I need to read hundreds of ids from a file and pass those to another shell script as parameters to be spawned as separate child requests. 我需要从文件中读取数百个ID,并将其作为另一个独立的子请求产生的参数传递给另一个Shell脚本。 [Done]
[完成]
But we cannot spawn more than 6 child requests ie, not more than 6 requests can be running at a given point in time. 但是我们不能产生超过6个子请求,即,在给定的时间点可以运行的请求不超过6个。
I have gone through this site (references given below) and others and came to know that you can get the PID of the spawned process using $! 我浏览了该站点(下面给出了引用)以及其他信息,然后才知道可以使用$获得派生进程的PID! but am faraway from implementing it as I do not know how to store them in an array and delete it once the spawned process is complete.
但是与实现它相去甚远,因为我不知道如何将它们存储在数组中,并在完成生成过程后将其删除。
Forking / Multi-Threaded Processes | 分叉/多线程进程| Bash
重击
How to wait in bash for several subprocesses to finish and return exit code !=0 when any subprocess ends with code !=0? 如何在bash中等待多个子进程完成并在任何子进程以代码!= 0结尾时返回退出代码!= 0?
#!/bin/bash
file="/usr/share/nginx/html/cron/userids.txt" //file containing the userids that needs to be spawned
MAXCOUNT=6 //maximum number of child request that can be spawned
while IFS= read -r line
do
#submit a background job here
sh fetch.sh $line & //this is the shell script that needs to be submitted
//check if the spawned request count is less than MAXCOUNT
//If it is then wait for one or more request to finish
//If the number of child requests is less than MAXCOUNT then spawn another request
//if all the lines are read then wait till all the child process completes and then exit
done <"$file"
Please be cognizant that I am newbie and do not know much about the shell process. 请认识到我是新手,对shell程序了解不多。
Will appreciate any directions and feedback. 将不胜感激任何指示和反馈。
Thanks 谢谢
您可以使用xargs
生成最大数量的进程,以传递从stdin读取的参数
xargs -n 1 -P 6 fetch.sh < "$file"
You can use GNU Parallel for this: 您可以为此使用GNU Parallel :
parallel -j6 -a "$file" fetch.sh
It has lots of options for handling failures, progress bars, logging etc. 它有很多选项可以处理故障,进度条,日志记录等。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.