[英]Submit a new script after all parallel jobs in R have completed
I have an R script that creates multiple scripts and submits these simultaneously to a computer cluster, and after all of the multiple scripts have completed and the output has been written in the respective folders, I would like to automatically launch another R script that works on these outputs. 我有一个R脚本,可以创建多个脚本并将这些脚本同时提交给计算机集群,并且在完成所有多个脚本并将输出写入相应的文件夹之后,我想自动启动另一个适用于这些输出。
I haven't been able to figure out whether there is a way to do this in R: the function 'wait' is not what I want since the scripts are submitted as different jobs and each of them completes and writes its output file at different times, but I actually want to run the subsequent script after all of the outputs appear. 我还无法弄清楚R中是否有办法做到这一点:函数'wait'不是我想要的,因为脚本是作为不同的作业提交的,并且每个脚本都完成并在不同的位置写入其输出文件次,但实际上我想在所有输出出现后运行后续脚本。
One way I thought of is to count the files that have been created and, if the correct number of output files are there, then submit the next script. 我想到的一种方法是对已创建的文件进行计数,如果存在正确数量的输出文件,则提交下一个脚本。 However to do this I guess I would have to have a script opened that checks for the presence of the files every now and then, and I am not sure if this is a good idea since it probably takes a day or more before the completion of the first scripts.
但是,要做到这一点,我想我将不得不不时打开一个脚本来检查文件的存在,而且我不确定这是否是一个好主意,因为它可能需要一天或更长的时间才能完成。第一个脚本。
Can you please help me find a solution? 您能帮我找到解决方案吗?
Thank you very much for your help -fra 非常感谢您的帮助-fra
I think you are looking at this the wrong way: 我认为您看错了这种方式:
Not an R problem at all, R happens to be the client of your batch job. 根本不是R问题,R恰好是您的批处理作业的客户。
This is an issue that queue / batch processors can address on your cluster. 这是队列/批处理处理器可以解决的问题。
Worst case you could just wait/sleep in a shell (or R script) til a 'final condition reached' file has been touched 最坏的情况是您只能在shell(或R脚本)中等待/睡眠,直到触摸到“最终条件”文件为止
Inter-dependencies can be expressed with make too 相互依赖关系也可以用make表示
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.