简体   繁体   English

多个qsub提交可以读取同一组文件吗?

[英]Can multiple qsub submissions read the same group of files?

I would like to use a bash script and qsub to run 30-40 python programs at once. 我想使用bash脚本和qsub一次运行30-40个python程序。 Each python program reads and searches through the same set of files (~400 total) for a set of sequences. 每个python程序读取并搜索同一组文件(共约400个)以查找一组序列。 Is there a problem where multiple python programs could be trying to read from the same file? 是否存在多个python程序可能试图从同一文件读取的问题? If so, what are the consequences? 如果是这样,后果是什么?

There are no problems with multiple jobs reading from the same files that are imposed by Torque. 从Torque施加的相同文件读取多个作业没有问题。 (I think that statement is likely to be true for any resource manager / scheduler.) (我认为该声明可能对任何资源管理器/调度程序都是正确的。)

The main issue I can imagine would be your file system's performance and whether or not it can keep up with potentially concurrent accesses to those files. 我可以想象的主要问题是文件系统的性能以及它是否可以跟上对这些文件的潜在并发访问。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM