[英]Writing multiple files concurrently
I'm trying to write 100 files in a parallel way. 我正在尝试以并行方式写入100个文件。
There is a list data
and items in it should be written to separate files. 有一个列表data
,其中的项目应写入单独的文件中。 I've tried concurrent.futures
but it didn't work(nothing in files). 我已经尝试过concurrent.futures
但它没有用(文件中没有任何内容)。
data = ['a','b','c']
def write2file(i):
outhandles[i].write(data[i])
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
executor.map(write2file,range(s,e))
Expected results: 预期成绩:
file1.txt:'a',
file2.txt:'b',
...
Put your data into a Queue object, then use threads to pop items off of the queue in parallel. 将数据放入Queue对象,然后使用线程并行地将项目弹出队列。 This should help and thread-safe depending on your logic and how you are combining the files. 这应该有所帮助并且是线程安全的,具体取决于您的逻辑以及如何组合文件。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.