简体   繁体   English

Python以最快的方式将大量小文件读入内存?

[英]Python fastest way to read a large number of small files into memory?

I'm trying to read a few thousands html files stored on disk. 我正在尝试读取存储在磁盘上的几千个html文件。

Is there any way to do better than; 有什么方法可以做得更好;

for files in os.listdir('.'):
    if files.endswith('.html') :
        with (open) files as f:
            a=f.read()
            #do more stuffs

For a similar problem I have used this simple piece of code: 对于类似的问题,我使用了这段简单的代码:

import glob
for file in glob.iglob("*.html"):
    with open(file) as f:
        a = f.read()

iglob doesn't stores all file simultaneously, this is perfect with a huge directory. iglob不会同时存储所有文件,这对于一个庞大的目录来说是完美的。
Remenber to close files after you have finished, the construct "with-open" make sure for you. 完成后关闭文件的修复程序, “with-open”构造确保为您服务。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM