[英]Python fastest way to read a large number of small files into memory?
I'm trying to read a few thousands html files stored on disk. 我正在尝试读取存储在磁盘上的几千个html文件。
Is there any way to do better than; 有什么方法可以做得更好;
for files in os.listdir('.'):
if files.endswith('.html') :
with (open) files as f:
a=f.read()
#do more stuffs
For a similar problem I have used this simple piece of code: 对于类似的问题,我使用了这段简单的代码:
import glob
for file in glob.iglob("*.html"):
with open(file) as f:
a = f.read()
iglob doesn't stores all file simultaneously, this is perfect with a huge directory. iglob不会同时存储所有文件,这对于一个庞大的目录来说是完美的。
Remenber to close files after you have finished, the construct "with-open" make sure for you. 完成后关闭文件的修复程序, “with-open”构造确保为您服务。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.