[英]Time efficient way to skip no of line from very large text file (16gb) using python
I have a very large text file of 16gb.我有一个非常大的 16gb 文本文件。 I need to skip no of line.I want to skip those line in time efficient manner.
我需要跳过任何一行。我想以省时的方式跳过这些行。 I am using python for code.how to do that?
我正在使用 python 作为代码。怎么做?
Just read the number of lines you want to skip and throw them away:只需阅读您要跳过的行数并将其丢弃:
with open(your_file) as f_in:
for i in range(number_of_lines_to_skip):
f_in.readline()
# your file is now at the line you want...
You can also use enumerate
to have a generator that only yields lines once you have skipped the lines you want to:您还可以使用
enumerate
来创建一个生成器,该生成器仅在您跳过您想要的行后才产生行:
with open(your_file) as f_in:
for line in (line for i, line in enumerate(f_in) if i>lines_to_skip):
# here only when you have skipped the first lines
The second there is likely faster.第二个可能更快。
beware, calling next
on a file object will raise StopIteration
if the end of file is reached.请注意,如果到达文件末尾,对文件 object 调用
next
将引发StopIteration
。
go_to_line_number = some_line_number
with open(very_large_file) as fp:
for _ in range(go_to_line_number):
next(fp)
for line in fp:
# start your work from desired line number
pass
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.