读取和删除 python 中 N 行的最快方法

Question

Fastest way to read and delete N lines in python.

First I read the file something like this: (I think this is the best way to read large files: Source )

N = 50
with open("ahref.txt", "r+") as f:
    link_list = [(next(f)).removesuffix("\n") for x in range(N)]

after that I run my code:

# My code here

After that I want to delete the first N line (I read it: Source ).

# Source: https://stackoverflow.com/questions/4710067/how-to-delete-a-specific-line-in-a-file/28057753#28057753
with open("target.txt", "r+") as f:
    d = f.readlines()
    f.seek(0)
    for i in d:
        if i != "line you want to remove...":
            f.write(i)
    f.truncate()

This code doesn't work for me. Because I read only N numbers of lines.

I can remove lines:

with open("xml\\ahref.txt", "r+") as f:
    N = 5
    all_lines = f.readlines()
    f.seek(0)
    f.truncate()
    f.writelines(all_lines[N:])

But there is a problem with that:

1. I have to read all the lines and after that I have to write all the lines. which is not a fast way (There are many ways, but it needs to read all line)

What is the fastest way in terms of performance? because the file is huge.

Answer 1

fastest way is not to read the entire file in memory and use a temporary output file that you can then move over the original file if required

try:

N = 50
mode = "r+"
if not os.path.isfile('output'): mode = "w+" 
with open('input', 'r') as fin, open('output', mode) as fout:
    for index, line in enumerate(fout): N += 1
    for index, line in enumerate(fin):
        if index > N: fout.write(line)
        # i haven't tested this you may need index > N or index >= N