Copy a file line by line in python

Question

I am writing a python program to copy a file line by line into a new file. The code I have is below in which I am using a loop to copy the file line by line.

However since the number of lines in the file may change, is there a way to copy a file line by line in python without using a loop which relies on numbers, and instead relies on the something like the EOF character to terminate the loop?

import os
import sys

i = 0
f = open("C:\\Users\\jgr208\\Desktop\\research_12\\sap\\beam_springs.$2k","r")
copy = open("C:\\Users\\jgr208\\Desktop\\research_12\\sap\\copy.$2k","wt")
#loop that copies file line by line and terminates loop when i reaches 10
while i < 10:
     line = f.readline()
     copy.write(str(line))
     i = i +1
f.close()
copy.close()

Answer 1

You can iterate over lines in a file object in Python by iterating over the file object itself:

for line in f:
    copy.write(line)

From the docs on file objects :

An alternative approach to reading lines is to loop over the file object. This is memory efficient, fast, and leads to simpler code:

>>> for line in f:
        print line,

Answer 2

Files can be iterated directly, without the need for an explicit call to readline :

f = open("...", "r")
copy = open("...", "w")
for line in f:
    copy.write(line)
f.close()
copy.close()

Answer 3

See shutil module for better ways of doing this than copying line-by-line:

shutil.copyfile(src, dst)

Copy the contents (no metadata) of the file named src to a file named dst. dst must be the complete target file name; look at shutil.copy() for a copy that accepts a target directory path. If src and dst are the same files, Error is raised. The destination location must be writable; otherwise, an IOError exception will be raised. If dst already exists, it will be replaced. Special files such as character or block devices and pipes cannot be copied with this function. src and dst are path names given as strings.

Edit: Your question says you are copying line-by-line because the source file is volatile. Something smells wrong about your design. Could you share more details regarding the problem you are solving?

Answer 4

Writing line by line can be slow when working with large data. You can accelerate the read/write operations by reading/writing a bunch of lines all at once. Please refer to my answer to a similar question here

Answer 5

Using with statements:

with open("input.txt", "r", encoding="utf-8") as input_file:
  with open("output.txt", "w", encoding="utf-8") as output_file:
    for input_line in input_file:
      output_line = f(input_line) # You can change the line here
      output_file.write(output_line)

Note that input_line contains the end-of-line character(s) ( \n or \r\n ), if there are any.