copying one file's contents to another in python

Question

I've been taught the best way to read a file in python is to do something like:

with open('file.txt', 'r') as f1:
    for line in f1:
        do_something()

But I have been thinking. If my goal is to copy the contents of one file completely to another, are there any dangers of doing this:

with open('file2.txt', 'w+') as output, open('file.txt', 'r') as input:
    output.write(input.read())

Is it possible for this to behave in some way I don't expect?

Along the same lines, how would I handle the problem if the file is a binary file, rather than a text file. In this case, there would be no newline characters, so readline() or for line in file wouldn't work (right?).

EDIT Yes, I know about shutil . There are many better ways to copy a file if that is exactly what I want to do. I want to know about the potential risks, if any, of this approach specifically, because I may need to do more advanced things than simply copying one file to another (such as copying several files into a single one).

Answer 1

Please note that the shutil module also contains copyfileobj() , basically implemented like Barmar's answer.

Or, to answer your question:

from shutil import copyfileobj

with open('file2.txt', 'wb') as output, open('file.txt', 'rb') as input:
    copyfileobj(input, output)

would be my suggestion. It avoids re-implementing the buffering mechanism and, should the implementation of the standard library improve, your code wins as well.

---

On Unix, there also is a non-standardised syscall called sendfile . It is used mostly for sending data from an open file to a socket (serving HTTP requests, etc.).

Linux allows using it for copying data between regular files as well though. Other platforms don't, check the Python doc and your man pages. By using a syscall the kernel copies the content without the need of copying buffers to and from userland.

The os module offers os.sendfile() since Python 3.3. You could use it like:

import io
import os

with open('file2.txt', 'wb') as output, open('file.txt', 'rb') as input:
    offset = 0 # instructs sendfile to start reading at start of input
    input_size = input.seek(0, io.SEEK_END)
    os.sendfile(output.fileno(), input.fileno(), offset, input_size)

Otherwise, there is a package on PyPi, pysendfile , implementing the syscall. It works exactly as above, just replace os.sendfile with sendfile.sendfile (and import sendfile ).

Answer 2

The only potential problem with your output.write(input.read()) version is if the size of the file is too large to hold all of it in memory. You can use a loop that reads smaller batches.

with open('file2.txt', 'wb+') as output, open('file.txt', 'rb') as input:
    while True:
        data = input.read(100000)
        if data == '':  # end of file reached
            break
        output.write(data)

This will work for both text and binary files. But you need to add the b modifier to the modes for portable operation on binary files.

Answer 3

虽然这可能不能完全回答您的问题，但对于没有对文件内容进行任何其他处理的纯复制，您应该考虑其他方法，例如shutil模块：

shutil.copy('file.txt', 'file2.txt')