Memory Error - Python (Huge File Changes)

Question

I am trying to change the End of Line(EOL) of a file from Windows to Linux, the file size is 50 GB and writing into the same file, Below is my code :

filename = "D:\AddressEvaluation\AddressStandardization\infu\InfutorFile.txt"
fileContents = open(filename,"r").read()
f = open(filename,"w", newline="\n")
f.write(fileContents)
f.close()

It gives me this error :

MemoryError                               Traceback (most recent call last)
<ipython-input-3-a87efb13f002> in <module>()
      1 filename = "D:\AddressEvaluation\AddressStandardization\infu\InfutorFile.txt"
----> 2 fileContents = open(filename,"r").read()
      3 f = open(filename,"w", newline="\n")
      4 f.write(fileContents)
      5 f.close()

MemoryError: 

Am I missing something ? Please help ?

Answer 1

This might be because of the size of file. (it might be too large)

Reading a large file into memory all at once throws this error generally.

You can process this by reading the file line-by-line with below code:

f1 = open(filename,"w", newline="\n")

with open('D:\AddressEvaluation\AddressStandardization\infu\InfutorFile.txt') as f:
    for fileContents in f:
        f1.write(fileContents)
f1.close()

This will solve the problem of MemoryError . You can also look at Memory error due to the huge input file size