读取大文本文件时出现内存错误

Question

I have a Text File Of Around 36gb which contains words per line, i am trying to read the file, but it says Memory Error, at which I am not shocked, but how do I work around to read it?

I am trying this:

for words in open("hugefile.txt").readlines():
    #do something

I have 2gb RAM, OS: Windows XP, Python 2.7

Thanks.

Answer 1

You are calling readlines() which loads the whole file into memory.

Iterate over the file instead:

for words in open("hugefile.txt"):

This'll iterate over lines one by one, reading more lines as needed.