Iterating over lines in a file python

Question

I am learning python. Would it be possible if someone could explain the different between the following for processing a file:

file = open("file.txt")
for line in file:
    #do something

file = open("file.txt")
contents = file.read()
for line in contents:
    # do something

I know that in the first case, the file will act as a list so we iterate over a file as we iterate over the elements of a list but in the second case, I am not sure how to explain what happens if I read the file first and then iterate over it?

Answer 1

In the first one you are iterating over the file, line by line. In this scenario, the entire file data is not read into the memory at once; instead, only the current line is read into memory. This is useful for handling very large files, and good for robustness if you don't know if the file is going to be large or not.

In the second one, file.read() returns the complete file data as a string. When you are iterating over it, you are actually iterating over the file's data character by character. This reads the complete file data into memory.

Here's an example to show this behavior.

a.txt file contains

Hello
Bye

Code:

>>> f = open('a.txt','r')
>>> for l in f:
...     print(l)
...
Hello

Bye


>>> f = open('a.txt','r')
>>> r = f.read()
>>> print(repr(r))
'Hello\nBye'
>>> for c in r:
...     print(c)
...
H
e
l
l
o


B
y
e

Answer 2

The second case reads in the contents of the file into one big string. If you iterate over a string, you get each character in turn. If you want to get each line in turn, you can do this:

for line in contents.split('\n'):
     # do something

Or you can read in the contents as a list of lines using readlines() instead of read() .

with open('file.txt','r') as fin:
    lines = fin.readlines()
for line in lines:
    # do something