Combining readline() and for loop without using next() in python

Question

I am trying to read multiple text files in a folder to plot a specific column(last column) of each of the file. I have used the following code to do that:

file_library = os.listdir(path)
for book in file_library:
 file = os.path.join(path, book)
  if file.endswith(".txt"):
    f = open(file, "r")
        for line in f:
            reads = read_ydata(f)
                print reads
        f.close()

where read_ydata function is defined as follows:

y_values = []
line = filename.readline()
while line != '':
 y_val = line[line.rfind(' ') + 1:]
 y_values.append(float(y_val))
 line = filename.readline()
return y_values

Now when i run this i get an error: ValueError: Mixing iteration and read methods would lose data and if i replace it with next() i get the error: StopIteration.

Plz advice as to how to get rid of these errors or to implement my logic..

Answer 1

Use either next() and looping, or use .readline() , but don't mix both.

Using the file object as an iterable creates an internal buffer that would create confusion for what position the next .readline() will read from (which does not use a buffer).

Just use next(filename) instead of filename.readline() and you are good.

Both next() and .readline() return the line with the newline included, make sure you strip the line of whitespace before you test for the empty string.

Note that you can use .rsplit() to split off a value from the end of a line:

y_val = line.rsplit(None, 1)[-1]

or use .rpartition() :

y_val = line.rpartition(' ')[-1]

Your function does not need to use a while loop; you could also just use a for loop and break when the line is empty:

y_values = []
for line in filename:
    if not line.strip():
        return y_values
    y_val = line.rsplit(None, 1)[-1]
    y_values.append(float(y_val))