Reading floats from file with python

Question

My input file has the form:

5.0, 1000.0, 100000000000000.0, 115.2712, 230.538, 345.796, 461.0408, 
1.053E-09, 1.839E-09, 1.632E-10, 1.959E-12, 4.109, 3.683, 3.586, 3.650 

where every number is essentially in a line.

What I want to do is read all the floats, and then append only columns 7 through 10 to an array.

Here's what I've written:

T=[]
with open("test.txt", "r") as file1:
    for line in file1.readlines():
        f_list = [float(i) for i in line.split(",")]
        T.append(float(f_list[7]))
        T.append(float(f_list[8]))
        T.append(float(f_list[9]))
        T.append(float(f_list[10]))

When I run the above I get:

ValueError: could not convert string to float:

I think there's something wrong with the float(i) part, but I can't find a way around it.

I've seen people having similar problems here, but none of the fixes I've tried so far have helped. Any help is greatly appreciated.

Answer 1

No the problem is that your first line ends with a comma :

5.0, 1000.0, 100000000000000.0, 115.2712, 230.538, 345.796, 461.0408,
1.053E-09, 1.839E-09, 1.632E-10, 1.959E-12, 4.109, 3.683, 3.586, 3.650 

As a result, you want to process a string that only contains spaces (like ' ' ). And float(' ') fails since it is not a number (it actually reports this):

>>> float(' ')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not convert string to float: 
>>> float('a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not convert string to float: 'a'

But a space simply is invisible when printed.

You can solve it by adding a filter statement to the list comprehension:

T = []
with open("test.txt", "r") as file1:
    for line in file1.readlines():
        f_list = [float(i) for i in line.split(",")]
        T += f_list[7:11]

Furthermore this will not work since none of the lines has 7-11 floats. So you will never add these floats anyway.

You can however use the following code:

with open("test.txt", "r") as file1:
    f_list = [float(i) for line in file1 for i in line.split(',') if i.strip()]
    T = f_list[7:11]

This will result in T being equal to:

>>> T
[1.053e-09, 1.839e-09, 1.632e-10, 1.959e-12]

Answer 2

Your problem is that when you split line , the resulting list most likely contains whitespace. This would cause float() to fail. You need to sanitize your split list first by testing if an element is actually a valid float number. eg:

>>> def is_float(n):
    try:
        float(n)
        return True
    except:
        return False


>>> 
>>> line = '5.0, 1000.0, 100000000000000.0, 115.2712, 230.538, 345.796, 461.0408,'
>>> lst = [float(n) for n in line.split(',') if is_float(n)]
>>> lst
[5.0, 1000.0, 100000000000000.0, 115.2712, 230.538, 345.796, 461.0408]
>>>