Reading a two-column file into Python

Question

Suppose I have a data file called "test.dat" which is of the form "1 2 \\n 3 4 \\n 5 6".

If I run the following code, I get two arrays with the numbers in them:

import csv
from numpy           import *
f2 = open('test.dat', 'r') 
lines = f2.readlines()
x = []
y = [] 
for line in lines:
    p = line.split()
    x.append(float(p[0]))
    y.append(float(p[1]))
f2.close() 

print x
print y

However, I tried writing a function of the form

def Read_Two_Column_File(file_name):
    data  = open(file_name, 'r')
    lines = data.readlines()
    x = []
    y = []
    for line in lines:
        p = line.split()
        x.append(float(p[0]))
        y.append(float(p[1]))
    data.close()
    return x, y

x, y = Read_Two_Column_File('test.dat')

print x 
print y

but this only returns the two last numbers. What is going wrong here?

Answer 1

From your comment it would suggest that you did have mixed tab and space indenting causing your problem. I could suggest a few minor changes to your code as I could see you were thinking of using the csv module:

Version 1 - use with to ensure the file is closed automatically

def Read_Two_Column_File(file_name):
    with open(file_name, 'r') as data:
        x = []
        y = []
        for line in data:
            p = line.split()
            x.append(float(p[0]))
            y.append(float(p[1]))

    return x, y

x, y = Read_Two_Column_File('test.dat')

print x 
print y

Version 2 - a possible solution using the csv module

import csv

def Read_Two_Column_File(file_name):
    with open(file_name, 'r') as f_input:
        csv_input = csv.reader(f_input, delimiter=' ', skipinitialspace=True)
        x = []
        y = []
        for cols in csv_input:
            x.append(float(cols[0]))
            y.append(float(cols[1]))

    return x, y

x, y = Read_Two_Column_File('test.dat')

print x 
print y

Both versions will display:

[1.0, 3.0, 5.0]
[2.0, 4.0, 6.0]

Answer 2

if your file looks like that

$ cat test.dat

1 2 \\n 3 4 \\n 5 6

Then your \\n is not a true \\n, so the readlines() function return the whole line '1 2 \\n 3 4 \\n 5 6'

Your file must look like that:

$ cat test.dat

1 2

3 4

5 6

And then your code is correct and it does work.