Python - Read in Comma Separated File, Create Two lists

Question

New to Python here and I'm trying to learn/figure out the basics. I'm trying to read in a file in Python that has comma separated values, one to a line. Once read in, these values should be separated into two lists, one list containing the value before the "," on each line, and the other containing the value after it.

I've played around with it for quite a while, but I just can't seem to get it.

Here's what I have so far...

with open ("mid.dat") as myfile:
    data = myfile.read().replace('\n',' ')
    print(data)

list1 = [x.strip() for x in data.split(',')]
print(list1)
list2 = ?

List 1 creates a list, but it's not correct. List 2, I'm not even sure how to tackle.

PS - I have searched other similar threads on here, but none of them seem to address this properly. The file in question is not a CSV file, and needs to stay as a .dat file.

Here's a sample of the data in the .dat file:

113.64,889987.226
119.64,440987774.55
330.43,446.21

Thanks.

Answer 1

Use string slicing:

    list1= []
    list2 = []
    with open ("mid.dat") as myfile:
        for line in myfile:
             line = line.split(",").rstrip()
             list1.append( line[0])
             list2.append( line[1])

Python's rstrip() method strips all kinds of trailing whitespace by default, so removes return carriage "\\n" too

Answer 2

If you want to use only builtin packages, you can use csv .

import csv

with open("mid.dat") as myfile:
    csv_records = csv.reader(myfile)
    list1 = []
    list2 = []
    for row in csv_records:
        list1.append(row[0])
        list2.append(row[1])

Answer 3

Could try this, which creates lists of floats not strings however:

from ast import literal_eval

with open("mid.dat") as f:
    list1, list2 = map(list, (zip(*map(literal_eval, f.readlines()))))

Can be simplified if you don't mind list1 and list2 as tuples.

The list(*zip(*my_2d_list)) pattern is a pretty common way of transposing 2D lists using only built-in functions. It's useful in this scenario because it's easy to obtain a list (call this result ) of tuples on each line in the file (where result[0] would be the first tuple, and result[n] would be the nth), and then transpose result (call this resultT ) such that resultT[0] would be all the 'left values' and resultT[1] would be the 'right values'.

Answer 4

You could do this with pandas.

import pandas as pd
df = pd.read_csv('data.csv', columns=['List 1','List 2'])

If your data is a text file the respective function also exists in the pandas package. Pandas is a very powerful tool for data such as yours.

After doing so you can split your data into two independent dataframes.

list1 = df['List 1']
list2 = df['List 2']

I would stick to a dataframe because data manipulation and analysis is much easier within the pandas framework.

Answer 5

let's keep it very simple.

list1 = []
list2 = []

with open ("mid.dat") as myfile:
    for line in myfile:
        x1,x2 = map(float,line.split(','))
        list1.append(x1)
        list2.append(x2)

print(list1)
print(list2)

Answer 6

Here is my suggestion to be short and readable, without any additional packages to install:

with open ("mid.dat") as myfile:
    listOfLines = [line.rstrip().split(',') for line in myfile]
    list1 = [line[0] for line in listOfLines]
    list2 = [line[1] for line in listOfLines]ility

Note: I used rstrip() to remove the end of line character.

Answer 7

Following is a solution obtained by correcting your own attempt:

with open("test.csv", "r") as myfile:
    datastr = myfile.read().replace("\n",",")
    datalist = datastr.split(",")
    list1 = []; list2=[]
    for i in range(len(datalist)-1):   # ignore empty last item of list
        if i%2 ==0:
            list1.append(datalist[i])
        else: 
            list2.append(datalist[i])
print(list1)
print(list2)

Output:

['113.64', '119.64', '330.43']
['889987.226', '440987774.55', '446.21']