Reading txt file into python

Question

I python rookie here. I have multiple text files each with the following format of floats 1x10, 1x10 and 10x10

0.0551 1500.0 [273.639, 273.331, 273.021, 272.711, 272.399, 272.087, 271.773, 271.46, 271.145]

0.0553 1532.5 [272.422, 273.96, 273.021, 273.321, 272.494, 273.129, 271.12, 271.23, 271.889]

0.0555 1560.0 [273.234, 273.44, 273.133, 272.065, 272.234, 272.012, 271.942, 271.43, 271.145]

0.0558 1582.5 [272.45, 273.011, 273.45, 273.331, 272.321, 273.234, 271.34, 271.531, 271.932]

I would like to read them as a column as following to be able to plot them:

column1 = [0.0551,0.0553,0.0555,0.0558,....]

column2 = [1500.0,1532.5,1560.0,1582.5,....]

column3 = [[273.639, 273.331, 273.021, 272.711, 272.399, 272.087, 271.773, 271.46, 271.145],[272.422, 273.96, 273.021, 273.321, 272.494, 273.129, 271.12, 271.23, 271.889],[273.234, 273.44, 273.133, 272.065, 272.234, 272.012, 271.942, 271.43, 271.145],[272.45, 273.011, 273.45, 273.331, 272.321, 273.234, 271.34, 271.531, 271.932]]

I tried numpy loadtxt and numerous other functions but was never able to successfully read them in python. What is the best way to read the text file in the desired format?

Answer 1

Your file structure is kinda weird, you should clean it upstream. Anyway here's the function to load your data. If the file structure changes too much, the function may not work.

def load_data(file):
    cols = [[] for _ in range(3)]
    to_remove = ['[', ']', '\n']
    with open(file, 'r') as f:
        for line in f.readlines():
            if len(line) > 1:
                split_line = line
                for x in to_remove: split_line = split_line.replace(x, '') 
                split_line = split_line.split(' ', 2)
                cols[0].append(float(split_line[0]))
                cols[1].append(float(split_line[1]))
                cols[2].append([float(i) for i in split_line[2].split(',')])
    return cols