如何读取 csv 文件，其中 pandas 在一列中有多个值？

Question

I have a csv file that looks like this:

timestamp (int), array(value1,value2,...), identifier (string)

The arrays with the values inside are exactly written like this:

List(value1, value2, value3)

where the values can be written in the formats

1.23E4, -123456.78910

So what I want to have eventually is a dataframe with a timestamp and an identifier, but multiple values at each point.

I have no idea how to read this csv file with pandas in python. If I just try

pd.read_csv("myFilePath")

it gives me

pandas.errors.ParserError: Error tokenizing data. C error: Expected 7 fields in line 3, saw 76

Obviously I didnt tell pandas how to read that file properly and honestly I dont quite know hot to. Can somebody help me? Thanks a lot

Answer 1

You could just read in all the data where each row was represented in a single column then extract what you need from there.

I think your data may look like this

sim_csv = io.StringIO(
'''2022-07-20,List(1.23E4, -123456.78910),ID001
2022-07-21,List(2.23E4, -223456.78910),ID002
2022-07-22,List(3.23E4, -323456.78910),ID003
2022-07-23,List(4.23E4, -423456.78910, 55),ID004''')

Read it all in - single column

df = pd.read_fwf(sim_csv, widths=[999999], header=None)
print(df)

                                                  0
0      2022-07-20,List(1.23E4, -123456.78910),ID001
1      2022-07-21,List(2.23E4, -223456.78910),ID002
2      2022-07-22,List(3.23E4, -323456.78910),ID003
3  2022-07-23,List(4.23E4, -423456.78910, 55),ID004

Extract what you need

dfs = df[0].str.extract(r'(?P<timestamp>.+),.*List\((?P<raw_values>.+)\),(?P<id>.+)')
print(dfs)

    timestamp                 raw_values     id
0  2022-07-20      1.23E4, -123456.78910  ID001
1  2022-07-21      2.23E4, -223456.78910  ID002
2  2022-07-22      3.23E4, -323456.78910  ID003
3  2022-07-23  4.23E4, -423456.78910, 55  ID004

That leave your in a list that is really still a string and not a Python list object. Not sure what you want to do with the values from there. Maybe make them into a real list.

Values to a real list

dfs['list_values'] = dfs['raw_values'].str.split(',')
print(dfs)

    timestamp                 raw_values     id                    list_values
0  2022-07-20      1.23E4, -123456.78910  ID001       [1.23E4,  -123456.78910]
1  2022-07-21      2.23E4, -223456.78910  ID002       [2.23E4,  -223456.78910]
2  2022-07-22      3.23E4, -323456.78910  ID003       [3.23E4,  -323456.78910]
3  2022-07-23  4.23E4, -423456.78910, 55  ID004  [4.23E4,  -423456.78910,  55]

At this point the values in the list are actually still strings. There are a lot of things you could do from here depending on what you are trying to accomplish.