Python: Pandas Csv Import, object to numeric

Question

I'm trying to import a csv-file with the following content in each line.

0;0;0;1;239.57558842082713;0.3690280072634046;[239.6369763080322, 239.5252233951102, 240.21580279356058, 239.86250730788123]

df = pd.read_csv('dataset', sep=';');

In the end df.dtypes shows that it is an object, but I want to get the values that I can make for example a line plot.

I tried to convert the object to a string, to remove the '[', ']' and than cast it to numeric, but I was not successful.

Any hints?

Thanks

Answer 1

You can read the "list" series as a string, use ast.literal_eval , then construct a dataframe and join onto your original dataframe. The resultant series will all have numeric dtypes.

Here's an example:

from io import StringIO
from ast import literal_eval

x = StringIO("""0;0;0;1;239.57558842082713;0.3690280072634046;[239.6369763080322, 239.5252233951102, 240.21580279356058, 239.86250730788123]
0;0;0;1;239.57558842082713;0.3690280072634046;[239.6369763080322, 239.5252233951102, 240.21580279356058, 239.86250730788123]""")

df = pd.read_csv(x, header=None, sep=';')

list_cols = pd.DataFrame(df.pop(6).apply(literal_eval).values.tolist()).add_suffix('_L')

df = df.join(list_cols)

print(df)

   0  1  2  3           4         5         0_L         1_L         2_L  \
0  0  0  0  1  239.575588  0.369028  239.636976  239.525223  240.215803   
1  0  0  0  1  239.575588  0.369028  239.636976  239.525223  240.215803   

          3_L  
0  239.862507  
1  239.862507