str encapsulated arrays to array with pandas

Question

I am working with a dataframe, that contains arrays. Upon read_cvs , pandas seems to be storing my vetors in str mode. Like this:

df['column'].iloc[3]
>>>'[50.6402809, 4.6667145]'

type(df['column'].iloc[3])
>>> str

How can I convert the entire column to array? Like so:

df['column'].iloc[3]
>>>[50.6402809, 4.6667145]

type(df['column'].iloc[3])
>>> array

Answer 1

If want numpy arrays use lambda function with ast.literal_eval and convert to arrays:

import ast

df['column'] = df['column'].apply(lambda x: np.array(ast.literal_eval(x)))

And if need lists:

df['column'] = df['column'].apply(ast.literal_eval)

df['column'] = [ast.literal_eval(x) for x in df['column']]

Answer 2

You could use the ast module to interpret the strings literally. However, this can be dangerous, especially when reading the data from a file or worse, online.

An alternative would be to parse the file directly using series.str functions:

In [19]: parsed = (
    ...:     df['column']
    ...:     .str.strip('[]')
    ...:     .str.split(', ', )
    ...:     .apply(lambda x: np.array(x).astype(float)))
    ...:

In [20]: parsed
Out[20]:
0    [0.45482146988492345, 0.40132331304489344]
1      [0.4820128044982769, 0.6930103661982894]
2      [0.15845986027370507, 0.825879918750825]
3      [0.08389109330674027, 0.031864037778777]
Name: column, dtype: object