How to split one cell of csv into columns of dataframe based on separator

Question

I have a csv file that all the data presented in on column, and I'd like to split the numerical data in that column into few columns . The data I have (after reading to data frame) looks like that:

      0
0     13:25:09 -> mm [ -5,  4,  15 ] dd [ 4, 77, 8 ]
1     13:25:09 -> mm [ -4,  9,  10 ] dd [ 8, 6, 10 ]
2     13:25:09 -> mm [ 0,  -4,  19 ] dd [ 3, 1, 66 ]

How can I do it?

Answer 1

I believe you need Series.str.extractall with Series.unstack :

df = df[0].str.extractall('(\d+)')[0].unstack()
print (df)
match   0   1   2  3  4   5  6   7   8
0      13  25  09  5  4  15  4  77   8
1      13  25  09  4  9  10  8   6  10
2      13  25  09  0  4  19  3   1  66

Answer 2

Having this csv file

csvfile = '''13:25:09 -> mm [ -5,  4,  15 ] dd [ 4, 77, 8 ]
13:25:09 -> mm [ -4,  9,  10 ] dd [ 8, 6, 10 ]
13:25:09 -> mm [ 0,  -4,  19 ] dd [ 3, 1, 66 ]'''

wrong result

by doing

import pandas as pd

lines = csvfile.split('\n')
df = pd.DataFrame(lines)

you get a wrong result:

                                                0
0  13:25:09 -> mm [ -5,  4,  15 ] dd [ 4, 77, 8 ]
1  13:25:09 -> mm [ -4,  9,  10 ] dd [ 8, 6, 10 ]
2  13:25:09 -> mm [ 0,  -4,  19 ] dd [ 3, 1, 66 ]

nicer result

You should do:

import pandas as pd

lines = csvfile.split('\n')

df = pd.DataFrame({'id': [1,2,3], 
                   'time': [line[:8] for line in lines], 
                   'mm': [line[15:30] for line in lines],
                   'dd': [line[34:50] for line in lines]})

and you get

   id      time               mm            dd
0   1  13:25:09  [ -5,  4,  15 ]  [ 4, 77, 8 ]
1   2  13:25:09  [ -4,  9,  10 ]  [ 8, 6, 10 ]
2   3  13:25:09  [ 0,  -4,  19 ]  [ 3, 1, 66 ]

what if I don't want strings but integers

Note that mm is going to be a string

print(type(df['mm'][0]))
<class 'str'>

It would be nice to have a list of integers

df['mm_list'] = df['mm'].str.replace('[', '').str.replace(']', '').str.split(',').values.tolist()
df['mm_list_int'] = [[int(i) for i in x] for x in df['mm_list']]
df

leads to a new column mm_list_int

   id      time               mm            dd            mm_list  mm_list_int
0   1  13:25:09  [ -5,  4,  15 ]  [ 4, 77, 8 ]  [ -5,   4,   15 ]  [-5, 4, 15]
1   2  13:25:09  [ -4,  9,  10 ]  [ 8, 6, 10 ]  [ -4,   9,   10 ]  [-4, 9, 10]
2   3  13:25:09  [ 0,  -4,  19 ]  [ 3, 1, 66 ]  [ 0,   -4,   19 ]  [0, -4, 19]

with correct type

print(type(df['mm_list_int'][0]))
<class 'list'>

print(type(df['mm_list_int'][0][0]))
<class 'int'>

that is a list of integers

what if I want the three mm values to be in different colums?

use

objs = [df, pd.DataFrame(df['mm_list_int'].tolist(), columns=['mm_x', 'mm_y', 'mm_z'])]
df_final = pd.concat(objs, axis=1)
df_final = df_final[['id', 'time', 'mm', 'dd', 'mm_x', 'mm_y', 'mm_z']]

obtaining

   id      time               mm            dd  mm_x  mm_y  mm_z
0   1  13:25:09  [ -5,  4,  15 ]  [ 4, 77, 8 ]    -5     4    15
1   2  13:25:09  [ -4,  9,  10 ]  [ 8, 6, 10 ]    -4     9    10
2   3  13:25:09  [ 0,  -4,  19 ]  [ 3, 1, 66 ]     0    -4    19

final touch

do the same with dd and you're done

df['dd_list'] = df['dd'].str.replace('[', '').str.replace(']', '').str.split(',').values.tolist()
df['dd_list_int'] = [[int(i) for i in x] for x in df['dd_list']]

objs = [df, 
        pd.DataFrame(df['mm_list_int'].tolist(), columns=['mm_x', 'mm_y', 'mm_z']),
        pd.DataFrame(df['dd_list_int'].tolist(), columns=['dd_x', 'dd_y', 'dd_z'])]
df_final = pd.concat(objs, axis=1)
df_final = df_final[['id', 'time', 'mm_x', 'mm_y', 'mm_z', 'dd_x', 'dd_y', 'dd_z']]

final result

   id      time  mm_x  mm_y  mm_z  dd_x  dd_y  dd_z
0   1  13:25:09    -5     4    15     4    77     8
1   2  13:25:09    -4     9    10     8     6    10
2   3  13:25:09     0    -4    19     3     1    66