convert a dataframe column from string to List of numbers

Question

I have created the following dataframe from a csv file:

id      marks
5155    1,2,3,,,,,,,,
2156    8,12,34,10,4,3,2,5,0,9
3557    9,,,,,,,,,,
7886    0,7,56,4,34,3,22,4,,,
3689    2,8,,,,,,,,

It is indexed on id . The values for the marks column are string. I need to convert them to a list of numbers so that I can iterate over them and use them as index number for another dataframe. How can I convert them from string to a list? I tried to add a new column and convert them based on " Add a columns in DataFrame based on other column " but it failed:

df = df.assign(new_col_arr=lambda x: np.fromstring(x['marks'].values[0], sep=',').astype(int))

Answer 1

Here's a way to do:

df = df.assign(new_col_arr=df['marks'].str.split(','))

# convert to int
df['new_col'] = df['new_col_arr'].apply(lambda x: list(map(int, [i for i in x if i != ''])))

Answer 2

I presume that you want to create NEW dataframe, since the number of items is differnet from number of rows. I suggest the following:

#source data
df = pd.DataFrame({'id':[5155, 2156, 7886], 
                   'marks':['1,2,3,,,,,,,,','8,12,34,10,4,3,2,5,0,9', '0,7,56,4,34,3,22,4,,,']

# create dictionary from df:
dd = {row[0]:np.fromstring(row[1], dtype=int, sep=',') for _, row in df.iterrows()}

{5155: array([1, 2, 3]),
 2156: array([ 8, 12, 34, 10,  4,  3,  2,  5,  0,  9]),
 7886: array([ 0,  7, 56,  4, 34,  3, 22,  4])}

# here you pad the lists inside dictionary so that they have equal length
...

# convert dd to DataFrame:
df2 = pd.DataFrame(dd)

Answer 3

I found two similar alternatives:

1.

df['marks'] = df['marks'].str.split(',').map(lambda num_str_list: [int(num_str) for num_str in num_str_list if num_str])

2.

df['marks'] = df['marks'].map(lambda arr_str: [int(num_str) for num_str in arr_str.split(',') if num_str])