How to extract values from column of dictionaries in pandas?

Question

I have a dataframe containing dictionaries in each column {'value': 2343}. I'm trying to get rid of the dictionary and set the cell equal to the value for all dictionaries in the dataframe.

import pandas as pd
df = pd.DataFrame([
    {'node': 'A', 'read': {'value': 2343}, 'write': {'value': 23}},
    {'node': 'B', 'read': {'value': 334}, 'write': {'value': 233444}},
])

I can set the cells to value for one column:

df['read'] = df['read'].apply(lambda x: x['value'])

Is there a way to do this on all columns in the dataframe? Should I just iterate over all columns?

The expected output should be a dataframe:

df = pd.DataFrame([
    {'node': 'A', 'read': 2343, 'write': 23},
    {'node': 'B', 'read': 334, 'write': 233444},
])

Answer 1

You can use applymap :

import pandas as pd
df = pd.DataFrame([
    {'node': 'A', 'read': {'value': 2343}, 'write': {'value': 23}},
    {'node': 'B', 'read': {'value': 334}, 'write': {'value': 233444}},
])

cols = ['read', 'write']
df[cols] = df[cols].applymap(lambda x: x['value'])

print(df)

[Out]:

  node  read   write
0    A  2343      23
1    B   334  233444

Answer 2

how about this:

df.apply(lambda x: x.apply(lambda y: y['value'] if isinstance(y, dict) else y))

using this you don't need to worry about selecting which columns are dictionaries.

Answer 3

If they are dictionaries you can use .str accessor like below

df['read'] = df['read'].str['value']
df['write'] = df['write'].str['value']