Creating columns from dictionary rows in dataframe is assigning random values

Question

I have a dataframe, something like:

|   | a | b                |
|---|---|------------------|
| 0 | a | {'d': 1, 'e': 2} |
| 1 | b | {'d': 3, 'e': 4} |
| 2 | c | NaN              |
| 3 | d | {'f': 5}         |
| 4 | d | {'e':8,'f': 5}   |
| 5 | d | {'e':9,'f': 5}   |
| 6 | d | {'f': 7}         |

I am using the following code from df.join(pd.DataFrame.from_records(df['b'].mask(df.b.isna(), {}).tolist())) How can I create column from dictionary keys in same dataframe? and getting result like:

|   | a | b                | d | e | f |
|---|---|------------------|---|---|---|
| 0 | a | {'d': 1, 'e': 2} | 1 | 2 |nan|
| 1 | b | {'d': 3, 'e': 4} | 3 | 8 |nan|
| 2 | c | NaN              |nan|nan|nan|
| 3 | d | {'f': 5}         |nan|nan| 5 |
| 4 | d | {'e':8,'f': 5}   |nan| 4 | 5 |
| 5 | d | {'e':9,'f': 5}   |nan|nan| 5 |
| 6 | d | {'f': 7}         |nan|nan| 7 |

Why are the values in e randomly getting allocated and not by there adjascent rows? How can I solve this issue?

Thanks in advance!

Answer 1

Reason should be original DataFrame has no default RangeIndex , so after join is wrongly assigned new DataFrame , which has by default default index.

You need set index values by df.index for correct align new DataFrame.

df.join(pd.DataFrame(df['b'].mask(df.b.isna(), {}).tolist(), index=df.index))