如何使字典键是 Pandas 数据框的一列到列？

Question

I have a dataframe with one column containing stringified list containing dictionaries. I was wondering how can I make new columns from these dictionary keys.

I am looking solution using pandas methods like apply stack etc and NOT USING FOR LOOP as far as possible.

Here is the problem:

speakers = ['Einstein','Newton']
views = [1000,2000]
ratings0 = ("[{'id': 7, 'name': 'Funny', 'count': 100}, {'id': 1, 'name': 'Sad', "
 "'count': 110}, {'id': 9, 'name': 'Happy', 'count': 120}]")

ratings1 = ("[{'id': 7, 'name': 'Happy', 'count': 200}, {'id': 3, 'name': 'Funny', "
 "'count': 210}, {'id': 2, 'name': 'Sad', 'count': 220}]")


ratings = [ratings0, ratings1]
df = pd.DataFrame({'speaker': speakers, 'ratings': ratings,'views':views})

print(df)
speaker                                            ratings  views
0  Einstein  [{'id': 7, 'name': 'Funny', 'count': 100}, {'i...   1000
1    Newton  [{'id': 7, 'name': 'Happy', 'count': 200}, {'i...   2000

My attempt so far,

# new dataframe only for ratings
dfr = df['ratings'].apply(ast.literal_eval)
dfr = dfr.apply(pd.DataFrame)
dfr = dfr.apply(lambda x: x.sort_values(by='name'))
dfr = dfr.apply(pd.DataFrame.stack)

print(dfr)

 0               1               2          
  count id   name count id   name count id   name
0   100  7  Funny   110  1    Sad   120  9  Happy
1   200  7  Happy   210  3  Funny   220  2    Sad

This gives multi-index dataframe. I tried sorting the dictionary, but still it is not sorted and the column name does not have the same values. Also, I am unsure how to move the values of column name to replace column count and remove other unwanted columns.

Final Wanted Solution

speaker   views Funny Sad Happy
Einstein  1000 100   110 120  
Newton    2000 210   220 200

Update

I am using Pandas 0.20 and the method .explode() is absent in my workplace and I am not permitted to update Pandas.

Answer 1

For pandas >= 0.25.0 you can use ast.literal_eval + explode + pivot

ii = df.set_index('speaker')['ratings'].apply(ast.literal_eval).explode()

u = pd.DataFrame(ii.tolist(), index=ii.index).reset_index()

u.pivot('speaker', 'name', 'count')

name      Funny  Happy  Sad
speaker
Einstein    100    120  110
Newton      210    200  220

---

For older versions of pandas

a = df['speaker']
b = df['ratings']

ii = [
  {**{'speaker': name}, **row}
  for name, element in zip(a, b) for row in ast.literal_eval(element)
]

pd.DataFrame(ii).pivot('speaker', 'name', 'count')

Answer 2

You may use sum , index.repeat to construct a new dataframe and join it df[['speaker', 'views']] and assign it to df1 . Next, set_index , unstack , and reset_index

df['ratings'] = df['ratings'].apply(ast.literal_eval)
df1 = (pd.DataFrame(df.ratings.sum(), index=df.index.repeat(df.ratings.str.len()))
                   .drop('id', 1).join(df[['speaker', 'views']]))
df1.set_index(['speaker', 'views', 'name'])['count'].unstack().reset_index()

Out[213]:
name   speaker  views  Funny  Happy  Sad
0     Einstein  1000   100    120    110
1     Newton    2000   210    200    220

---

Note : name in the final output is the label of the columns axis. If you don't want to see it, just chain additional rename_axis as follows

df1.set_index(['speaker', 'views', 'name'])['count'].unstack().reset_index() \
                                                    .rename_axis([None], axis=1)

Out[214]:
    speaker  views  Funny  Happy  Sad
0  Einstein  1000   100    120    110
1  Newton    2000   210    200    220

Answer 3

For loops are not always bad. You can give it a try:

dfr = pd.DataFrame(columns=['id','name','count'])

for i in range(len(df)):
    x = pd.DataFrame(df['ratings'].apply(ast.literal_eval)[i])
    x.index = [i]*len(x)
    dfr = dfr.append(x)


dfr = dfr.reset_index()   
dfr = (dfr.drop('id',axis=1)
         .pivot_table(index=['index'], columns='name',
                      values='count',aggfunc='sum')
         .rename_axis(None, axis=1).reset_index())

df_final = df.join(dfr)
df_final.drop(['index','ratings'],axis=1,inplace=True)

df_final

Gives:

    speaker  views  Funny  Happy  Sad
0  Einstein   1000    100    120  110
1    Newton   2000    210    200  220