Pandas - groupby，其中每行都有多个存储在列表中的值

Question

I'm working with last.fm listening data and have a DataFrame that looks like this:

           Artist Plays                                   Genres
0   John Coltrane    10             [jazz, modal jazz, hard bop]
1     Miles Davis    15  [jazz, cool jazz, modal jazz, hard bop]
2  Charlie Parker    20                            [jazz, bebop]

I want to group the data by the genres and then aggregate by the sum of plays for each genre, to get something like this:

        Genre Plays
0        jazz    45
1  modal jazz    25
2    hard bop    25
3       bebop    20
4   cool jazz    15

Been trying to figure this out for a while now but can't seem to find the solution. Do I need to change the way that the genre data is stored?

I was able to find this post which addresses a similar question, but that user was only looking to get the count of each list value. This gets me about halfway there, but I couldn't figure out how to use that to aggregate another column in the dataframe.

Answer 1

In general, you should not store lists in a DataFrame , so yes, probably best to change how they are stored. With this you can use some join + str.get_dummies + .multiply . Choose a sep that doesn't appear in any of your strings.

sep = '*'
df.Genres.apply(sep.join).str.get_dummies(sep=sep).multiply(df.Plays, axis=0).sum()

Output

bebop         20
cool jazz     15
hard bop      25
jazz          45
modal jazz    25
dtype: int64

---

An easier form to work with would be if your lists were split across lines as in:

import pandas as pd
df1 = pd.concat([pd.DataFrame(df.Genres.values.tolist()).stack().reset_index(1, drop=True).to_frame('Genres'),
                 df[['Plays', 'Artist']]], axis=1)

       Genres  Plays          Artist
0        jazz     10   John Coltrane
0  modal jazz     10   John Coltrane
0    hard bop     10   John Coltrane
1        jazz     15     Miles Davis
1   cool jazz     15     Miles Davis
1  modal jazz     15     Miles Davis
1    hard bop     15     Miles Davis
2        jazz     20  Charlie Parker
2       bebop     20  Charlie Parker

Making it a simple sum within genres:

df1.groupby('Genres').Plays.sum()

Genres
bebop         20
cool jazz     15
hard bop      25
jazz          45
modal jazz    25
Name: Plays, dtype: int64