元组列表中的元素计数，每一项具有增加的权重

Question

I have a list of tuples :

for i, item in enumerate(tags_and_weights):
    tags = item[0]
    weight = item[1]

which prints:

1 (['alternative country', 'alternative pop', 'alternative rock', 'art rock', 'brill building pop', 'country rock', 'dance rock', 'experimental', 'folk', 'folk rock', 'garage rock', 'gbvfi', 'indie rock', 'jangle pop', 'lo-fi', 'melancholia', 'noise pop', 'post-punk', 'power pop', 'protopunk', 'psychedelic rock', 'pub rock', 'rock', 'roots rock', 'slow core'], 3)
2 (['funk', 'soul'], 4)
3 (['folk-pop', 'new americana'], 2)
4 ([], 4)
5 (['alternative pop', 'boston rock', 'lilith', 'melancholia'], 2)
6 (['acoustic pop', 'chamber pop', 'folk-pop', 'indie folk', 'indie pop', 'modern rock', 'neo mellow', 'new americana', 'stomp and holler'], 7)
7 (['slow core'], 3)
8 (['alternative rock', 'art rock', 'britpop', 'dance rock', 'electronic', 'madchester', 'new romantic', 'new wave', 'new wave pop', 'permanent wave', 'post-punk', 'rock', 'synthpop', 'uk post-punk'], 4)
9 (['funk', 'neo soul', 'soul'], 6)
10 (['blues-rock', 'classic rock', 'psychedelic rock', 'rock'], 2)

item[0] corresponds to a song (which has many tags associated to it).

item[1] corresponds to the count of song occurrences .

However, I need the total count by tag , and not by song.

I now I can isolate flattened tags in a list, like so:

def flatten(list):
    for sublist in list:
        for item in sublist:
            yield item

only_tags = [i[0] for i in tags_and_weights]
tags = list(flatten(only_tags))

and then, with pandas , quickly count them:

import pandas as pd
pd.Series(tags).value_counts()

but then I lose track of each tag weight...and total tag counts are misrepresented.

Considering I'll be doing these calculations with much bigger lists, what is the most efficient way of counting all tags, keeping track of tag weight, and then multiply each count by it to get the final count by tag?

Answer 1

You can try:

l = [(['alternative country', 'alternative pop', 'alternative rock', 'art rock', 'brill building pop', 'country rock', 'dance rock', 'experimental', 'folk', 'folk rock', 'garage rock', 'gbvfi', 'indie rock', 'jangle pop', 'lo-fi', 'melancholia', 'noise pop', 'post-punk', 'power pop', 'protopunk', 'psychedelic rock', 'pub rock', 'rock', 'roots rock', 'slow core'], 3)
,(['funk', 'soul'], 4)
,(['folk-pop', 'new americana'], 2)
,([], 4)
,(['alternative pop', 'boston rock', 'lilith', 'melancholia'], 2)
,(['acoustic pop', 'chamber pop', 'folk-pop', 'indie folk', 'indie pop', 'modern rock', 'neo mellow', 'new americana', 'stomp and holler'], 7)
,(['slow core'], 3)
,(['alternative rock', 'art rock', 'britpop', 'dance rock', 'electronic', 'madchester', 'new romantic', 'new wave', 'new wave pop', 'permanent wave', 'post-punk', 'rock', 'synthpop', 'uk post-punk'], 4)
,(['funk', 'neo soul', 'soul'], 6)
,(['blues-rock', 'classic rock', 'psychedelic rock', 'rock'], 2)]

tags, counts = zip(*l)

(pd.concat([pd.Series(counts[i], index=tags[i]) for i in range(len(tags))])
   .sum(level=0)
   .sort_values(ascending=False))

Using list comprehension with pd.concat and sum , after you unzip your list of tuples into two lists.

Output:

funk                   10
soul                   10
rock                    9
folk-pop                9
new americana           9
acoustic pop            7
indie folk              7
post-punk               7
dance rock              7
art rock                7
alternative rock        7
chamber pop             7
stomp and holler        7
neo mellow              7
modern rock             7
indie pop               7
slow core               6
neo soul                6
alternative pop         5
melancholia             5
psychedelic rock        5
britpop                 4
permanent wave          4
uk post-punk            4
synthpop                4
new wave pop            4
new wave                4
new romantic            4
madchester              4
electronic              4
brill building pop      3
gbvfi                   3
country rock            3
experimental            3
folk                    3
folk rock               3
garage rock             3
alternative country     3
indie rock              3
jangle pop              3
lo-fi                   3
noise pop               3
power pop               3
protopunk               3
pub rock                3
roots rock              3
blues-rock              2
boston rock             2
lilith                  2
classic rock            2
dtype: int64

Answer 2

Assuming that you created a DataFrame:

d = [(['alternative country', ... # Your data
df = pd.DataFrame(data=d, columns=['tags', 'weight'])

one of possible solutions, using pure Pandas , without any list comprehensions is as follows:

df.tags.apply(pd.Series).stack().reset_index(level=1, drop=True)\
    .rename('tag').to_frame().join(df.weight).groupby('tag').sum()\
    .sort_values(['weight', 'tag'], ascending=[False, True])

For learning purpose, you can try consecutive steps as separate operations and look at results.

Maybe an advantage is that tags are sorted, within groups with the same weight.