Pandas dataframe 在其他列中找到每个组的不同值计数

Question

I have a Pandas dataframe a sample input of which looks like below:

vendor  filename    language    score         text
Vendor 1    File 1  chinese 0.67717278        text1  
Vendor 2    File 1  chinese 0.644506991       text2
Vendor 1    File 2  chinese 0.67717278        text1  
Vendor 2    File 1  chinese 0.644506991       text3
Vendor 1    File 2  Arabic 0.999999523        text3
Vendor 1    File 1  Arabic 0.756420255        text2
Vendor 2    File 3  Arabic 0.999999523        text4
Vendor 1    File 1  Arabic 0.756420255        text4

What I am trying to do is for each language and within that language for each file, count the distinct number of values in text column where score is greater than 0.5 . So my ideal output for above sample input should be:

Chinese  File 1  3
         File 2  1

Arabic   File 1  2
         File 2  1
         File 3  1

Note that File 1 and File 2 are both used by Chinese and Arabic but I want to count their unique text values separately for each language.

I tried to use pandas groupby and unique function in below code but this is not working as it throws error as 'DataFrameGroupBy' object has no attribute 'unique' :

df_1 = df[df["score"] > 0.5].groupby(['language', 'filename']).unique().size()
    
print("Number of unique text greater than 0.5 score:{}".format(df_1))

What is the most ideal way to resolve this issue achieve the intended outcome?

Answer 1

Use DataFrameGroupBy.nunique with specify column text for count number of unique values:

df_1 = df[df["score"] > 0.5].groupby(['language', 'filename'], sort=False)['text'].nunique()

print("Number of unique text greater than 0.5 score:\n{}".format(df_1))
Number of unique text greater than 0.5 score:
language  filename
chinese   File 1      3
          File 2      1
Arabic    File 2      1
          File 1      2
          File 3      1
Name: text, dtype: int64