我想计算数据集中的精度、召回率和准确率

Question

I have two sets of Data: Predicted and Actual.

An algorithm predicts at max five unique pre-defined parameters that are in Predicted. Assuming the parameters are alphabets from a to z. So for one row, I see how many out of these paramaters were predicted correctly.

Predicted:

Index P1 P2 P3 P4 P5

1     a  b  c  q

2     g

3     s  f  g  v  t


Actual:

Index P1 P2 P3 P4 P5

1     a  s  q  r  t  

2     g

3     t  v

code to generate these dataframes:

import pandas as pd

predicted = pd.DataFrame.from_records(columns =['P'+str(i) for i in range(1,6)],
                                      data=[['a','b','c','q'],['g'], ['s','f','g','v','t']])


actual = pd.DataFrame.from_records(columns =['P'+str(i) for i in range(1,6)],
                                      data=[['a','s','q','r','t'],['g'], ['t','v']])

For Row1: Correctly predicted parameters: a,q
For Row2: Correctly predicted parameters: g
For Row3: Correctly predicted parameters: t,v

How do I calculate Precision, Recall and Accuracy for the same?

Answer 1

To calculate the accuracy here, you just need to count the number of occurrences where the predicted parameter matched the actual parameter, ignoring cases where they were both None . Plenty of ways to do that, I'd take this simple option:

accuracy = ((predicted==actual) & (predicted.notna() | actual.notna())).sum().sum() / (predicted.notna() | actual.notna()).sum().sum()

You can verify that it fits the desired result (2/11).

precision/recall is a bit trickier for multi-class data - you can do it per label but you certainly don't have enough data here. I'd stick with accuracy for this case...

PS I've assumed that your accuracy calculation is pretty straight-forward. If it isn't - it should be specified explicitly in your question...