如何在 python 中使用多处理计算 f1 分数？

Question

I've got an array of paired binary labels: y_true , y_pred . My array contains ~50 million elements, and I wish to evaluate success using f1 score preferably, or AUC.

However, calculating f1 using sklearn takes relatively long time – about half the time needed for an entire epoch. Calculating AUC was faster, but too slow as well. Similar question yielded Faster AUC in sklearn or python , but I'm not sure I can try this one.

Is there a way to speed up those calculations, perhaps with multiprocessing?

Answer 1

Alright, so apparently Scikit-learn implementation ran too slow on my two vectors, so I modified this implementation , so it fits numpy arrays and it is now much faster (0.25 seconds compared to 50+ seconds on sk-learn). Original implementation using torch.Tensors.

def f1_loss(y_true, y_pred, beta=1) -> numpy.float32:
    '''Calculate F1 score.
    
    The original implmentation is written by Michal Haltuf on Kaggle.
    
    Reference
    ---------
    - https://www.kaggle.com/rejpalcz/best-loss-function-for-f1-score-metric
    - https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html#sklearn.metrics.f1_score
    - https://discuss.pytorch.org/t/calculating-precision-recall-and-f1-score-in-case-of-multi-label-classification/28265/6
    
    '''
    assert y_true.shape[1] == 1
    assert y_pred.shape[1] == 1
        
    
    tp = (y_true * y_pred).sum()
    tn = ((1 - y_true) * (1 - y_pred)).sum()
    fp = ((1 - y_true) * y_pred).sum()
    fn = (y_true * (1 - y_pred)).sum()
    
    epsilon = 1e-7
    
    precision = tp / (tp + fp + epsilon)
    recall = tp / (tp + fn + epsilon)
    
    f1 = (1 + beta**2)* (precision*recall) / (beta**2 * precision + recall + epsilon)

    return f1