f1_score : ValueError all the input arrays must have same number of dimensions

Question

I would like to compute f1_score .

The code is as shown below:

if __name__ == '__main__':
    y_pred_df = pd.read_csv('file1.csv', skipinitialspace=True, sep='\t', header=None, dtype= str)
    y_pred = y_pred_df.values

    y_true_df = pd.read_csv('file2.csv', header=None, dtype= str)
    y_true = y_true_df.values

    test_score = accuracy_score(y_true[:,0], y_pred[:,0])
    print("\n Accuracy score (Random Forest with 100 estimators) : {}%".format(round(test_score*100,2)))

    print (y_true[:,0])
    print (y_pred[:,0])

    score_test = f1_score(y_true[:,0], y_pred[:,0],pos_label=list(set(y_true[:,0])),average = 'weighted')


    print (score_test)

When executing the above code, I get the following error when computing f1_score :

Accuracy score (Random Forest with 100 estimators) : 61.62%
['4' '4' '4' '4' '4' '12' '12' '12' '12' '12' '12' '12' '12' '4' '4' '4'
'4' '4' '4' '4' '4' '4' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12'     '12'
'12' '12' '12' '12' '4' '4' '4' '4' '4' '4' '4' '4' '4' '4' '4' '4' '4'
'4' '4' '4' '4' '4' '12' '12' '4' '4' '4' '12' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '4' '4'
'4' '4' '4' '4']

['4' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '4' '12' '4' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12' '12'
'12' '12' '12' '12' '12' '12' '12' '12' '12']
Traceback (most recent call last):
  File "<ipython-input-25-f80f0ca3aea2>", line 1, in <module>
runfile('C:/Anaconda3/envs/python27/Scripts/spade/examples/project/Fmeasure.py', wdir='C:/Anaconda3/envs/python27/Scripts/spade/examples/project')

  File "C:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 714, in runfile
    execfile(filename, namespace)

  File "C:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 89, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Anaconda3/envs/python27/Scripts/spade/examples/project/Fmeasure.py", line 47, in <module>
    score_test = f1_score(y_true[:,0], y_pred[:,0],pos_label=list(set(y_true[:,0])),average = 'binary')

  File "C:\Anaconda3\lib\site-packages\sklearn\metrics\classification.py", line 639, in f1_score
    sample_weight=sample_weight)

  File "C:\Anaconda3\lib\site-packages\sklearn\metrics\classification.py", line 756, in fbeta_score
    sample_weight=sample_weight)

  File "C:\Anaconda3\lib\site-packages\sklearn\metrics\classification.py", line 992, in precision_recall_fscore_support
    assume_unique=True)])

  File "C:\Anaconda3\lib\site-packages\numpy\core\shape_base.py", line 280, in hstack
    return _nx.concatenate(arrs, 1)

ValueError: all the input arrays must have same number of dimensions

Can you please tell me the problem source?

Answer 1

pos_label must contain only one element, you are passing a list of labels.

pos_label is intended to compute the f1 score of one label at a time, when you pass a list it crashes. If you want to compute the f1 per label you should then make a loop where you iterate through the set of labels as follows:

for label in set(yt)
    score_test = f1_score(yt_, yp_, pos_label=[label])
    print( 'f1', label, score_test )

If what you want is the weighted average of the f1 scores, then you should not use pos_label, instead

score_test = f1_score(yt_, yp_, average = 'weighted')

However, On sklearn 0.20 the following works but it gives you a warning

from sklearn.metrics import f1_score

if __name__ == '__main__':
    yt_ =  ['4', '4', '4', '4', '4', '12', '12', '12', '12', '12', '12', '12', '12', '4', '4', '4', '4', '4', '4', '4', '4', '4', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '4', '12', '12', '4', '4', '4', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '4', '4', '4', '4', '4', '4']

    yp_ = ['4', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '4', '12', '4', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12', '12'] 

    score_test = f1_score(yt_, yp_, pos_label=list(set(yt_)),average = 'weighted')

    print (score_test)  

the warning:

UserWarning: Note that pos_label (set to ['12', '4']) is ignored when average != 'binary' (got 'weighted'). You may use labels=[pos_label] to specify a single positive class.