ValueError：分类指标无法处理多类和多标记指标目标的混合

Question

I have Multi class labeled text classification problem with 2000 different labels. Doing classification using LSTM with Glove Embedding.

1. Label Encoder of target variable
2. LSTM layer with Embedd Layer
3. Error metric is F2 score

LabelEncoded target variable:

le = LabelEncoder()  
le.fit(y)
train_y = le.transform(y_train)
test_y = le.transform(y_test)

LSTM network is like below with Glove Embeddings

np.random.seed(seed)
K.clear_session()
model = Sequential()
model.add(Embedding(max_features, embed_dim, input_length = X_train.shape[1],
         weights=[embedding_matrix]))#,trainable=False
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(num_classes, activation='softmax'))
model.compile(optimizer='rmsprop', loss='sparse_categorical_crossentropy')
print(model.summary())

My error metric is F1 score. I build below function for Error metric

ValueError: Classification metrics can't handle a mix of multiclass and continuous-multioutput targets

Model fit is

 model.fit(X_train, train_y, validation_data=(X_test, test_y),epochs=10, batch_size=64, callbacks=[metrics]) 

Getting below error after 1st epoch:

 ValueError: Classification metrics can't handle a mix of multiclass and continuous-multioutput targets 

Can you please tell me where I did mistake in my code?

I tried a lot to resolve my self, but I did not get any clue. Can you please help me on this

Answer 1

F1 score, recall and precision are metrics for binary classification for using it in a multiclass/multilabel problem you need to add a parameter to your function f1_score , recall_score and precision_score .

Try with this :

_val_f1 = f1_score(val_targ, val_predict, average='weighted')
_val_recall = recall_score(val_targ, val_predict, average='weighted')
_val_precision = precision_score(val_targ, val_predict, average='weighted')

Find more information for the average parameter here :
https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html

Answer 2

Your problem is caused by presence of continuous values in val_predict in this line of code

_val_f1 = f1_score(val_targ, val_predict)

You should round your predictions in val_predict before calculating f1_score.

Example solution:

 _val_f1 = f1_score(val_targ,np.round(val_predict))

Want to mention: If you want to change the threshold of the round function (0.5 default.) you can add or subtract values in [0,1] interval:

>>> a = np.arange(0,1,0.1)
>>> print(a, abs(np.round(a-0.1)), sep='\n')
>>> print(a, abs(np.round(a+0.3)), sep='\n')

[0.  0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9]
array([0.  0.  0.  0.  0.  0.  1.  1.  1.  1.])

[0.  0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9]
array([0., 0., 0., 1., 1., 1., 1., 1., 1., 1.])

Hope that helps!