Keras 中的自定义损失和准确性

Question

I want to extend the output of the network to two outputs in this code

# Model architecture
  input = Input(shape = (max_len,))
  model = Embedding(input_dim = len(words) + 2, output_dim = embedding, input_length = max_len, mask_zero = True)(input)
  model = Bidirectional(LSTM(units = 50, return_sequences=True, recurrent_dropout=0.1))(model)
  model = TimeDistributed(Dense(50, activation="relu"))(model)
  crf = CRF(num_tags+1)  # CRF layer
  out = crf(model)  # output

  model = Model(input, out)
  model.compile(optimizer="rmsprop", loss=crf.loss_function, metrics=[crf.accuracy])

(The original code is at https://github.com/Akshayc1/named-entity-recognition.git ). I want the loss function to be the summation of losses for these two outputs, and accuracy to be mean of accuracy for these two outputs, I followed the available instructions:

model = TimeDistributed(Dense(50, activation="relu"))(model)
model1 = TimeDistributed(Dense(50, activation="relu"))(model)
crf1 = CRF(num_tags+1)  # CRF layer
crf2 = CRF(num_tags+1)
out1 = crf1(model)  # output
out2 = crf2(model1) 

model = Model(input, [out1, out2])

def custom_loss(y_true, y_pred1,  y_pred2):
    loss1 = losses.categorical_crossentropy(y_true,y_pred1)
    loss2 = losses.categorical_crossentropy(y_true,y_pred2)
    return (loss1 + loss2)/2


def Custom_accuracy(y_true, y_pred1,y_pred2, k=10):
    acc1 = K.mean(K.in_top_k(y_pred, K.argmax(y_true, axis=-1), k), axis=-1)
    acc2 = K.mean(K.in_top_k(y_pred1, K.argmax(y_true, axis=-1), k), axis=-1)
    acc = (acc1 + acc2)/2
    return acc
model.compile(optimizer="rmsprop", loss=custom_loss,metrics=Custom_accuracy) 

but this shows the error:

TypeError: custom_loss() missing 1 required positional argument: 'y_pred2'

Answer 1

It can be solved by passing two loss functions to loss argument in model.compile than to pass three variables in loss function as described in the documentation and also make classes for custom metric and loss . Make the following changes -

...

crf1 = CRF(num_tags+1,name="out1") <-- # change 1
crf2 = CRF(num_tags+1,name="out2") <-- # change 2
out1 = crf1(model)  
out2 = crf2(model1) 

model = Model(input, [out1, out2])

<define accuracy class and create its object> <-- # change 3
<define loss class and create its object> <-- # change 4

model.compile(optimizer="rmsprop", 
            loss={"out1":<loss_object_1>,"out2":<loss_object_2>},
            metrics={"out1":<accuracy_object_1>,"out2":<accuracy_object_2>}) <-- # change 5 

Answer 2

I think that is because the output of the model is not two outputs values( like o1,o2 ), but a list of two values(like [o1,o2] ). Try this custom loss function, hopefully it gets the issue gone:

def custom_loss(y_true, y_predsList):
    y_pred1,  y_pred2 = y_predsList
    loss1 = losses.categorical_crossentropy(y_true,y_pred1)
    loss2 = losses.categorical_crossentropy(y_true,y_pred2)
    return (loss1 + loss2)/2