了解 Keras MNIST 连体网络并将其调整为三元组

Question

I am currently adapting this siamese network in Python with Keras. However, I currently do not understand how the loss works (not the function itself, but which parameters get passed where)

Okay, now step by step how I think this works:

distance = Lambda(euclidean_distance,
              output_shape=eucl_dist_output_shape)([processed_a, processed_b])

This is the line where the outputs of both individual networks get combined and the custom layer applies the following functions:

def euclidean_distance(vects):
    x, y = vects
    sum_square = K.sum(K.square(x - y), axis=1, keepdims=True)
    return K.sqrt(K.maximum(sum_square, K.epsilon()))


def eucl_dist_output_shape(shapes):
    shape1, shape2 = shapes
    return (shape1[0], 1)

So when the input to this layer is (128, 128) the output would be (128, 1). In the last step the loss is calculated with:

def contrastive_loss(y_true, y_pred):
    '''Contrastive loss from Hadsell-et-al.'06
    http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf
    '''
    margin = 1
    square_pred = K.square(y_pred)
    margin_square = K.square(K.maximum(margin - y_pred, 0))
    return K.mean(y_true * square_pred + (1 - y_true) * margin_square)

Here, the predicted 128D vector is compared to the 128D ground truth vector.

Now I changed the Lambda layer to:

distance = Lambda(euclidean_distance,
                  output_shape=eucl_dist_output_shape)([processed_a, processed_b, processed_c])

so I have three networks now with the following adapted functions (which should just combine the three outputs to one output, with a shape of (128, 3)):

def euclidean_distance(vects):
    return vects


def eucl_dist_output_shape(shapes):
    shape1, shape2, shape3 = shapes
    return (shape1, shape2, shape3)

and then the new loss function:

def loss_desc_triplet(vects, margin=5):
    """Triplet loss.
    """
    d1, d2, d3 = vects
    d_pos = K.sqrt(K.sum(K.square(d1 - d2), axis=1))
    pair_dist_1_to_3 = K.sqrt(K.sum(K.square(d1 - d3), axis=1))
    d_neg = pair_dist_1_to_3

    return Activation.relu(d_pos - d_neg + margin)

But now I get this error:

File "DeepLearningWithAugmentationWithTriplets.py", line 233, in output_shape=eucl_dist_output_shape)([processed_a, processed_b, processed_c])

File "lib/python3.7/site-packages/keras/engine/base_layer.py", line 497, in call arguments=user_kwargs)

File "lib/python3.7/site-packages/keras/engine/base_layer.py", line 565, in _add_inbound_node output_tensors[i]._keras_shape = output_shapes[i]

IndexError: list index out of range

But I am not sure what causes this.

Answer 1

I fixed the problem by concatenating the outputs:

merged_vector = concatenate([processed_a, processed_b, processed_c], axis=-1, name='merged_layer')

and then disassembling the vector in my loss function:

d1 = y_pred[:,0:128]
d2 = y_pred[:,128:256]
d3 = y_pred[:,256:384]

Though, I am not sure if that is the best solution.