Implement shared convolutional layers with keras tensorflow

Question

I have a network that has as a part of it a unet-like structure. I would like to make the convolutional layers shared between two inputs. An example of my code:

conv_layer = Conv(parameters)
out1 = con_layer(input1)
out2 = con_layer(input2)

Does this part create two outputs that each one of them depends only on the correspondent input and the shared weights?Or does it concatenate the inputs and passes them from convolution? Are the weights the same in the two calls of this layer? Also, a question about the learning. When it comes to backpropagate, does the loss propagate once from the shared layers? Is there any change in learning?

Answer 1

First of all, U-Net doesn't exactly have any shared layer. It uses skip connections and concatenation to reuse features.

A shared layer looks something like this

x --> F(x)
          ==> G(F(x),F(y))
y --> F(y) 

Does this part create two outputs that each one of them depends only on the correspondent input and the shared weights?

• Yes, the shared weight is fixed for both inputs and the output depends on each input.

Does it concatenate the inputs and passes them from convolution?

• No, it doesn't perform any concatenation. Even the two inputs are completely independent and don't interact directly - they only pass through the conv_layer.

Are the weights the same?

• Yes, the weights of the shared layer are exactly the same.

When it comes to backpropagate, does the loss propagate once from the shared layers? Is there any change in learning?

• Not sure what do you mean but the weights will be updated for both of the inputs but at the same time. If by a change in learning you mean with relative to non-shared layer, yes.

Some useful reading: http://neural.vision/blog/deep-learning/backpropagation-with-shared-weights/

https://datascience.stackexchange.com/questions/26755/cnn-how-does-backpropagation-with-weight-sharing-work-exactly

https://datascience.stackexchange.com/questions/27506/back-propagation-in-cnn