如何创建具有本地连接层和密集父层的本地连接层？

Question

I currently have a network ( image of toy example ) with two input layers. in1 is just a short, flat vector of values but in2 is a 27-channel image. I want my network to be structured on locally-connected layers but I don't know a good way to sprinkle in in1 's data with in2 . I currently flatten in2 's branch after a few layers, merge with in1 , and add dense layers onward.

How can I densely introduce in1 's data while maintaining the locally-connected architecture? The image linked above shows this goal with a red arrow.

One possible solution that I came up with is to copy in1 's vector as channels to in2 such that in2 's dimension would be width * height * (num_original_channels + len(in1) ) . This seems inelegant because it would be copying in1 many times. There must be a better way.

I'm new to keras so please pardon my shaky vocabulary. Also, this is a toy example just to illustrate my idea so there may be some other/unrelated architectural criticisms.

Thanks in advance for any advice!

fwiw, here is the code I am using:

input1 = Input( ... ) #small flat vec
input2 = Input( ... ) #deep picture

pre = Reshape( ... )( input2 )
l1 = LocallyConnected2D( ... )( pre )
l2 = LocallyConnected2D( ... )( l1 )
l3 = LocallyConnected2D( ... )( l2 )
flat = Flatten( ... )( l3 )
merge = tensorflow.keras.layers.concatenate( [flat, input1], ... )
l4 = Dense( ... )( merge )
l5 = Dense( ... )( l4 )
output = Dense( ... )( l5 )

Answer 1

Answering my own question here. It seems like the best solution is to have both input1 and input2 create two separate tensors with the same layer and no activation funciton, sum them together, then add the activation.

Using my example from before it would look something like this:

(I'm adding example dimensions to hopefully clarify what I mean. They're made up from thin air)

input1 = Input( ... ) #small flat vec, 1x200
input2 = Input( ... ) #deep picture,   50x50x10

l1 = LocallyConnected2D( activation=None, ... )( input2 ) # 40x40x5

num_elements = 40 * 40 * 5
d1 = Dense( units=num_elements, activation=None, ... )( input1 ) # 1x8000
d1_3D = Reshape( target_shape=(40, 40, 5,) )( d1 ) #40x40x5

merge = Add()([ l1, d1_3D ]) #40x40x5
l2 = LeakyReLU( ... )( merge ) #Or whatever activation function you want, 40x40x5

# ...