ValueError: Shape must be rank 3 but is rank 2.一个`Concatenate`层需要具有匹配形状的输入，除了concat

Question

I am trying to use Tensorflow Functional API to define a multi input neural network.

This is my code:

from keras_self_attention import SeqSelfAttention
from tensorflow import keras
Input1 = Input(shape=(120, ),name="Input1")
Input2 = Input(shape=(10, ),name="Input2")
embedding_layer = Embedding(30,5,  input_length=120,)(Input1) 
lstm_layer = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(units=512))(embedding_layer)                  
attention=SeqSelfAttention(attention_activation='sigmoid')(lstm_layer) 
merge = concatenate([attention, Input2])                                  

However, I get the following error:

ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, None, 1024), (None, 10)]. 

If I change shape of Input2 to (None,10, ), then I get this error:

ValueError: Shape must be rank 3 but is rank 2 for '{{node model/concatenate/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32](model/dense/BiasAdd, model/Cast_1, model/concatenate/concat/axis)' with input shapes: [?,?,1024], [?,10], [].

and if I change shape of Input2 to (1,10, ), then I get this error:

ValueError: Shape must be rank 3 but is rank 2 for '{{node model/concatenate/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32](model/dense/BiasAdd, model/Cast_1, model/concatenate/concat/axis)' with input shapes: [?,?,1024], [?,10], [].

How can I reshape output of attention layer from (None, None, 1024) to something which I can concatenate with (None, 10)?

Answer 1

The inputs for concatenate layer will not have matching dimensions. You can add a reshape layer in front of the concatenate layer to alleviate this issue.

[(None, None, 1024), (None, 10)].

Here one is three and one is two. Reshape the first input to [None,1024] or reshape the second input to [None, 1 , 10] whichever suits your need.

reshape_layer = tf.keras.layers.Reshape((1024,))(attention)
merge = concatenate([reshapelayer, Input2])