Given the following model :
Layer (type) Output Shape Param #
=================================================================
input_91 (InputLayer) [(None, 25)] 0
_________________________________________________________________
token_and_position_embedding (None, 25, 400) 5938800
_________________________________________________________________
transformer_block_97 (Transf (None, 25, 400) 74832
_________________________________________________________________
global_average_pooling1d_82 (None, 400) 0
_________________________________________________________________
dropout_337 (Dropout) (None, 400) 0
_________________________________________________________________
dense_339 (Dense) (None, 25) 22575
_________________________________________________________________
dropout_338 (Dropout) (None, 25) 0
_________________________________________________________________
dense_340 (Dense) (None, 25) 570
=================================================================
Total params: 3,709,907
Trainable params: 3,709,907
Non-trainable params: 0
In keras, how to change the output layer to (None, 25, 7)
dimension? This is the current model configuration:
embed_dim = 400 # Embedding size for each token
num_heads = 2 # Number of attention heads
ff_dim = 32 # Hidden layer size in feed forward network inside transformer
inputs = layers.Input(shape=(25,))
embedding_layer = TokenAndPositionEmbedding(maxlen, vocab_size, embed_dim)
X = embedding_layer(inputs)
transformer_block = TransformerBlock(embed_dim, num_heads, ff_dim)
X = transformer_block(X)
X = layers.GlobalAveragePooling1D()(X)
X = layers.Dropout(0.1)(X)
X = layers.Dense(25, activation="relu")(X)
X= layers.Dropout(0.1)(X)
outputs = layers.Dense(25, activation="softmax")(x)
You are looking for tf.keras.layers.Reshape
. Per our discussion in the comments, see how to reshape a layer from (None, 25)
to (None, 5, 5)
.
inp = tf.keras.layers.Input((25))
layer = tf.keras.layers.Dense((25))(inp)
reshaped = tf.keras.layers.Reshape((5,5))(layer)
model = tf.keras.Model(inp, reshaped)
model.summary()
yields
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_3 (InputLayer) [(None, 25)] 0
_________________________________________________________________
dense_1 (Dense) (None, 25) 650
_________________________________________________________________
reshape_2 (Reshape) (None, 5, 5) 0
=================================================================
Total params: 650
Trainable params: 650
Non-trainable params: 0
EDIT:
To clarify how you would implement this into your code, add the following after outputs = layers.Dense(25, activation="softmax")(x)
reshaped_outputs = layers.Reshape((5,5))(outputs)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.