如何更改 keras 模型中密集层的输出？

Question

Given the following model :

Layer (type)                 Output Shape              Param #   
=================================================================
input_91 (InputLayer)        [(None, 25)]              0         
_________________________________________________________________
token_and_position_embedding (None, 25, 400)           5938800   
_________________________________________________________________
transformer_block_97 (Transf (None, 25, 400)           74832    
_________________________________________________________________
global_average_pooling1d_82  (None, 400)               0         
_________________________________________________________________
dropout_337 (Dropout)        (None, 400)               0         
_________________________________________________________________
dense_339 (Dense)            (None, 25)                22575     
_________________________________________________________________
dropout_338 (Dropout)        (None, 25)                0         
_________________________________________________________________
dense_340 (Dense)            (None, 25)                570      
=================================================================
Total params: 3,709,907
Trainable params: 3,709,907
Non-trainable params: 0

In keras, how to change the output layer to (None, 25, 7) dimension? This is the current model configuration:

embed_dim = 400  # Embedding size for each token
num_heads = 2  # Number of attention heads
ff_dim = 32  # Hidden layer size in feed forward network inside transformer

inputs = layers.Input(shape=(25,))


embedding_layer = TokenAndPositionEmbedding(maxlen, vocab_size, embed_dim)
X = embedding_layer(inputs)
transformer_block = TransformerBlock(embed_dim, num_heads, ff_dim)
X = transformer_block(X)
X = layers.GlobalAveragePooling1D()(X)
X = layers.Dropout(0.1)(X)
X = layers.Dense(25, activation="relu")(X)
X= layers.Dropout(0.1)(X)

outputs = layers.Dense(25, activation="softmax")(x)

Answer 1

You are looking for tf.keras.layers.Reshape . Per our discussion in the comments, see how to reshape a layer from (None, 25) to (None, 5, 5) .

inp = tf.keras.layers.Input((25))                                                                                   
layer = tf.keras.layers.Dense((25))(inp)                                                                            
reshaped = tf.keras.layers.Reshape((5,5))(layer)                                                                    
model = tf.keras.Model(inp, reshaped)

model.summary() yields

_________________________________________________________________                                                       
Layer (type)                 Output Shape              Param #                                                          
=================================================================                                                       
input_3 (InputLayer)         [(None, 25)]              0                                                                
_________________________________________________________________                                                       
dense_1 (Dense)              (None, 25)                650                                                              
_________________________________________________________________                                                       
reshape_2 (Reshape)          (None, 5, 5)              0                                                                
=================================================================                                                       
Total params: 650                                                                                                       
Trainable params: 650                                                                                                   
Non-trainable params: 0   

EDIT:

To clarify how you would implement this into your code, add the following after outputs = layers.Dense(25, activation="softmax")(x)

reshaped_outputs = layers.Reshape((5,5))(outputs)