Tensorflow 中嵌入层的工作

Question

Can someone please explain me the inputs and outputs along with the working of the layer mentioned below

model.add(Embedding(total_words, 64, input_length=max_sequence_len-1))

total_words = 263

max_sequence_len=11

Is 64, the number of dimensions?

And why is the output of this layer (None, 10, 64)

Shouldn't it be a 64 dimension vector for each word, ie (None, 263, 64)

Answer 1

In the Embedding layer the first argument represents the input dimensions (which is typically of considerable dimensionality). The second argument represents the output dimensions, aka the dimensionality of the reduced vector. The third argument is for the sequence length. In essence, an Embedding layer is simply learning a lookup table of shape (input dim, output dim) . The weights of this layer reflect that shape. The output of the layer, however, will of course be of shape (output dim, seq length) ; one dimensionality-reduced embedding vector for each element in the input sequence. The shape you were expecting is actually the shape of the weights of an embedding layer.

Answer 2

You can find all the information about the Embedding Layer of Tensorflow Here .

The first two parameters are input_dimension and output_dimension .

• The input dimensions basically represents the vocabulary size of your model. You can find this out by using the word_index function of the Tokenizer() function.
• The output dimensions are going to be Dimensions of the input of the next Dense Layer

The output of the Embedding layer is of the form (batch_size, input_length, output_dim) . But since you specified the input_length parameter, your layers input will be of the form (batch, input_length) . That's why the output is of the form (None, 10 ,64).

Hope that clears up your doubt ☺️