Keras 中嵌入层的输入形状出现问题

Question

I am new at modeling in Keras Python. I wanted to create a model that will generate text answer to my messages like chatbot. I read that I need to use tokenizer and to_categorical method. There is my code:

import numpy as np
from keras.layers import Dense, LSTM, Input, Dropout, Embedding
from keras.models import Sequential
from keras.optimizers import Adam
from keras.preprocessing.text import Tokenizer, text_to_word_sequence
from keras.utils import pad_sequences, to_categorical


X = ["Hello!", "Greetings!"]

tokenizer = Tokenizer(1000)

X_seq = pad_sequences(tokenizer.texts_to_sequences(X), 20)

model = Sequential()

model.add(Embedding(1000, 100, input_length=20))
model.add(LSTM(100))
model.add(Dense(1000, "softmax"))

model.summary()

model.compile(loss='categorical_crossentropy', metrics=['accuracy'])

history = model.fit(X_seq, to_categorical(X_seq), epochs=10, batch_size=20)

print(model.predict(pad_sequences(tokenizer.texts_to_sequences(["Greetings!"]), 20)))

value 1000 is max count of unique words in tokenizer vocabulary X is my input list of messages. 20 is max length of text. I wanted to create self-learning model but I don't really inderstand how to do it. In inte.net I found that I need to pass same input and output values but my model should return something like that [0.1,0.2,0.3....] - the max value represents the word which model predicted. But when I try to fit that it raises exception:

ValueError: Shapes (None, 20) and (None, 1000) are incompatible

I guess that it is a trouble with shapes of input list and embedding layer. Guys please help me deal with that. Thank you in advance!

Answer 1

In this particular example using to_categorical(X_seq) gives you targets having [num_of_samples, 20] shape. It means that if you use this target variable to fit model it will expect that number of classes will equal 20 (model output shape = (None, 20) ). So to fix your example I would suggest you change model.add(Dense(1000, "softmax")) to model.add(Dense(20, "softmax")) .

Additionaly before using tokenizer.texts_to_sequences(X) I would suggest you to fit tokenizer using tokenizer.fit_on_texts(X) .