如何在Tensorflow和Keras中正确建模LSTM

Question

I have a dataset in a CSV format that looks like this:

 1,dont like the natives
 2,Keep it local always
 2,Karibu kenya

The label 1 indicates a hate speech while 2 indicates a positive.

Here is my code:

import numpy as np
import csv

import tensorflow as tf
from tensorflow.keras.layers import (
            Masking, LSTM, Dense, TimeDistributed, Activation)

def tokenize(text):
    """
    Change text string into number and
    make sure they resulting np.array is of the same size
    """
    Tokenizer = tf.keras.preprocessing.text.Tokenizer

    t = Tokenizer()
    t.fit_on_texts(text)
    tokenized_text = t.texts_to_sequences(text)

    tokenized_text = [item for sublist in tokenized_text for item in sublist]
    return np.resize(np.array(tokenized_text), (1, 30))


x_train = []
y_train = []

# Reading data from CSV
with open('data.csv') as csv_file:
    csv_reader = csv.reader(csv_file, delimiter=',')
    line_count = 0
    for row in csv_reader:
      line_count = line_count+1
      if line_count == 1:
        continue

      # Tokenize input data
      tokenized = tokenize(row[1])
      x_train.append(tokenized)
      y_train.append(row[0])

x_train = np.array(x_train).astype('float32')
y_train = np.array(y_train).astype('float32')

x_test = x_train[:3]
y_test = y_train[:3]

input_shape = x_train[0].shape
output_shape = y_train.shape
batch_size = len(y_train)

model = tf.keras.models.Sequential()
model.add(Masking(mask_value=-1, input_shape=input_shape))
model.add(LSTM(batch_size, dropout=0.2))
model.add(Dense(input_dim=batch_size, units=output_shape[-1]))
model.add(Activation('softmax'))

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=100, batch_size=batch_size)
model.evaluate(x_test, y_test)

for text in ["Karibu kenya", ]:
    tokenized_text = tokenize(text)
    prediction = model.predict(tokenized_text, batch_size=1, verbose=1)

    # Results
    print("Text: {}: Prediction: {}".format(text, prediction))

The rest of the code seems to be running well but I'm not able to run the model.predict(tokenized_text, batch_size=1, verboze=1)

I get the following error instead:

Epoch 97/100
19/19 [==============================] - 0s 196us/sample - loss: 0.8753 - accuracy: 0.5789
Epoch 98/100
19/19 [==============================] - 0s 246us/sample - loss: 0.8525 - accuracy: 0.6842
Epoch 99/100
19/19 [==============================] - 0s 169us/sample - loss: 0.7961 - accuracy: 0.6842
Epoch 100/100
19/19 [==============================] - 0s 191us/sample - loss: 0.7745 - accuracy: 0.7368
3/3 [==============================] - 0s 115ms/sample - loss: 0.5518 - accuracy: 1.0000
Traceback (most recent call last):
  File "start.py", line 65, in <module>
    prediction = model.predict(tokenized_text, batch_size=1, verbose=1)
  File "/home/felix/Projects/keras/.env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 821, in predict
    use_multiprocessing=use_multiprocessing)
  File "/home/felix/Projects/keras/.env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 705, in predict
    x, check_steps=True, steps_name='steps', steps=steps)
  File "/home/felix/Projects/keras/.env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 2428, in _standardize_user_data
    exception_prefix='input')
  File "/home/felix/Projects/keras/.env/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_utils.py", line 512, in standardize_input_data
    'with shape ' + str(data_shape))
ValueError: Error when checking input: expected masking_input to have 3 dimensions, but got array with shape (1, 30)

Not sure what I'm doing wrong. I have tried to change the data shape but still not working.

Thanks in advance.

Answer 1

Replace

prediction = model.predict(tokenized_text, batch_size=1, verbose=1)

with

prediction = model.predict(tokenized_text[None], batch_size=1, verbose=1)