如何使用 Keras 预测 NLP 模型的 function？

Question

I have created an NLP classification model with keras with no problems with my model showing 83.5% accuracy upon evaluation. However, when I want to use my model to predict a new set of tokenized words, my model returns x number of arrays where x is the number of tokens in a tokenized sentence I have given to my model to predict

` here is the code example

    toPredict = np.array([1,2])

    prediction = self.model.predict(toPredict)
    print(prediction)

` The values 1 and 2 are obviously just token values, but this will return an output of '

    [[0.24091144 0.20921658 0.3415633  0.20830865]
    [0.20159791 0.46421158 0.19968869 0.13450184]]

' I may be missing something, but i thought the output would be only 1 array to classify the whole tokenized sentence, not each word individually. Am I feeding in the model a badly formatted input? Please help!

Answer 1

to predict you should feed model in the same shape that training data fed into model; so the sequence must have been in 2-dim shape and even the same length as you set before when padded sequences. you could tf.expand_dims(toPredict, 0) and then feed it into model.

for instance here i will define a function for prediction;

#def prediction
def predict_text(#define input text and model
                 input_text, tokenizer, model, 
                 #define tokenizer maximum length of sequence
                 maxlen_seq, padding = 'post', truncating = 'post'
                 ):
    
    #prediction
    text = str(input_text)
    sequence = tokenizer.texts_to_sequences([text])
    sequence = keras.preprocessing.sequence.pad_sequences(sequence, maxlen = maxlen_seq,
                                                          padding = padding, truncating = truncating)
    predict = model.predict(sequence)
    
    return predict