简体   繁体   中英

Is texts_to_sequences, pad_sequences available in TensorFlow.js or is there any other alternative for this?

I have build a Keras model for next word prediction and I am trying to use my model in front-end for predicting next word based on input from the text field, I have to convert the following code from Python to JavaScript but did not find any suitable option. Is there any way to work around this?

from keras.preprocessing.sequence import pad_sequences
input_text = input().strip().lower()

encoded_text = tokenizer.texts_to_sequences([input_text])[0]
pad_encoded = pad_sequences([encoded_text], maxlen=seq_len, truncating='pre')

for i in (model.predict(pad_encoded)[0]).argsort()[-10:][::-1]:
  pred_word = tokenizer.index_word[i]
  print("Next word suggestion:",pred_word)

I am getting the following predictions for I in Python :

  • Next word suggestion: have
  • Next word suggestion: am
  • Next word suggestion: know
  • Next word suggestion: think
  • Next word suggestion: never
  • Next word suggestion: do
  • Next word suggestion: want
  • Next word suggestion: ever
  • Next word suggestion: will
  • Next word suggestion: see

I just wrote an alternative in node.js, maybe it can help you

Repo: https://github.com/Shadowhusky/node_tokenizer

You can install it with

npm install --save tf_node_tokenizer

Example:

const { Tokenizer } = require("tf_node_tokenizer");
const tokenizer = new Tokenizer({ num_words: 5, oov_token = "<unk>", });

const text = [
  "<start> Cake and frosting all over a face and hands tells a happy story.  <end>",
  "<start> A baby is feeding himself with his hands and is smeared with food. <end>",
  "<start> A baby eating pink dessert in a highchair <end>"
];

tokenizer.fitOnTexts(text);
tokenizer.texts_to_sequences(text);

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM