如何让多类 Keras 模型输出字符串而不是 OHE？

Question

I understand the idea of passing labels in training data to a Keras model as a one-hot encoding, but I'm trying to create a model that will return a string during inference, and not a one-hot encoding that I'd have to decode myself.

Specifically, I DON'T want to have to do the following:

encoder = LabelEncoder() 
encoder = encoder.fit(labels)
encoded_Y = encoder.transform(labels)
y_true = np_utils.to_categorical(encoded_Y). # Model accepts this during training

prediction = model.predict(query)
label_string = encoder.inverse_transform(prediction)

How can I create a model that will call .predict() and return something customized, such as the string of the highest prediction and its corresponding probability?

Answer 1

You can take a complete model, and create a custom model on top of it that includes a post-processing function.

Let's take a pre-trained model:

import tensorflow as tf
from skimage.data import chelsea

model = tf.keras.applications.MobileNetV2()

That model comes with a function that takes the output probability matrix, and turns it into strings (you'll have to make your own)

decoder = tf.keras.applications.mobilenet_v2.decode_predictions

Then, we create a model that has two parts: 1) the model 2) the post-processing function

class NewModel(tf.keras.models.Model):
    def __init__(self, model, decoder):
        super(NewModel, self).__init__()
        self.model = model
        self.decoder = decoder

    def call(self, x):
        x = self.model(x).numpy()
        x = self.decoder(x)
        return x


m = NewModel(model, decoder)

Now, just call it with appropriate input:

cat_picture = tf.image.resize(chelsea(), (224, 224))[None, ...]/255

m(cat_picture)

[[('n02124075', 'Egyptian_cat', 0.7610773),
  ('n02123045', 'tabby', 0.12039327),
  ('n02123159', 'tiger_cat', 0.039847013),
  ('n02127052', 'lynx', 0.009957394),
  ('n04553703', 'washbasin', 0.0015057808)]]

It returns the output of the post-processing function.