Keras preprocessing layer

Question

I am trying to feed a neural network 50 features (All Yes/No values) to predict the probability of one Yes/No label. I am trying to do this with keras CategoryEncoding , but running into some issues.

The start of my code is below:

model = Sequential([
    tf.keras.Input(shape = (50,)),
    tf.keras.layers.CategoryEncoding(num_tokens=100, output_mode='one_hot'),
    tf.keras.layers.LayerNormalization(),
    tf.keras.layers.Dense(1024, activation='relu'),
    tf.keras.layers.Dense(32, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(1, activation='softmax')
])

However, I get this error below:

ValueError: Exception encountered when calling layer "category_encoding_12" (type CategoryEncoding).

When output_mode is not `'int'`, maximum supported output rank is 2. Received output_mode one_hot and input shape (None, 50), which would result in output rank 3.

Call arguments received:
  • inputs=tf.Tensor(shape=(None, 50), dtype=float32)
  • count_weights=None

I am looking through the documentation, and I don't think I fully understand what a token is in its context here. Also, how would I preprocess my label here? I could use pd.get_dummies , but I don't know if tensorflow has anything that could do that automatically?

Answer 1

As error message suggests when the output mode is not int , use multi_hot instead of one_hot.

num_tokens

The total number of tokens the layer should support. All inputs to the layer must integers in the range 0 <= value < num_tokens, or an error will be thrown.

import tensorflow as tf
x = tf.keras.Input(shape = (50,))
y = tf.keras.layers.CategoryEncoding(
          num_tokens=100, output_mode="multi_hot")
y(x)

Output

<KerasTensor: shape=(None, 100) dtype=float32 (created by layer 'category_encoding_15')>