Keras: Using Predict with a Model Trained with Normalized Data?

Question

I'm creating a deep neural network in Keras to perform an NN regression using tabular data. Best practice is to normalize the inputs and output series. I'd also like to use the predict function to provide estimates of the model's output for various sets of inputs. If the training data was normalized, I assume I'll need to also normalize the predict data set using the same scaling parameters. What's the best way to do this? Is there a way to automatically normalize the data within the model?

Answer 1

I typically like to use sklearn for this, and it does save the parameters and allows you to "inverse transform" back to the original values. For predictions you would send them through the inverse_transform function to get their real predicted values.

Here is a working example for you to reference. The parameters of the scalers can be easily adjusted.

from sklearn.preprocessing import MinMaxScaler, StandardScaler
import numpy as np

example = np.array([0., 1., 1., 0., 2., 3., 4., 4., 5.]).reshape(-1, 1)

# MinMax Scaling Example
scaler = MinMaxScaler(feature_range=(0.01, 0.99))
min_max_scaled = scaler.fit_transform(example)
min_max_orig = scaler.inverse_transform(min_max_scaled)

# Normalizing Example  (mean 0, std 1)
norm = StandardScaler()
normalized = norm.fit_transform(example)
normalized_orig = norm.inverse_transform(normalized)

Answer 2

There is no best way to do this (it depends on the problem), but the most common thing to do is to normalize both the train and the test data so that they have mean 0 and standard deviation 1.

Yes, with Batch Normalization you can automatically normalize the data within the model provided that you feed batches of a reasonable size into the network. This might produce a similar effect to data augmentation, because the signals the network will see during training will rarely repeat (as signals for one example now depend on its entire batch). In Keras, this can be implemented by adding a BatchNorm layer right after the input layer.