当输出变量是连续的时，我们如何调整神经网络的超参数？

Question

I've been looking into resources for tuning hyperparameters of ANN models using TensorFlow in Python and by far, every solution/discussion I've come across deals with classification problems with categorical output dataset and use simple logistic regression models or GridSearch through sklearn (so far, those that deal with binary output only).

My problem, however, has a continuous output and I was trying to implement GridSearch, but I ran into an error:
"ValueError: continuous is not supported"

1. I'm not sure if GridSearch works when the output variable is continuous, but even if it doesn't, there must be some other functions that can help me optimize my parameters?
   
2. Do we need to normalize input as well as output variables in this scenario? What is the best way of doing so for continuous input and output variables?

from keras.wrappers.scikit_learn import KerasClassifier
from sklearn.model_selection import GridSearchCV
from keras.models import Sequential
from keras.layers import Dense
def build_classifier(optimizer):
    classifier = Sequential()
    classifier.add(Dense(10, input_dim = 5, activation = 'relu', kernel_initializer="uniform"))
    classifier.add(Dense(5, activation = 'relu', kernel_initializer = 'uniform'))
    classifier.add(Dense(1, activation = 'linear', kernel_initializer = 'uniform'))
    classifier.compile(optimizer = optimizer, loss = 'mse', metrics = ['accuracy'])
    return classifier
classifier = KerasClassifier(build_fn = build_classifier)
parameters = {'batch_size': [25, 32],
              'epochs': [100, 500],
              'optimizer': ['adam','rmsprop']}
grid_search = GridSearchCV(estimator = classifier, param_grid = parameters, scoring = 'accuracy', cv = 3)
grid_search = grid_search.fit(X_train, y_train)
best_parameters = grid_search.best_params_
best_accuracy = grid_search.best_score_

Answer 1

First, define your model in a function(you could also define as a class with the keras subclassing API), then, since your output is a continuous value, call keras scikit-learn wrapper KerasRegressor in order to work with GridSearchCV or RandomizedSearchCV.

Read this to better understand the differences between classification and regression problems.

def build_model(n_neurons=30, learning_rate=1e-3, input_shape=[8]):
    model = keras.models.Sequential()
    model.add(keras.layers.InputLayer(input_shape))
    model.add(keras.layers.Dense(n_neurons, activation="relu"))
    model.add(keras.layers.Dense(1))
    optimizer = keras.optimizers.SGD(lr=learning_rate)
    model.compile(loss="mse", optimizer=optimizer)
    return model

keras_reg = keras.wrappers.scikit_learn.KerasRegressor(build_model)
params_distrib = { #  params you want try with }
rnd_search_cv = RandomizedSearchCV(keras_reg, params_distrib, n_iter=5, cv=3)
rnd_search_cv.fit(x_train, y_train, epochs=50, 
                    validation_data=(x_valid, y_valid))