超參數調整 (Keras) 一種神經網絡回歸

Question

我們在 Python 中開發了一個人工神經網絡，在這方面，我們希望使用 GridSearchCV 調整超參數以找到可能的最佳超參數。 我們的人工神經網絡的目標是根據其他相關特征來預測溫度，到目前為止，這是對神經網絡性能的評估：

Coefficient of Determination (R2)    Root Mean Square Error (RMSE)    Mean Squared Error (MSE)    Mean Absolute Percent Error (MAPE)    Mean Absolute Error (MAE)    Mean Bias Error (MBE)
0.9808840288506496                   0.7527763482280911               0.5666722304516204          0.09142692180578049                   0.588041786518511           -0.07293321963266877

到目前為止，我們不知道如何正確使用 GridSearchCV，因此我們尋求幫助以使我們朝着滿足我們目標的解決方案前進。 我們有一個 function 可能有效，但無法將其正確應用於我們的代碼。

這是超參數調優 function (GridSearchCV)：

def hyperparameterTuning():
    # Listing all the parameters to try
    Parameter_Trials = {'batch_size': [10, 20, 30],
                    'epochs': [10, 20],
                    'Optimizer_trial': ['adam', 'rmsprop']
                    }

    # Creating the regression ANN model
    RegModel = KerasRegressor(make_regression_ann, verbose=0)

    # Creating the Grid search space
    grid_search = GridSearchCV(estimator=RegModel,
                           param_grid=Parameter_Trials,
                           scoring=None,
                           cv=5)

    # Running Grid Search for different paramenters
    grid_search.fit(X, y, verbose=1)

    print('### Printing Best parameters ###')
    grid_search.best_params_

---

我們主要的function：

if __name__ == '__main__':

    print('--------------')

    dataframe = pd.read_csv("/.../file.csv")
    
    # Splitting data into training and tesing data
    X_train, X_test, y_train, y_test, PredictorScalerFit, TargetVarScalerFit = splitData(dataframe=dataframe)
    
    # Making the Regression Artificial Neural Network (ANN)
    ann = ANN(X_train=X_train, y_train=y_train, X_test=X_test, y_test=y_test, PredictorScalerFit=PredictorScalerFit, TargetVarScalerFit=TargetVarScalerFit)

    # Evaluation of the performance of the Aritifical Neural Network (ANN)
    eval = evaluation(y_test_orig=ann['temp'], y_test_pred=ann['Predicted_temp'])

我們的 function 將數據拆分為訓練和測試數據：

def splitData(dataframe):

    X = dataframe[Predictors].values
    y = dataframe[TargetVariable].values

    ### Sandardization of data ###
    PredictorScaler = StandardScaler()
    TargetVarScaler = StandardScaler()

    # Storing the fit object for later reference
    PredictorScalerFit = PredictorScaler.fit(X)
    TargetVarScalerFit = TargetVarScaler.fit(y)

    # Generating the standardized values of X and y
    X = PredictorScalerFit.transform(X)
    y = TargetVarScalerFit.transform(y)

    # Split the data into training and testing set
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

    return X_train, X_test, y_train, y_test, PredictorScalerFit, TargetVarScalerFit

我們的 function 適合 model 並利用人工神經網絡 (ANN)

def ANN(X_train, y_train, X_test, y_test, TargetVarScalerFit, PredictorScalerFit):

    model = make_regression_ann()

    # Fitting the ANN to the Training set
    model.fit(X_train, y_train, batch_size=5, epochs=100, verbose=1)

    # Generating Predictions on testing data
    Predictions = model.predict(X_test)

    # Scaling the predicted temp data back to original price scale
    Predictions = TargetVarScalerFit.inverse_transform(Predictions)

    # Scaling the y_test temp data back to original temp scale
    y_test_orig = TargetVarScalerFit.inverse_transform(y_test)

    # Scaling the test data back to original scale
    Test_Data = PredictorScalerFit.inverse_transform(X_test)

    TestingData = pd.DataFrame(data=Test_Data, columns=Predictors)
    TestingData['temp'] = y_test_orig
    TestingData['Predicted_temp'] = Predictions
    TestingData.head()

    # Computing the absolute percent error
    APE = 100 * (abs(TestingData['temp'] - TestingData['Predicted_temp']) / TestingData['temp'])
    TestingData['APE'] = APE

    # ...
    TestingData = TestingData.round(2)

    TestingData.to_csv("TestingData.csv")

    return TestingData

我們的 function 使 ANN 的 model

def make_regression_ann():
    # create ANN model
    model = Sequential()

    # Defining the Input layer and FIRST hidden layer, both are same!
    model.add(Dense(units=8, input_dim=7, kernel_initializer='normal', activation='sigmoid'))

    # Defining the Second layer of the model
    # after the first layer we don't have to specify input_dim as keras configure it automatically
    model.add(Dense(units=6, kernel_initializer='normal', activation='sigmoid'))

    # The output neuron is a single fully connected node
    # Since we will be predicting a single number
    model.add(Dense(1, kernel_initializer='normal'))

    # Compiling the model
    model.compile(loss='mean_squared_error', optimizer='adam')

    return model

我們的 function 用於評估 ANN 的性能

def evaluation(y_test_orig, y_test_pred):

    # Computing the Mean Absolute Percent Error
    MAPE = mean_absolute_percentage_error(y_test_orig, y_test_pred)

    # Computing R2 Score
    r2 = r2_score(y_test_orig, y_test_pred)

    # Computing Mean Square Error (MSE)
    MSE = mean_squared_error(y_test_orig, y_test_pred)

    # Computing Root Mean Square Error (RMSE)
    RMSE = mean_squared_error(y_test_orig, y_test_pred, squared=False)

    # Computing Mean Absolute Error (MAE)
    MAE = mean_absolute_error(y_test_orig, y_test_pred)

    # Computing Mean Bias Error (MBE)
    MBE = np.mean(y_test_pred - y_test_orig)  # here we calculate MBE

    print('--------------')

    print('The Coefficient of Determination (R2) of ANN model is:', r2)
    print("The Root Mean Squared Error (RMSE) of ANN model is:", RMSE)
    print("The Mean Squared Error (MSE) of ANN model is:", MSE)
    print('The Mean Absolute Percent Error (MAPE) of ANN model is:', MAPE)
    print("The Mean Absolute Error (MAE) of ANN model is:", MAE)
    print("The Mean Bias Error (MBE) of ANN model is:", MBE)

    print('--------------')

    eval_list = [r2, RMSE, MSE, MAPE, MAE, MBE]
columns = ['Coefficient of Determination (R2)', 'Root Mean Square Error (RMSE)', 'Mean Squared Error (MSE)',
           'Mean Absolute Percent Error (MAPE)', 'Mean Absolute Error (MAE)', 'Mean Bias Error (MBE)']

    dataframe = pd.DataFrame([eval_list], columns=columns)

    return dataframe

Answer 1

如果您更新make_regression_ann function 以包含您想要優化的任何超參數作為輸入，您的代碼應該可以工作，但擬合參數除外。

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import make_regression

def make_regression_ann(initializer='uniform', activation='relu', optimizer='adam', loss='mse'):

    model = Sequential()
    model.add(Dense(units=8, input_dim=7, kernel_initializer=initializer, activation=activation))
    model.add(Dense(units=6, kernel_initializer=initializer, activation=activation))
    model.add(Dense(1, kernel_initializer=initializer))
    model.compile(loss=loss, optimizer=optimizer)

    return model

param_grid = {
    'initializer': ['normal', 'uniform'],
    'activation': ['relu', 'sigmoid'],
    'optimizer': ['adam', 'rmsprop'],
    'loss': ['mse', 'mae'],
    'batch_size': [32, 64],
    'epochs': [5, 10],
}

grid_search = GridSearchCV(
    estimator=KerasRegressor(make_regression_ann, verbose=0),
    param_grid=param_grid,
    scoring='neg_mean_absolute_percentage_error',
    cv=3,
)

X, y = make_regression(n_features=7, n_samples=100, random_state=42)

grid_search.fit(X, y, verbose=1)

grid_search.best_params_
# {'activation': 'sigmoid',
#  'batch_size': 32,
#  'epochs': 10,
#  'initializer': 'normal',
#  'loss': 'mae',
#  'optimizer': 'adam'}

Answer 2

我最近成功使用 GridSearchCV 的方式是：

tuned_parameters2 = {'C': [10000], 'max_iter':[50000]}
model2 = GridSearchCV(svm.LinearSVC(), tuned_parameters2)
model2.fit(features, y_train)

因此，使用超參數分離字典，然后將您的 model 分配給 GridSearchCV(make_regression_ann, the_hyperparam_dict)。 然后用數據擬合它。

在您的情況下，這種方法需要更多的重構。 由您決定將 ANN 提供給 GridSearchCV 是否更好。