[英]SciKeras - RandomizedSearchCV for best hyper-parameters
I'm trying to follow the example on chapter 10 of the book Hands-On Machine Learning with SciKit-Learn, Keras and TensorFlow which regard the optimization of the hyperparameters of a DNN model.我正在尝试遵循使用 SciKit-Learn、Keras 和 TensorFlow 进行机器学习实践一书第 10 章的示例,该示例涉及 DNN 模型的超参数的优化。
The dataset is the MNIST fashion model and the goal of the project is the classification of the images in 10 classes.数据集是 MNIST 时装模型,该项目的目标是对 10 个类别的图像进行分类。
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()
There is no validation, so I'm going to create that set using the first 5k elements:没有验证,所以我将使用前 5k 个元素创建该集合:
X_valid, X_train = X_train_full[:5000], X_train_full[5000:]
y_valid, y_train = y_train_full[:5000], y_train_full[5000:]
A possible implementation of a simple DNN model is the following, using sequential Keras API:一个简单的 DNN 模型的可能实现如下,使用顺序 Keras API:
model = keras.models.Sequential()
model.add(keras.layers.Flatten(input_shape=[28, 28]))
model.add(keras.layers.Dense(300, activation="relu"))
model.add(keras.layers.Dense(200, activation="relu"))
model.add(keras.layers.Dense(100, activation="relu"))
model.add(keras.layers.Dense(10, activation="softmax"))
The book then suggests to study the hyper-parameter space to found the best ones, using RandomizedSearchCV
.然后,本书建议使用
RandomizedSearchCV
研究超参数空间以找到最好的空间。 The example uses keras.wrappers.scikit_learn.KerasRegressor
which is now deprecated in favor of KerasRegressor
by SciKeras.该示例使用
keras.wrappers.scikit_learn.KerasRegressor
现在已被 SciKeras 弃用,取而代之的是KerasRegressor
。
I created a function containing the ML model:我创建了一个包含 ML 模型的函数:
input_shape=X_train[0].shape
def build_model(n_hidden = 1, n_neurons = 30, learning_rate=3e-3, input_shape=input_shape):
grid_model = keras.models.Sequential()
grid_model.add(keras.layers.Flatten(input_shape=input_shape))
for layer in range(n_hidden):
grid_model.add(keras.layers.Dense(n_neurons, activation="relu"))
grid_model.add(keras.layers.Dense(10, activation="softmax"))
opt = tf.keras.optimizers.SGD(learning_rate=learning_rate)
grid_model.compile(loss="sparse_categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
return grid_model
Then I defined the model hyperparameters to explore:然后我定义了模型超参数来探索:
param_distribs = {
"n_hidden": [0, 1, 2, 3, 4, 5],
"n_neurons": np.arange(1, 300),
"learning_rate": reciprocal(3e-4, 3e-2)
}
I then used SciKeras to create a wrapper around the Keras model, feeding the parameter space:然后我使用 SciKeras 围绕 Keras 模型创建一个包装器,提供参数空间:
keras_reg = KerasRegressor(build_model, n_hidden=param_distribs["n_hidden"], n_neurons=param_distribs["n_neurons"], learning_rate=param_distribs["learning_rate"], verbose=1)
The last step is to define a RandomizedSearchCV
object and start the research using the fit
method:最后一步是定义一个
RandomizedSearchCV
对象并使用fit
方法开始研究:
rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=10, cv=3)
rnd_search_cv.fit(X_train, y_train, epochs=100, validation_data=(X_valid, y_valid), callbacks=[keras.callbacks.EarlyStopping(patience=10)])
This last row gives me the following error, for each epoch:对于每个时代,最后一行给了我以下错误:
/home/docker_user/.local/lib/python3.8/site-packages/sklearn/model_selection/_validation.py:776: UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan. Details:
Traceback (most recent call last):
File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/model_selection/_validation.py", line 767, in _score
scores = scorer(estimator, X_test, y_test)
File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/metrics/_scorer.py", line 429, in _passthrough_scorer
return estimator.score(*args, **kwargs)
File "/home/docker_user/.local/lib/python3.8/site-packages/scikeras/wrappers.py", line 1100, in score
return self.scorer(y, y_pred, sample_weight=sample_weight, **score_args)
File "/home/docker_user/.local/lib/python3.8/site-packages/scikeras/wrappers.py", line 1697, in scorer
return sklearn_r2_score(y_true, y_pred, **kwargs)
File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/metrics/_regression.py", line 911, in r2_score
y_type, y_true, y_pred, multioutput = _check_reg_targets(
File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/metrics/_regression.py", line 100, in _check_reg_targets
check_consistent_length(y_true, y_pred)
File "/home/docker_user/.local/lib/python3.8/site-packages/sklearn/utils/validation.py", line 387, in check_consistent_length
raise ValueError(
ValueError: Found input variables with inconsistent numbers of samples: [18334, 183340]
A factor 10 on the second dimension makes me thinking... I also checked the shapes of the data and they are fine...第二维的因子 10 让我思考......我还检查了数据的形状,它们很好......
print(X_train.shape, y_train.shape)
(55000, 28, 28) (55000,)
Can you please help me dealing with this error?你能帮我处理这个错误吗?
As the name suggests, scikeras.wrappers.KerasRregressor
should be employed for regression and uses the r2_score
by default.顾名思义,
scikeras.wrappers.KerasRregressor
应该用于回归并默认使用r2_score
。 Hence, it will not convert the softmax output to a class index like the ones you have in your validation set.因此,它不会将 softmax 输出转换为您在验证集中拥有的类索引。 The last dense layer's output is of size
10
(hence the factor of 10).最后一个密集层的输出大小为
10
(因此因子为 10)。 Instead, you should use scikeras.wrappers.KerasClassifier
that uses accuracy as scoring by default.相反,您应该使用
scikeras.wrappers.KerasClassifier
,它默认使用准确性作为评分。 Simply swapping KerasRegressor
out for KerasClassifier
should make it work.只需将
KerasRegressor
KerasClassifier
就可以了。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.