使用手動度量與Hyperas進行Keras超參數調整

Question

我正在使用hyperas文檔示例來調整網絡參數，但基於f1得分而不是准確性。

我將以下實現用於f1得分：

from keras import backend as K

def f1(y_true, y_pred):
    def recall(y_true, y_pred):
        """Recall metric.
        Only computes a batch-wise average of recall.
        Computes the recall, a metric for multi-label classification of
        how many relevant items are selected.
        """
        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))
        recall = true_positives / (possible_positives + K.epsilon())
        return recall

    def precision(y_true, y_pred):
        """Precision metric.
        Only computes a batch-wise average of precision.
        Computes the precision, a metric for multi-label classification of
        how many selected items are relevant.
        """
        true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))
        predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))
        precision = true_positives / (predicted_positives + K.epsilon())
        return precision
    precision = precision(y_true, y_pred)
    recall = recall(y_true, y_pred)
    return 2*((precision*recall)/(precision+recall+K.epsilon()))

在以下代碼行中更新用於編譯功能的度量參數：

model.compile(loss='categorical_crossentropy', metrics=['accuracy'],
                  optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})

至

model.compile(loss='categorical_crossentropy', metrics=[f1],
                  optimizer={{choice(['rmsprop', 'adam', 'sgd'])}})

上面的指標在不使用hyperas的情況下可以完美運行，而當我嘗試在調整過程中使用它時，出現以下錯誤：

Traceback (most recent call last):
  File "D:/path/test.py", line 96, in <module>
    trials=Trials())
  File "C:\Python35\lib\site-packages\hyperas\optim.py", line 67, in minimize
    verbose=verbose)
  File "C:\Python35\lib\site-packages\hyperas\optim.py", line 133, in base_minimizer
    return_argmin=True),
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 367, in fmin
    return_argmin=return_argmin,
  File "C:\Python35\lib\site-packages\hyperopt\base.py", line 635, in fmin
    return_argmin=return_argmin)
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 385, in fmin
    rval.exhaust()
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 244, in exhaust
    self.run(self.max_evals - n_done, block_until_done=self.asynchronous)
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 218, in run
    self.serial_evaluate()
  File "C:\Python35\lib\site-packages\hyperopt\fmin.py", line 137, in serial_evaluate
    result = self.domain.evaluate(spec, ctrl)
  File "C:\Python35\lib\site-packages\hyperopt\base.py", line 840, in evaluate
    rval = self.fn(pyll_rval)
  File "D:\path\temp_model.py", line 86, in keras_fmin_fnct
NameError: name 'f1' is not defined

Answer 1

如果您正在遵循鏈接到的代碼示例，則不會使hyperas意識到自定義f1函數。 包作者也提供了一個示例來執行此操作 。

簡而言之，您需要在optim.minimize()調用中添加一個附加的functions參數。 就像是

    best_run, best_model = optim.minimize(model=model,
     data=data,
     functions=[f1],
     algo=tpe.suggest,
     max_evals=5,
     trials=Trials())

我實際上只是在今天實現了它，所以我相信您也可以使它起作用:)