Keras Tuner - 首席運行試驗而不是工人

Question

設置
keras 調諧器==1.1.0
張量流==2.8.0
Python 3.10.2

Chief 和 Tuner0 在一台機器上運行
Tuner1 在另一台機器上運行

超頻初始化：

hp = Hyperband(
    hypermodel=em.get_model,
    objective='val_accuracy',
    max_epochs=int(config.get(eid, 'epochs')),
    project_name=project_folder,
    hyperband_iterations=int(config.get(eid, 'tuner_iterations'))
)

print(hp.search_space_summary())

# TensorBoard logs
# tlogs = 'tboard_logs/' + eid

lr_schedule = LearningRateScheduler(exp_scheduler)
early_stop = int(config.get(eid, 'early_stop'))
if len(output_keys) > 1:
    hp.search(train, steps_per_epoch=train_steps,
              validation_data=test, validation_steps=test_steps, verbose=2,
              callbacks=[EarlyStopping(patience=early_stop), lr_schedule, Combined_Accuracy(len(output_keys))])
else:
    hp.search(train, steps_per_epoch=train_steps,
              validation_data=test, validation_steps=test_steps, verbose=2,
              callbacks=[EarlyStopping(patience=early_stop), lr_schedule])

問題：
Tuner0 和 Tuner1 完成搜索后，酋長開始運行試驗。 理想情況下，主管應該只為工人進行的試驗提供變量。 另外，因為我已經限制酋長只能在 CPU 上運行，所以速度很慢。 以下是主要腳本的日志：

Oracle server on chief is exiting in 10s.The chief will go on with post-search code.
Search space summary
Default search space size: 18
enc_dropout (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.4, 'step': None, 'sampling': None}
enc_layer_norm (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.6, 'step': None, 'sampling': None}
enc_l2_reg (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.6, 'step': None, 'sampling': None}
pos_dropout (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.4, 'step': None, 'sampling': None}
pos_layer_norm (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.6, 'step': None, 'sampling': None}
pos_l2_reg (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.6, 'step': None, 'sampling': None}
decoder_dropout (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.4, 'step': None, 'sampling': None}
decoder_layer_norm (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.6, 'step': None, 'sampling': None}
decoder_l2_reg (Float)
{'default': 0.0, 'conditions': [], 'min_value': 0.0, 'max_value': 0.6, 'step': None, 'sampling': None}
learning_rate (Float)
{'default': 1e-05, 'conditions': [], 'min_value': 1e-05, 'max_value': 9e-05, 'step': None, 'sampling': None}
enc_dense_stack (Choice)
{'default': 2, 'conditions': [], 'values': [2, 3, 4], 'ordered': True}
bert_url (Choice)
{'default': 'https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-8_H-256_A-4/2', 'conditions': [], 'values': ['https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-8_H-256_A-4/2'], 'ordered': False}
pos_enc_blocks (Choice)
{'default': 2, 'conditions': [], 'values': [2, 3, 4], 'ordered': True}
pos_attn_heads (Choice)
{'default': 2, 'conditions': [], 'values': [2, 3, 4], 'ordered': True}
pos_dense_stack (Choice)
{'default': 2, 'conditions': [], 'values': [2, 3, 4], 'ordered': True}
decoder_enc_blocks (Choice)
{'default': 2, 'conditions': [], 'values': [2, 3, 4], 'ordered': True}
decoder_attn_heads (Choice)
{'default': 2, 'conditions': [], 'values': [2, 3, 4], 'ordered': True}
decoder_dense_stack (Choice)
{'default': 2, 'conditions': [], 'values': [2, 3, 4], 'ordered': True}
None

Search: Running Trial #218

Hyperparameter    |Value             |Best Value So Far 
enc_dropout       |0.37332           |0.10642           
enc_layer_norm    |0.15571           |0.12288           
enc_l2_reg        |0.48613           |0.57864           
pos_dropout       |0.17162           |0.14473           
pos_layer_norm    |0.11009           |0.26961           
pos_l2_reg        |0.49191           |0.20803           
decoder_dropout   |0.24864           |0.051037          
decoder_layer_norm|0.46016           |0.57878           
decoder_l2_reg    |0.41414           |0.013985          
learning_rate     |7.8417e-05        |6.716e-05         
enc_dense_stack   |4                 |3                 
bert_url          |https://tfhub.d...|https://tfhub.d...
pos_enc_blocks    |2                 |4                 
pos_attn_heads    |4                 |4                 
pos_dense_stack   |2                 |4                 
decoder_enc_blocks|2                 |3                 
decoder_attn_heads|2                 |3                 
decoder_dense_s...|2                 |2                 
tuner/epochs      |50                |50                
tuner/initial_e...|0                 |17                
tuner/bracket     |0                 |2                 
tuner/round       |0                 |2                 

Epoch 1/50
85/85 - 215s - loss: 149.9310 - accuracy: 0.8909 - val_loss: 103.2796 - val_accuracy: 0.9896 - lr: 6.4203e-05 - 215s/epoch - 3s/step
Epoch 2/50
85/85 - 220s - loss: 94.1549 - accuracy: 0.9897 - val_loss: 83.6212 - val_accuracy: 0.9896 - lr: 6.4203e-05 - 220s/epoch - 3s/step
Epoch 3/50
85/85 - 210s - loss: 75.2738 - accuracy: 0.9897 - val_loss: 67.1717 - val_accuracy: 0.9896 - lr: 6.4203e-05 - 210s/epoch - 2s/step
Epoch 4/50
85/85 - 190s - loss: 60.2264 - accuracy: 0.9898 - val_loss: 53.5418 - val_accuracy: 0.9896 - lr: 6.4203e-05 - 190s/epoch - 2s/step

Answer 1

根據Keras Tuner - Distributed Tuning ，您應該將 Distributed_strategy 參數添加到 Hyperband 構造函數。