[英]Learning Rate Decay in Lasagne
我正在使用Nesterov動量來更新卷積神經網絡的權重。 我正在使用Lasagne構建CNN。 如何實現每個時期的學習率衰減?
prediction = lasagne.layers.get_output(network)
loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)
params = lasagne.layers.get_all_params(network, trainable=True)
lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
train_fn = theano.function([input_var_1, input_var_2, target_var], loss, updates=updates)
將學習率定義為圖形元素
...
learning_rate = T.scalar(name='learning_rate')
updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
...
將您的學習率節點添加到train_fn
train_fn = theano.function([input_var_1, input_var_2, target_var, learning_rate], loss, updates=updates)
現在,您可以調整學習率,例如:
base_lr = 1e-4
lr_decay = 0.8
for epoch in range(epochs):
lr = base_lr * (lr_decay ** epoch)
loss, updates = train_fn(input_1, input_2, target, lr)
我認為您缺少了一些東西:
prediction = lasagne.layers.get_output(network)
loss = lasagne.objectives.categorical_crossentropy(prediction, target_var)
params = lasagne.layers.get_all_params(network, trainable=True)
updates = lasagne.updates.nesterov_momentum(loss, params, learning_rate, momentum=0.9)
train_fn = theano.function([input_var_1, input_var_2, target_var], loss, updates=updates)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.