[英]How to calculate bias during batch training of a Perceptron
我正在對單節點感知器實施批量訓練,但不知道如何更新我的偏差。
我正在更新重量如下:
對於每個批次,我在單個批次中運行以下臨時重量更新
# update weight_update where y[i] is the actual label and o1 is the predicted output and x[i] is the input
weight_update = weight_update + (self.weights + self.learning_rate * (y[i] - o1)*x[i])
然后一個完整的批次完成我更新我的 class 中的主要重量
# update main weights (self.weights) where len(x) is the number of samples
self.weights = self.weights + (weight_update / len(x))
我假設 (y[i] - o1)*x[i]) 是損失 function 重量的偏導數,我不確定你使用了什么,但假設你使用了 -1/2 * (y[i] - o1)^2
now let o1 = wx + b, where w is weight matrix and b is bias vector,
also let, L = -1/2 * (y[i] - o1)^2
you have already calculated dLdw = dLd(o1) * d(o1)dw = (y[i] - o1) * x
In a summilar way calculate dLdb,
dLdb = dLd(o1) * d(o1)db
dLd(o1) = (y[i] - o1)
d(o1)db = d/db (wx + b) = 0 + 1 = 1
so dLdb = (y[i] - o1) * 1
現在這條線,
weight_update = weight_update + (self.weights + self.learning_rate * (y[i] - o1)*x[i])
此時無需添加權重,只需添加漸變即可
weight_update += self.learning_rate * dLdw
# similarily
bias_update += self.learning_rate * dLdb
當一批完成后,就做
# update main weights (self.weights) and biases (self.biases)
# where len(x) is the number of samples
self.weights += (weight_update / len(x))
self.biases+= (bias_update / len(x))
# dont forget to set the values of weight_update, bias_update to 0
這是我幾天前寫的(MNIST 示例的 1 個隱藏分層網絡),您可能會發現這很有用
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.