计算损耗相对于层输入的偏导数| 连锁规则| 蟒蛇

Question

The task of this assignment is to calculate the partial derivative of the loss with respect to the input of the layer. You must implement the Chain Rule.

I am having a difficult time understanding conceptually how to set up the function. Any advice or tips would be appreciated!

The example data for the function variables are at the bottom.

 def dense_grad_input(x_input, grad_output, W, b): """Calculate the partial derivative of the loss with respect to the input of the layer # Arguments x_input: input of a dense layer - np.array of size `(n_objects, n_in)` grad_output: partial derivative of the loss functions with respect to the ouput of the dense layer np.array of size `(n_objects, n_out)` W: np.array of size `(n_in, n_out)` b: np.array of size `(n_out,)` # Output the partial derivative of the loss with respect to the input of the layer np.array of size `(n_objects, n_in)` """ ################# ### YOUR CODE ### ################# return grad_input 

 #x_input [[ 0.29682018 0.02620921 0.03910291 0.31660917 0.6809823 0.67731154 0.85846755 0.96218481 0.90590621 0.72424189 0.33797153 0.68878736 0.78965605 0.23509894 0.7241181 0.28966239 0.31927664 0.85477801] [ 0.9960161 0.4369152 0.89877488 0.78452364 0.22198744 0.04382131 0.4169376 0.69122887 0.25566736 0.44901459 0.50918353 0.8193029 0.29340534 0.46017931 0.64337706 0.63181193 0.81610792 0.45420877] [ 0.24633573 0.1358581 0.07556498 0.85105726 0.99732196 0.00668041 0.61558841 0.22549151 0.20417495 0.90856472 0.43778948 0.5179694 0.77824586 0.98535274 0.37334145 0.77306608 0.84054839 0.59580074] [ 0.68575595 0.48426868 0.17377837 0.5779052 0.7824412 0.14172426 0.93237195 0.71980057 0.04890449 0.35121393 0.67403124 0.71114348 0.32314314 0.84770232 0.10081962 0.27920494 0.52890886 0.64462433] [ 0.35874758 0.96694283 0.374106 0.40640907 0.59441666 0.04155628 0.57434682 0.43011294 0.55868019 0.59398029 0.22563919 0.39157997 0.31804255 0.63898075 0.32462043 0.95516196 0.40595824 0.24739606]] #grad_output [[ 0.30650667 0.66195042 0.32518952 0.68266843 0.16748198] [ 0.87112224 0.66131922 0.03093839 0.61508666 0.21811778] [ 0.95191614 0.70929627 0.42584023 0.59418774 0.75341628] [ 0.32523626 0.90275084 0.3625107 0.52354435 0.23991962] [ 0.89248732 0.55744782 0.02718998 0.82430586 0.73937504]] #W [[ 0.8584596 0.28496554 0.6743653 0.81776177 0.28957213] [ 0.96371309 0.19263171 0.78160551 0.07797744 0.21341943] [ 0.5191679 0.02631223 0.37672431 0.7439749 0.53042904] [ 0.1472284 0.46261313 0.18701797 0.17023813 0.63925535] [ 0.6169004 0.43381192 0.93162705 0.62511267 0.45877614] [ 0.30612274 0.39457724 0.26087929 0.34826782 0.71235394] [ 0.66890267 0.70557853 0.48098531 0.76937604 0.10892615] [ 0.17080091 0.57693496 0.19482135 0.07942299 0.7505965 ] [ 0.61697062 0.1725569 0.21757211 0.64178749 0.41287085] [ 0.96790726 0.22636129 0.38378524 0.02240361 0.08083711] [ 0.67933 0.34274892 0.55247312 0.06602492 0.75212193] [ 0.00522951 0.49808998 0.83214543 0.46631055 0.48400103] [ 0.56771735 0.70766078 0.27010417 0.73044053 0.80382 ] [ 0.12586939 0.18685427 0.66328521 0.84542463 0.7792 ] [ 0.21744701 0.90146876 0.67373118 0.88915982 0.5605676 ] [ 0.71208837 0.89978603 0.34720491 0.79784756 0.73914921] [ 0.48384807 0.10921725 0.81603026 0.82053322 0.45465871] [ 0.56148353 0.31003923 0.39570321 0.7816182 0.23360955]] #b [ 0.10006862 0.36418521 0.56036054 0.32046732 0.57004243] 

Answer 1

It's as simple as

def dense_grad_input(x_input, grad_output, W, b):
  return grad_output.dot(W.T)

The formula is the backpropagation error signal through the matrix multiplication. Derivation can be found here . Note that it doesn't depend on the loss function, only on the error signal from the following layer grad_output .

Also note that backpropagation also requires to find the gradient with respect to W and b .