[英]ReLU derivative with NumPy
import numpy as np
def relu(z):
return np.maximum(0,z)
def d_relu(z):
z[z>0]=1
z[z<=0]=0
return z
x=np.array([5,1,-4,0])
y=relu(x)
z=d_relu(y)
print("y = {}".format(y))
print("z = {}".format(z))
The code above prints out: 上面的代码输出:
y = [1 1 0 0]
z = [1 1 0 0]
instead of 代替
y = [5 1 0 0]
z = [1 1 0 0]
From what I understand the function calls I've used should only be doing passing by value,passing a copy of the variable. 据我了解,我使用过的函数调用只应该按值传递,传递变量的副本。
Why is my d_relu function affecting the y variable? 为什么我的d_relu函数会影响y变量?
Your first mistake is in assuming python passes objects by value... it doesn't - it's pass by assignment (similar to passing by reference, if you're familiar with this concept). 您的第一个错误是假设python按值传递对象...否-而是按赋值传递(如果您熟悉此概念,则类似于按引用传递)。 However, only mutable objects, as the name suggests, can be modified in-place.
但是,仅顾名思义,可变对象可以就地修改。 This includes, among other things, numpy arrays.
其中包括numpy数组。
You shouldn't have d_relu
modify z
inplace, because that's what it's doing right now, through the z[...] = ...
syntax. 您不应该
d_relu
就地修改z
,因为这就是现在通过z[...] = ...
语法所做的。 Try instead building a mask using broadcasted comparison and returning that instead. 尝试使用广播比较构建遮罩,然后返回该遮罩。
def d_relu(z):
return (z > 0).astype(int)
This returns a fresh array instead of modifying z
in-place, and your code prints 这将返回一个新数组,而不是就地修改
z
,并且您的代码会打印
y = [5 1 0 0]
z = [1 1 0 0]
If you're building a layered architecture, you can leverage the use of a computed mask during the forward pass stage: 如果您要构建分层体系结构,则可以在前向通过阶段利用计算出的蒙版:
class relu:
def __init__(self):
self.mask = None
def forward(self, x):
self.mask = x > 0
return x * self.mask
def backward(self, x):
return self.mask
Where the derivative is simply 1 if the input during feedforward if > 0, else 0. 如果前馈期间的输入> 0,则导数简单为1,否则为0。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.