Keras：如何创建自定义的Noisy Relu函数？

Question

How could I create a noisy Relu function in Keras? Especially how do I create the noise Y~N(0,1).

def relu_noise(x):
return x*(x>0) + N(0,1)

Any ideas? Thanks!

Answer 1

You can use a Lambda layer for that task.

Define a function normally, but using the keras backend functions:

def relu_noise(x):

    isPositive = K.greater(x,0) 
    noise = K.random_normal((shape of x), mean=0.5, stddev=0.5)
         #I'm just not sure this is exactly the kind of noise you want. 

    return (x * isPositive) + noise

Then use it in a lambda layer:

from keras.layers import *

layer = Lambda(relu_noise, output_shape=(shape of x))

Add this layer to a Sequential model as any other layer, or call it with an input in a Model .

You can probably use it directly as an activation function as well:

layer = Dense(units, activation=relu_noise)