Error using 'selu' activation function with Keras

Question

I'm using Keras with Tensorflow backend. When I'm trying to use the 'selu' activation function using:

model.add(Dense(32, input_shape=(input_length - 1,)))
model.add(Activation('selu'))

The error I get is:

ValueError: Unknown activation function:selu

Is there any solution to this?

Answer 1

Selu is not in your activations.py of keras (most likely because it was added Jun 14, 2017, only 22 days ago). You can just add the missing code in the activations.py file or create your own selu activation in the script.

Example code

from keras.activations import elu

def selu(x):
    """Scaled Exponential Linear Unit. (Klambauer et al., 2017)
    # Arguments
        x: A tensor or variable to compute the activation function for.
    # References
        - [Self-Normalizing Neural Networks](https://arxiv.org/abs/1706.02515)
    """
    alpha = 1.6732632423543772848170429916717
    scale = 1.0507009873554804934193349852946
    return scale * elu(x, alpha)

model.add(Dense(32, input_shape=(input_length - 1,)), activation=selu)

---

NOTE:

With tensorflow 2.0 keras is included. You can get the selu activation with:

from tensorflow.keras.activations import selu