keras custom sigmoid adding bias

Question

There is a need to add bias to my custom sigmod function and apply this as a last activation layer in NN. But my recall goes rightly into 1. That shows me that something is wrong with the formula.

Custom sigmoid function

Recall goes strictly into 1

def custom_sigmoid(x):
    return 1 / (1 + K.exp(-20*x - 0.5))

At the same time, custom sigmoid without multiplier and bias works great.

def custom_sigmoid(x):
    return 1 / (1 + K.exp(x))

as can be seen here

self.model_.add(keras.layers.Dense(1, activation=custom_sigmoid))
self.model_.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=[precision_threshold(0.7), recall_threshold(0.7)])

How to modify the custom sigmoid function to make it work?

Answer 1

Your formula has no apparent problem, but it's likely to cause arithmetic overflow for -20*x - 0.5 , can you check the range of x . For example, if x is in [-100, 100] , the original sigmoid won't overflow while your customized sigmoid will. You can do a simple experiment in numpy:

import numpy as np

def original_sigmoid(x):
    return 1 / (1 + np.exp(x))


def custom_sigmoid(x):
    return 1 / (1 + np.exp(-20 * x - 0.5))


x = np.linspace(-100, 100)
print(original_sigmoid(x)) 
print(custom_sigmoid(x)) # this will output a warning: "RuntimeWarning: overflow encountered in exp"