RuntimeWarning: overflow encountered in exp in computing the logistic function

Question

I'm getting this error when I try to compute the logistic function for a data mining method I'm implementing:

RuntimeWarning: overflow encountered in exp

My code:

def logistic_function(x):
#     x = np.float64(x)
    return 1.0 / (1.0 + np.exp(-x))

If I understood correctly from some related questions that the problem is that np.exp() is returning a huge value. I saw suggestions to let numpy ignore the warnings, but the problem is that when I get this error, then the results of my method are horrible. However when I don't get it, then they are as expected. So making numpy ignoring the warning is not a solution for me at all. I don't know what is wrong or how to deal with.

I don't even know if this is a result of a bug because sometimes I get this error and sometimes not! I went through my code many times and everything looks correct!

Answer 1

You should compute the logistic function using either scipy.special.expit , which in recent enough SciPy is more stable than your solution (although earlier versions got it wrong), or by reducing it to tanh :

def logistic_function(x):
    return .5 * (1 + np.tanh(.5 * x))

This version of the function is stable, fast, and fairly accurate.