In sklearn logisticRegression, what is the Fit_intercept=False MEANING?

Question

I tried to compare logistic regression result from statsmodel with sklearn logisticRegression result. actually I tried to compare with R result also. I made the options C=1e6(no penalty) but I got almost same coefficients except the intercept.

model = sm.Logit(Y, X).fit()
print(model.summary())

==> intercept = 5.4020

model = LogisticRegression(C=1e6,fit_intercept=False)
model = model.fit(X, Y)

===> intercept = 2.4508

so I read the user guide, they said Specifies if a constant (aka bias or intercept) should be added to the decision function. what is this meaning? due to this, sklearn logisticRegression gave a different intercept value?

please help me

Answer 1

LogisticRegression is in some aspects similar to the Perceptron Model and LinearRegression. You multiply your weights with the data points and compare it to a threshold value b :

w_1 * x_1 + ... + w_n*x_n > b

This can be rewritten as:

-b + w_1 * x_1 + ... + w_n*x_n > 0  

or

 w_0 * 1 + w_1 * x_1 + ... + w_n*x_n > 0

For linear regression we keep this, for the perceptron we feed this to a chosen function and here for the logistic regression pass this to the logistic function.

Instead of learning n parameters now n+1 are learned. For the perceptron it is called bias, for regression intercept.

For linear regression it's easy to understand geometrically. In the 2D case you can think about this as a shifting the decision boundary by w_0 in the y direction**.
or y = m*x vs y = m*x + c So now the decision boundary does not go through (0,0) anymore.

For the logistic function it is similar it shifts it away for the origin.

Implementation wise what happens, you add one more weight and a constant 1 to the X values. And then you proceed as normal.

if fit_intercept:
   intercept = np.ones((X_train.shape[0], 1))
   X_train   = np.hstack((intercept, X_train))
   weights   = np.zeros(X_train.shape[1])