简体   繁体   中英

Logestic regression model is not learning

I wrote logistic regression algorithm using data with 9 attributes and one vector of labels, but it is not training.
I think I have to transpose some of the inputs when updating the weights but not sure, tried a bit of trial and error but no luck.
If anyone can help thanks.

class logistic_regression(neural_network):
    def __init__(self,data):

        self.data = data   # to store the the data location in a varable 
        self.data1 = load_data(self.data) # load the data 
        self.weights =  np.random.normal(0,1,self.data1.shape[1] -1)   # use the number of attributes to get the number of weights
        self.bias = np.random.randn(1) #  set the bias to a random number 
        self.x = self.data1.iloc[:,0:9] # split the xs and ys
        self.y = self.data1.iloc[:,9:10]
        self.x = np.array(self.x)
        self.y = np.array(self.y)

        print(self.weights)
        print(np.dot(self.x[0].T,self.weights))
    def load_data(self,file):
        data = pd.read_csv(file)
        return data
    def sigmoid(self,x): # acivation function to limit the value to 0 and 1
        return 1 / (1 + np.exp(-x))
    def sigmoid_prime(self,x):
        return self.sigmoid(x) * (1 - self.sigmoid(x))
    def train(self):
        error = 0 # init the error to zero
        learning_rate = 0.01
        for interation in range(100):
            for i in range(len(self.x)): # loop though all the data
                pred = np.dot(self.x[i].T,self.weights) + self.bias # calculate the output
                pred1 = self.sigmoid(pred)
                error = (pred1 - self.y[i])**2 # check the accuracy of the network

                self.bias -= learning_rate * pred1 - self.y[i] * self.sigmoid_prime(pred1)
                self.weights -= learning_rate * (pred1 - self.y[i]) * self.sigmoid_prime(pred1) *  self.x[i]

            print(str(pred1)+"pred")
            print(str(error) + "error")  # print the result
            print(pred1[0] - self.y[i][0])
    def test(self):

You cannot train any machine learning model using only one label. The resulting model will only have one response, no matter what test data is being used - the label provided while training.

Broken derivatives

You've got a bug in the self.bias adjustment, missing parenthesis around pred1-self.y[i].

Also, you're calculating the derivative from the wrong variable - it seems that instead of self.sigmoid_prime(pred1) you'd need self.sigmoid_prime(pred).

Test on a toy example

For any such code, I'd suggest that you first test it on a very simple function one where it's trivial to print out all the intermediate values and verify them on paper. For example, boolean AND and OR functions. That will show you whether you've got the update formulas correct, isolating the learning code from the peculiarities of your actual learning task.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM