I wrote logistic regression algorithm using data with 9 attributes and one vector of labels, but it is not training.
I think I have to transpose some of the inputs when updating the weights but not sure, tried a bit of trial and error but no luck.
If anyone can help thanks.
class logistic_regression(neural_network):
def __init__(self,data):
self.data = data # to store the the data location in a varable
self.data1 = load_data(self.data) # load the data
self.weights = np.random.normal(0,1,self.data1.shape[1] -1) # use the number of attributes to get the number of weights
self.bias = np.random.randn(1) # set the bias to a random number
self.x = self.data1.iloc[:,0:9] # split the xs and ys
self.y = self.data1.iloc[:,9:10]
self.x = np.array(self.x)
self.y = np.array(self.y)
print(self.weights)
print(np.dot(self.x[0].T,self.weights))
def load_data(self,file):
data = pd.read_csv(file)
return data
def sigmoid(self,x): # acivation function to limit the value to 0 and 1
return 1 / (1 + np.exp(-x))
def sigmoid_prime(self,x):
return self.sigmoid(x) * (1 - self.sigmoid(x))
def train(self):
error = 0 # init the error to zero
learning_rate = 0.01
for interation in range(100):
for i in range(len(self.x)): # loop though all the data
pred = np.dot(self.x[i].T,self.weights) + self.bias # calculate the output
pred1 = self.sigmoid(pred)
error = (pred1 - self.y[i])**2 # check the accuracy of the network
self.bias -= learning_rate * pred1 - self.y[i] * self.sigmoid_prime(pred1)
self.weights -= learning_rate * (pred1 - self.y[i]) * self.sigmoid_prime(pred1) * self.x[i]
print(str(pred1)+"pred")
print(str(error) + "error") # print the result
print(pred1[0] - self.y[i][0])
def test(self):
You cannot train any machine learning model using only one label. The resulting model will only have one response, no matter what test data is being used - the label provided while training.
You've got a bug in the self.bias adjustment, missing parenthesis around pred1-self.y[i].
Also, you're calculating the derivative from the wrong variable - it seems that instead of self.sigmoid_prime(pred1) you'd need self.sigmoid_prime(pred).
For any such code, I'd suggest that you first test it on a very simple function one where it's trivial to print out all the intermediate values and verify them on paper. For example, boolean AND and OR functions. That will show you whether you've got the update formulas correct, isolating the learning code from the peculiarities of your actual learning task.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.