Python 3中的渐变下降

Question

I have a code Gradient Descent in Python 3 and error appear I do not know how I can solve it ...

and I have a question why the result of the dataset give me a zero for accuracy anyone can help me, please :(

I try to change some things but no benefit.

I divide my dataset into two parts for the training set and for the test set

 import csv import numpy as np import matplotlib.pyplot as plt N_EPOCHS = 10 LEARNING_RATE = .1 PLOT = False LAMBDA = .00000001 def load_data(filename): X = [] Y = [] with open(filename, 'r') as csvfile: X = [[float(x) for x in line] for line in csv.reader(csvfile, delimiter=',')] for line in X: Y.append([line[-1]]) line.pop() print(X[0]) print(Y) X = np.array(X, dtype=np.longdouble) Y = np.array(Y, dtype=np.longdouble) return X, Y def sigmoid(weight_param, x_param): denom_sigmoid = np.longdouble(1 + np.exp(np.dot(-weight_param, x_param))) sig = np.longdouble(np.divide(1, denom_sigmoid, where=denom_sigmoid!=0.0)) return sig def gradient_descent(X, Y, L2_Regularization=False): example_accuracy = [] X = np.c_[np.ones((X.shape[0])), X] # Add bias of 1 to each example feature_len = X.shape[1] example_count = np.long(X.shape[0]) print("X.shape ", X.shape) # Random weight vector with shape equal to number of features w = np.zeros(feature_len) l2_reg = 0 step = 0 correct_count = 0 while(step < N_EPOCHS): print("Iteration: ", step) grad = np.zeros(feature_len, dtype=np.float) for example in range(example_count): # y_hat is the predicted output y_hat = sigmoid(wT, X[example]) if L2_Regularization: l2_reg = LAMBDA * w # = d/dw(.5*lambda*||w^2||) if y_hat >= .5: y_hat = 1 loss = y_hat - Y[example] if loss[0] == 0: correct_count += 1 print(correct_count) grad += loss[0] * X[example] + l2_reg w += -LEARNING_RATE * grad step += 1 example_accuracy.append(np.float(correct_count / example_count)) correct_count = 0 print(" Accuracy per Epoch: ", example_accuracy) return w, example_accuracy def main(): X, Y = load_data("/Users/mahaalmotiri/PycharmProjects/desktop/GradientDescent /data.csv") X_test, Y_test = load_data("/Users/mahaalmotiri/PycharmProjects/desktop/GradientDescent /data2.csv") w, example_accuracy = gradient_descent(X, Y) epoch_list = [epoch for epoch in range(N_EPOCHS)] if PLOT: plt.plot(epoch_list, example_accuracy) plt.xlabel('Epoch') plt.ylabel('Accuracy') plt.show() w_L2_train, example_accuracy_L2_train = gradient_descent(X, Y, L2_Regularization=True) w_L2_test, example_accuracy_L2_test = gradient_descent(X_test, Y_test, L2_Regularization=True) print("Example accuracy no L2_Regularization: ", example_accuracy) print("example_accuracy_L2_train: ", example_accuracy_L2_train) print("example_accuracy_L2_test: ", example_accuracy_L2_test) if __name__ == "__main__": main() 

error is

Answer 1

This warning means that degree np.dot(-weight_param, x_param) is too big here:

denom_sigmoid = np.longdouble(1 + np.exp(np.dot(-weight_param, x_param)))

so you get large number after computing exponent in the degree.

If you want to get rid of this you just need to make it smaller. Maybe you want to normalize you input data using sklearn .