[英]Plotting decision boundary in logistic regression
I am running logistic regression on a small dataset which looks like this:我在一个看起来像这样的小数据集上运行逻辑回归:
After implementing gradient descent and the cost function, I am getting a 89% accuracy in the prediction stage, However I want to be sure that everything is in order so I am trying to plot the decision boundary line which separates the two datasets.在实施梯度下降和成本 function 之后,我在预测阶段获得了 89% 的准确率,但是我想确保一切正常,所以我试图 plot 分隔两个数据集的决策边界线。
Below I present plots showing the cost function and theta parameters.下面我展示了显示成本 function 和 theta 参数的图表。 As can be seen, currently I am printing the decision boundary line incorrectly.
可以看出,目前我正在错误地打印决策边界线。
When I am zooming out the decision boundary graph I can see the following:当我缩小决策边界图时,我可以看到以下内容:
My decision boundary is being plotted below the dataset.我的决策边界绘制在数据集下方。 One thing to note is that I have used feature scaling.
需要注意的一件事是我使用了特征缩放。
Below are the code used by me:以下是我使用的代码:
Main Program主程序
%% Initialization
clear ; close all; clc
%% Load Data
% The first two columns contains the exam scores and the third column
% contains the label.
data = load('ex2data1.txt');
X = data(:, [1, 2]); y = data(:, 3);
%% ==================== Part 1: Plotting ====================
% We start the exercise by first plotting the data to understand the
% the problem we are working with.
fprintf(['Plotting data with + indicating (y = 1) examples and o ' ...
'indicating (y = 0) examples.\n']);
plotData(X, y);
% Put some labels
hold on;
% Labels and Legend
xlabel('Exam 1 score')
ylabel('Exam 2 score')
% Specified in plot order
legend('Admitted', 'Not admitted')
hold off;
fprintf('\nProgram paused. Press enter to continue.\n');
pause;
%% ============ Part 2: Compute Cost and Gradient ============
% In this part of the exercise, you will implement the cost and gradient
% for logistic regression. You neeed to complete the code in
% costFunction.m
% Setup the data matrix appropriately, and add ones for the intercept term
[m, n] = size(X);
%Normalize Feature
[X_norm mu sigma] = featureNormalize(X);
% Add intercept term to x and X_test
X = [ones(m, 1) X];
X_norm = [ones(m, 1) X_norm];
% Initialize fitting parameters
initial_theta = zeros(n + 1, 1);
% Compute and display initial cost and gradient
J = computeCostgrad(X_norm, y, initial_theta);
fprintf('Cost at initial theta (zeros): %f\n', J);
fprintf('Expected cost (approx): 0.693\n');
fprintf('\nProgram paused. Press enter to continue.\n');
pause;
%% ============= Part 2a: Gradient Descent =====================
alpha=0.1;
iter=1000;
[theta, J_hist]=gradientDescent(initial_theta, X_norm, y, alpha, iter);
fprintf('Theta found by gradient descent:\n');
fprintf('%f\n', theta);
% Plot the convergence graph
figure;
plot(1:numel(J_hist), J_hist, '-b', 'LineWidth', 2);
xlabel('Nnumelumber of iterations');
ylabel('Cost J');
% Plot Boundary
plotDecisionBoundary(theta, X, y);
% Put some labels
hold on;
% Labels and Legend
xlabel('Exam 1 score')
ylabel('Exam 2 score')
% Specified in plot order
legend('Admitted', 'Not admitted')
hold off;
fprintf('\nProgram paused. Press enter to continue.\n');
pause;
%% ============== Part 4: Predict and Accuracies ==============
% After learning the parameters, you'll like to use it to predict the outcomes
% on unseen data. In this part, you will use the logistic regression model
% to predict the probability that a student with score 45 on exam 1 and
% score 85 on exam 2 will be admitted.
%
% Furthermore, you will compute the training and test set accuracies of
% our model.
%
% Your task is to complete the code in predict.m
% Predict probability for a student with score 45 on exam 1
% and score 85 on exam 2
%prob = sigmoid([1 45 85] * theta);
pred_admit=[45 85];
norm_pred_admit=[1,(pred_admit-mu)./sigma];
prob = norm_pred_admit*theta;
fprintf(['For a student with scores 45 and 85, we predict an admission ' ...
'probability of %f\n'], prob);
fprintf('Expected value: 0.775 +/- 0.002\n\n');
% Compute accuracy on our training set
p = predict(theta, X_norm);
fprintf('Train Accuracy: %f\n', mean(double(p == y)) * 100);
fprintf('Expected accuracy (approx): 89.0\n');
fprintf('\n');
computeCostgrad计算成本梯度
function [J] = computeCostgrad(X, y, theta)
% Initialize some useful values
m = length(y); % number of training examples
% You need to return the following variables correctly
J = 0;
prediction=sigmoid(X*theta);
prob1=-y'*log(prediction);
prob0=(1-y')*log(1-prediction);
J=1/m*(prob1-prob0);
endfunction
gradientDescent梯度下降
function [theta, J_hist] = gradientDescent(theta, X, y, alpha, iter)
m=length(y);
J_hist=zeros(iter, 1);
for (i=1:iter)
prediction=sigmoid(X*theta);
err=prediction-y;
newDecrement = (alpha * (1/m) * err' * X);
theta=theta-newDecrement';
J_hist(i)=computeCostgrad(X,y,theta);
end
endfunction
plotDecisionBoundary绘图决策边界
function plotDecisionBoundary(theta, X, y)
plotData(X(:,2:3), y);
hold on
if size(X, 2) <= 3
% Only need 2 points to define a line, so choose two endpoints
plot_x = [min(X(:,2))-2, max(X(:,2))+2];
% Calculate the decision boundary line
plot_y = (-1./theta(3)).*(theta(2).*plot_x + theta(1));
% Plot, and adjust axes for better viewing
plot(plot_x, plot_y)
% Legend, specific for the exercise
legend('Admitted', 'Not admitted', 'Decision Boundary')
axis([30, 100, 30, 100])
else
% Here is the grid range
u = linspace(-1, 1.5, 50);
v = linspace(-1, 1.5, 50);
z = zeros(length(u), length(v));
% Evaluate z = theta*x over the grid
for i = 1:length(u)
for j = 1:length(v)
z(i,j) = mapFeature(u(i), v(j))*theta;
end
end
z = z'; % important to transpose z before calling contour
% Plot z = 0
% Notice you need to specify the range [0, 0]
contour(u, v, z, [0, 0], 'LineWidth', 2)
end
hold off
end
featureNormalize特征归一化
function [X_norm, mu, sigma] = featureNormalize(X)
X_norm = X;
mu = zeros(1, size(X, 2));
sigma = zeros(1, size(X, 2));
mu=mean(X);
sigma=std(X);
X_norm1=(X(:,1)-mu(1))/sigma(1);
X_norm2=(X(:,2)-mu(2))/sigma(2);
X_norm=[X_norm1,X_norm2];
Can anybody please help me in plotting the decision boundary correctly.任何人都可以帮助我正确绘制决策边界。 I think there is some mistake in calculation of yintercept in plotting of decision boundary.
我认为在绘制决策边界时计算 yintercept 存在一些错误。
Because you have used feature scaling, your weights don't match your original data.因为您使用了特征缩放,所以您的权重与原始数据不匹配。
You need to pass your X_norm
to your plotDecisionBoundary
function, instead of the original data, X
, like this:您需要将
X_norm
传递给您的plotDecisionBoundary
function,而不是原始数据X
,如下所示:
plotDecisionBoundary(theta, X_norm, y);
Likewise, when you want to predict a new example, you first need to scale it with the same values you computed to normalize your training examples.同样,当您想要预测一个新示例时,您首先需要使用您计算的相同值对其进行缩放以标准化您的训练示例。
In addition, in plotDecisionBoundary
the line axis([30, 100, 30, 100])
is fitting to X
and not to X_norm
.此外,在
plotDecisionBoundary
中, axis([30, 100, 30, 100])
适合X
而不是X_norm
。 Therefore, you need to change it to fit the range of X_norm
(it is is just for comfort, you always can change it by changing the zoom till you find the line).因此,您需要更改它以适应
X_norm
的范围(这只是为了舒适,您始终可以通过更改缩放来更改它,直到找到线)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.