标签：
%% 方法一：梯度下降法
x = load(‘E:\workstation\data\ex3x.dat‘);
y = load(‘E:\workstation\data\ex3y.dat‘);
x = [ones(size(x,1),1) x];
meanx = mean(x);%求均值
sigmax = std(x);%求标准偏差
x(:,2) = (x(:,2)-meanx(2))./sigmax(2);
x(:,3) = (x(:,3)-meanx(3))./sigmax(3);
figure
itera_num = 100; %尝试的迭代次数
sample_num = size(x,1); %训练样本的次数
alpha = [0.01, 0.03, 0.1, 0.3, 1, 1.3];%因为差不多是选取每个3倍的学习率来测试，所以直接枚举出来
plotstyle = {‘b‘, ‘r‘, ‘g‘, ‘k‘, ‘b--‘, ‘r--‘};
theta_grad_descent = zeros(size(x(1,:)));
for alpha_i = 1:length(alpha) %尝试看哪个学习速率最好
theta = zeros(size(x,2),1); %theta的初始值赋值为0
Jtheta = zeros(itera_num, 1);
for i = 1:itera_num %计算出某个学习速率alpha下迭代itera_num次数后的参数
Jtheta(i) = (1/(2*sample_num)).*(x*theta-y)‘*(x*theta-y);%Jtheta是个行向量
grad = (1/sample_num).*x‘*(x*theta-y);
theta = theta - alpha(alpha_i).*grad;
end
plot(0:49, Jtheta(1:50),char(plotstyle(alpha_i)),‘LineWidth‘, 2)%此处一定要通过char函数来转换
hold on
if(1 == alpha(alpha_i)) %通过实验发现alpha为1时效果最好，则此时的迭代后的theta值为所求的值
theta_grad_descent = theta
end
end
legend(‘0.01‘,‘0.03‘,‘0.1‘,‘0.3‘,‘1‘,‘1.3‘);
xlabel(‘Number of iterations‘)
ylabel(‘Cost function‘)
%下面是预测公式
price_grad_descend = theta_grad_descent‘*[1 (1650-meanx(2))/sigmax(2) (3-meanx(3)/sigmax(3))]‘
%%方法二：normal equations
x = load(‘E:\workstation\data\ex3x.dat‘);
y = load(‘E:\workstation\data\ex3y.dat‘);
x = [ones(size(x,1),1) x];
theta_norequ = inv((x‘*x))*x‘*y
price_norequ = theta_norequ‘*[1 1650 3]‘
1 %% 方法一：梯度下降法 2 x = load(‘E:\workstation\data\ex3x.dat‘); 3 y = load(‘E:\workstation\data\ex3y.dat‘); 4 5 x = [ones(size(x,1),1) x]; 6 meanx = mean(x);%求均值 7 sigmax = std(x);%求标准偏差 8 x(:,2) = (x(:,2)-meanx(2))./sigmax(2); 9 x(:,3) = (x(:,3)-meanx(3))./sigmax(3); 10 11 figure 12 itera_num = 100; %尝试的迭代次数 13 sample_num = size(x,1); %训练样本的次数 14 alpha = [0.01, 0.03, 0.1, 0.3, 1, 1.3];%因为差不多是选取每个3倍的学习率来测试，所以直接枚举出来 15 plotstyle = {‘b‘, ‘r‘, ‘g‘, ‘k‘, ‘b--‘, ‘r--‘}; 16 17 theta_grad_descent = zeros(size(x(1,:))); 18 for alpha_i = 1:length(alpha) %尝试看哪个学习速率最好 19 theta = zeros(size(x,2),1); %theta的初始值赋值为0 20 Jtheta = zeros(itera_num, 1); 21 for i = 1:itera_num %计算出某个学习速率alpha下迭代itera_num次数后的参数 22 Jtheta(i) = (1/(2*sample_num)).*(x*theta-y)‘*(x*theta-y);%Jtheta是个行向量 23 grad = (1/sample_num).*x‘*(x*theta-y); 24 theta = theta - alpha(alpha_i).*grad; 25 end 26 plot(0:49, Jtheta(1:50),char(plotstyle(alpha_i)),‘LineWidth‘, 2)%此处一定要通过char函数来转换 27 hold on 28 29 if(1 == alpha(alpha_i)) %通过实验发现alpha为1时效果最好，则此时的迭代后的theta值为所求的值 30 theta_grad_descent = theta 31 end 32 end 33 legend(‘0.01‘,‘0.03‘,‘0.1‘,‘0.3‘,‘1‘,‘1.3‘); 34 xlabel(‘Number of iterations‘) 35 ylabel(‘Cost function‘) 36 37 %下面是预测公式 38 price_grad_descend = theta_grad_descent‘*[1 (1650-meanx(2))/sigmax(2) (3-meanx(3)/sigmax(3))]‘ 39 40 41 %%方法二：normal equations 42 x = load(‘E:\workstation\data\ex3x.dat‘); 43 y = load(‘E:\workstation\data\ex3y.dat‘); 44 x = [ones(size(x,1),1) x]; 45 46 theta_norequ = inv((x‘*x))*x‘*y 47 price_norequ = theta_norequ‘*[1 1650 3]‘
Multivariance Linear Regression练习
本文要解决的问题是给出了47个训练样本，训练样本的y值为房子的价格，x属性有2个，一个是房子的大小，另一个是房子卧室的个数。需要通过这些训练数据来学习系统的函数，从而预测房子大小为1650，且卧室有3个的房子的价格。
实验基础：
dot(A,B):表示的是向量A和向量B的内积。
又线性回归的理论可以知道系统的损失函数如下所示：
其向量表达形式如下：
当使用梯度下降法进行参数的求解时，参数的更新公式如下：
当然它也有自己的向量形式（程序中可以体现）。
两种方法比较预测值：
l ：梯度下降法：
1训练数据：
x = load(‘E:\workstation\data\ex3x.dat‘);
y = load(‘E:\workstation\data\ex3y.dat‘);
x = [ones(size(x,1),1) x];
meanx = mean(x) 均值
sigmax = std(x) 标准差
x(:,2) = (x(:,2)-meanx(2))./sigmax(2);
x(:,3) = (x(:,3)-meanx(3))./sigmax(3);
2：Gradient descen梯度迭代
重点选择合适的下降下降梯度；
it‘s time to select a learning rate The goal of this part is to pick a good learning rate in the range of
|
|
|
You will do this by making an initial selection, running gradient descent and observing the cost function, and adjusting the learning rate accordingly. Recall that the cost function is defined as
|
|
|
The cost function can also be written in the following vectorized form,
|
|
|
where
|
|
画图;展示每个学习速率对应的下降图像选取最佳的下降图像
预测值为;
price_grad_descend = theta_grad_descent‘*[1 (1650-meanx(2))/sigmax(2) (3-meanx(3)/sigmax(3))]‘
l 方法二 普通方法直接
x = load(‘E:\workstation\data\ex3x.dat‘);
y = load(‘E:\workstation\data\ex3y.dat‘);
x = [ones(size(x,1),1) x];
theta_norequ = inv((x‘*x))*x‘*y
price_norequ = theta_norequ‘*[1 1650 3]‘
Multivariance Linear Regression练习
标签：
原文地址：http://www.cnblogs.com/meng-qing/p/4604404.html
