标签：
As an alternative for generative models and discriminative models, a discriminant directly assigns a feature vector to one of K classes. One of the simplest discriminant function for 2-class problems should be something like y(x)=sign(w‘*x+b), where w is the pending parameter vector and b is a pending bias. Here x is different from the one we talk about in regression models since it no more comprises a bias term.
To obtain proper parameters, we can draw on a simple algorithm called Perceptron, which gurantees all the training data shall be correctly classified. This is done by minimizing an error function, each of whose terms should be something like -(w‘*xn+b)*tn, in an iterative way, and this procedure will never terminate if the problem is not linearly separable.
1 function w = percept(X,t)
2 % Peceptron Algorithm for Linear Classification
3 % Precondtion: X is a set of data columns,
4 % row vector t is the labels of X (+1 or -1)
5 % Postcondition: w is the linear model parameter
6 % such that y = sign(w‘* x)
7 [m,n] = size(X);
8 w = zeros(m,1);
9 cnt = 0; % consecutive hit number
10 cur = 1; % current data item
11 while (cnt<n)
12 % until no misclassification exists
13 if (t(cur)*w‘*X(:,cur)<=0)
14 % error correction, step = 0.2
15 w = w + 0.2*t(cur)*X(:,cur);
16 cnt = 0;
17 else
18 cnt = cnt+1;
19 end
20 cur = mod(cur,n)+1;
21 end
22 end
Fisher‘s Linear Discriminant is another linear classifier, which makes every endeavor to maximize the class separation by choosing a deisired direction on which the projections of two mean vectors have the largest distance. This goal is attained by finding a maximum point for the Fisher criterion: J(w)=(m2-m1)2/(S12+S22), where m1, m2 and S1, S2 are the means and variances of the projected data respectively.
1 function w = fisher(X,t) 2 % Fisher‘s Linear Discriminant for 2-class problems 3 % Precondtion: X is a set of data columns, 4 % row vector t is the labels of X (+1 or -1) 5 % Postcondition: w is the linear model parameter 6 % such that y = sign(w‘* x) 7 d = size(X,1)-1; 8 % calculate the mean vectors of the 2 classes: 9 m1 = zeros(d,1); 10 m2 = zeros(d,1); 11 n1 = 0; n2 = 0; 12 for i = 1:size(t,2) 13 if (t(1,i)>0) 14 n1 = n1+1; 15 m1 = m1+X(1:d,i); 16 else 17 n2 = n2+1; 18 m2 = m2+X(1:d,i); 19 end 20 end 21 m1 = m1/n1; 22 m2 = m2/n2; 23 % calculate the within-class covariance matrix: 24 Sw = zeros(d); 25 for i = 1:size(t,2) 26 if (t(1,i)>0) 27 Sw = Sw+(X(1:d,i)-m1)*(X(1:d,i)-m1)‘; 28 else 29 Sw = Sw+(X(1:d,i)-m2)*(X(1:d,i)-m2)‘; 30 end 31 end 32 w = Sw\(m1-m2); 33 % choose a proper threshold: 34 w0Min = inf; 35 w0Max = -inf; 36 for i = 1:size(t,2) 37 y = w‘*X(1:d,i); 38 if (t(1,i)>0 & y+w0Max<0) 39 w0Max = -y; 40 elseif (t(1,i)0) 41 w0Min = -y; 42 end 43 end 44 w = [w;(w0Min+w0Max)/2]; 45 end
Support Vector Machine (SVM) is another kind of discriminant that can effectively help us consummate the classification on a linearly separable data set, and its objective is to maximize the margin, the distance between the decision boundary and any samples. This is equivalent to the optimization problem of maximizing ½||w||2 given the restrictions yi(w‘*xi+b)≥1 (i=1,2,3...N):
1 function w = supvect(X,t)
2 % Support Vector Machine for Linear Classification
3 % Precondtion: X is a set of data columns,
4 % row vector t is the labels of X (+1 or -1)
5 % Postcondition: w is the linear model parameter
6 % such that y = sign(w‘* x)
7 [m,n] = size(X);
8 x0 = zeros(m,1);
9 A = zeros(n,m);
10 for i = 1:n
11 A(i,:) = -t(i)*X(:,i)‘;
12 end
13 b = -ones(n,1);
14 w = fmincon(‘norm‘,x0,A,b);
15 end
References:
1. Bishop, Christopher M. Pattern Recognition and Machine Learning [M]. Singapore: Springer, 2006
标签：
原文地址：http://www.cnblogs.com/DevinZ/p/4472477.html
