码迷,mamicode.com
首页 > 系统相关 > 详细

Machine Learning Techniques -3-Dual Support Vector Machine

时间:2015-08-21 23:09:58      阅读:1009      评论:0      收藏:0      [点我收藏+]

标签:

For the naive thought to practise my academic English skill, the rest of my notes will be wrriten in my terrrible English.XD

If you have any kind of uncomfortable feel, please close this window and refer to the original edition from Mr. Lin.

I will be really appriciate for your understanding.

OK, so much for this.

We discussed the powerful tool Dual SVM with a hard bound in the last class, which helps us to better understand the meaning of SVM.

But we did not solve the problem brought from a big ~d~

技术分享

 

One large ~d~ my cause disaster  when caculating Qd, which is the bottlenect of our model.

Here we introduce one tool called kernel function to better our situation.

If we can get the result of a specific kind of function with the parameters x and x‘, we can cut down the process in computing and give the output diectly for the input.

That‘s what we call kernel function.

技术分享

We use a 2nd order polynomial transform to illustrate.

The idea to make the computing sampler is to deal with xTx‘ and the polynomial rrelationship at the same time.

So here we get the kernel function whose input is x and x‘:

技术分享

 

For the above question, we can apply the kernel function:

quadratic coefficient q n,m = y n y m z n T z m = y n y m K (x n , x m ) to get the matrix Qd.

So that we need not to de the caculation in space of Z, but we could use KERNEL FUNCTION to get ZnT*Zm used xn and xm.

技术分享

Kernel trick: plug in efficient kernel function to avoid dependence on d ?

So if we give this method a name called Kernel SVM:

技术分享

Let us come back to the 2nd polynomial, if we add some factor into  expansion equation, we may get some new kernel function:

技术分享

From the aspect of geometry, different kernel means different geometry distance, which affects the appearance of mapping, the defination of margin.

技术分享

Now, we already have the common represention of polynomial kernel:

技术分享

Particularly, Q = 1 means the linear condiction.

Then, how about infinite dimensional Φ(x)? Yes, it is also avalible.

技术分享

Linear combination of Gaussians centered at SVs xn ,  also called Radial Basis Function (RBF) kernel

技术分享

The value of 伽马 may determine the the extent of the tip for the gaussian function, which means the shadow of overfiting is still alive.

In the end, Mr. Lin compares these three kernel functions and shows their pros and cons, which are In line with the intuition.

An important point is that one potential manual kernel have to be" ZZ must always be positive semi-definite "

Machine Learning Techniques -3-Dual Support Vector Machine

标签:

原文地址:http://www.cnblogs.com/windniu/p/4749195.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!