Smooth Support Vector Machine - Python实现

时间：2020-03-05 13:30:27 阅读：123 评论：0 收藏：0 [点我收藏+]

标签：== closed spi matrix event ref ret pac 技术

算法特征:
①. 所有点尽可能正确分开; ②. 极大化margin; ③. 极小化非线性可分之误差.
算法推导:
Part Ⅰ
线性可分之含义:
包含同类型所有数据点的最小凸集合彼此不存在交集.
引入光滑化手段:
plus function: \begin{equation*}(x)_{+} = max \{ x, 0 \}\end{equation*}
p-function: \begin{equation*}p(x, \beta) = \frac{1}{\beta}ln(e^{\beta x} + 1)\end{equation*}
可采用p-function近似plus function. 相关函数图像如下:
下面给出p-function的1阶及2阶导数:
1st order (sigmoid function): \begin{equation*} s(x, \beta) = \frac{1}{e^{-\beta x} + 1} \end{equation*}
2nd order (delta function): \begin{equation*} \delta (x, \beta) = \frac{\beta e^{\beta x}}{(e^{\beta x} + 1)^2} \end{equation*}
相关函数图像如下:
Part Ⅱ
SVM决策超平面方程如下:
\begin{equation}\label{eq_1}
h(x) = W^Tx + b
\end{equation}
其中, $W$是超平面法向量, $b$是超平面偏置参数.
本文拟采用$L_2$范数衡量非线性可分之误差, 则SVM原始优化问题如下:
\begin{equation}
\begin{split}
\min &\quad\frac{1}{2}\left\|W\right\|_2^2 + \frac{c}{2}\left\|\xi\right\|_2^2 \\
s.t. &\quad D(A^TW + 1b) \geq 1 - \xi
\end{split}\label{eq_2}
\end{equation}
其中, $A = [x^{(1)}, x^{(2)}, \cdots , x^{(n)}]$, $D = diag([\bar{y}^{(1)}, \bar{y}^{(2)}, \cdots, \bar{y}^{(n)}])$, $\xi$是由所有样本非线性可分之误差组成的列矢量.
根据上述优化问题(\ref{eq_2}), 误差$\xi$可采用plus function等效表示:
\begin{equation}\label{eq_3}
\xi = (1 - D(A^TW + 1b))_{+}
\end{equation}
由此可将该有约束最优化问题转化为如下无约束最优化问题:
\begin{equation}\label{eq_4}
\min\quad \frac{1}{2}\left\|W\right\|_2^2 + \frac{c}{2}\left\| \left(1 - D(A^TW + 1b)\right)_{+} \right\|_2^2
\end{equation}
同样, 根据优化问题(\ref{eq_2})KKT条件中稳定性条件有:
\begin{equation}\label{eq_5}
W = AD^T\alpha
\end{equation}
其中, $\alpha$为对偶变量. 代入式(\ref{eq_4})变数变换可得:
\begin{equation}\label{eq_6}
\min\quad \frac{1}{2}\alpha^TD^TA^TAD\alpha + \frac{c}{2}\left\| \left(1 - D(A^TAD^T\alpha + 1b)\right)_{+} \right\|_2^2
\end{equation}
由于权重参数$c$的存在, 上述最优化问题等效如下多目标最优化问题:
\begin{equation}
\begin{cases}
\min& \frac{1}{2}\alpha^TD^TA^TAD\alpha \\
\min& \frac{1}{2}\left\| \left(1 - D(A^TAD^T\alpha + 1b)\right)_{+} \right\|_2^2
\end{cases}\label{eq_7}
\end{equation}
由于$D^TA^TAD\succeq 0$, 有如下等效:
\begin{equation}\label{eq_8}
\min\quad \frac{1}{2}\alpha^TD^TA^TAD\alpha \quad\Rightarrow\quad \min\quad \frac{1}{2}\alpha^T\alpha
\end{equation}
根据Part Ⅰ中光滑化手段, 有如下等效:
\begin{equation}\label{eq_9}
\min\quad \frac{1}{2}\left\| \left(1 - D(A^TAD^T\alpha + 1b)\right)_{+} \right\|_2^2 \quad\Rightarrow\quad \min\quad \frac{1}{2}\left\| p(1 - DA^TAD\alpha - D1b, \beta)\right\|_2^2, \quad\text{where $\beta\to\infty$}
\end{equation}
结合式(\ref{eq_8})、式(\ref{eq_9}), 优化问题(\ref{eq_6})等效如下:
\begin{equation}\label{eq_10}
\min\quad \frac{1}{2}\alpha^T\alpha + \frac{c}{2}\left\| p(1 - DA^TAD\alpha - D1b, \beta)\right\|_2^2, \quad\text{where $\beta\to\infty$}
\end{equation}
为增强模型得分类能力, 引入kernel trick, 得最终优化问题:
\begin{equation}\label{eq_11}
\min\quad \frac{1}{2}\alpha^T\alpha + \frac{c}{2}\left\| p(1 - DK(A^T, A)D\alpha - D1b, \beta)\right\|_2^2, \quad\text{where $\beta\to\infty$}
\end{equation}
其中, $K$代表kernel矩阵, 内部元素$K_{ij}=k(x^{(i)}, x^{(j)})$由kernel函数决定. 常见的kernel函数如下:
①. 多项式核: $k(x^{(i)}, x^{(j)}) = ({x^{(i)}}\cdot x^{(j)})^n$
②. 高斯核: $k(x^{(i)}, x^{(j)}) = e^{-\mu\| x^{(i)} - x^{(j)}\|_2^2}$
结合式(\ref{eq_1})、式(\ref{eq_5})及kernel trick可得最终决策超平面方程:
\begin{equation}
h(x) = \alpha^TDK(A^T, x) + b
\end{equation}

Part Ⅲ
以下给出优化相关协助信息. 拟设式(\ref{eq_11})之目标函数符号为$J$, 并令:
\begin{equation}
J = J_1 + J_2\quad\quad\text{其中,}
\begin{cases}
J_1 = \frac{1}{2}\alpha^T\alpha \\
J_2 = \frac{c}{2}\left\| p(1 - DK(A^T, A)D\alpha - D1b, \beta)\right\|_2^2
\end{cases}
\end{equation}
$J_1$相关:
\begin{gather*}
\frac{\partial J_1}{\partial \alpha} = \alpha \quad \frac{\partial J_1}{\partial b} = 0 \\
\frac{\partial^2J_1}{\partial\alpha^2} = I \quad \frac{\partial^2J_1}{\partial\alpha\partial b} = 0 \quad \frac{\partial^2J_1}{\partial b\partial\alpha} = 0 \quad \frac{\partial^2J_1}{\partial b^2} = 0
\end{gather*}
\begin{gather*}
\nabla J_1 = \begin{bmatrix} \frac{\partial J_1}{\partial \alpha} \\ \frac{\partial J_1}{\partial b}\end{bmatrix} \\ \\
\nabla^2 J_1 = \begin{bmatrix}\frac{\partial^2J_1}{\partial\alpha^2} & \frac{\partial^2J_1}{\partial\alpha\partial b} \\ \frac{\partial^2J_1}{\partial b\partial\alpha} & \frac{\partial^2J_1}{\partial b^2}\end{bmatrix}
\end{gather*}
$J_2$相关:
\begin{gather*}
z_i = 1 - \sum_{j}K_{ij}\bar{y}^{(i)}\bar{y}^{(j)}\alpha_j - \bar{y}^{(i)}b \\
\frac{\partial J_2}{\partial \alpha_k} = -c\sum_{i}\bar{y}^{(i)}\bar{y}^{(k)}K_{ik}p(z_i, \beta)s(z_i, \beta) \\
\frac{\partial J_2}{\partial b} = -c\sum_{i}\bar{y}^{(i)}p(z_i, \beta)s(z_i, \beta) \\
\frac{\partial^2 J_2}{\partial \alpha_k\partial \alpha_l} = c\sum_{i}\bar{y}^{(k)}\bar{y}^{(l)}K_{ik}K_{il}[s^2(z_i, \beta) + p(z_i, \beta)\delta(z_i, \beta)] \\
\frac{\partial^2 J_2}{\partial \alpha_k\partial b} = c\sum_{i}\bar{y}^{(k)}K_{ik}[s^2(z_i, \beta) + p(z_i, \beta)\delta(z_i, \beta)] \\
\frac{\partial^2 J_2}{\partial b\partial\alpha_l} = c\sum_{i}\bar{y}^{(l)}K_{il}[s^2(z_i, \beta) + p(z_i, \beta)\delta(z_i, \beta)] \\
\frac{\partial^2 J_2}{\partial b^2} = c\sum_{i}[s^2(z_i, \beta) + p(z_i, \beta)\delta(z_i, \beta)]
\end{gather*}
\begin{gather*}
\nabla J_2 = \begin{bmatrix} \frac{\partial J_2}{\partial\alpha_k} \\ \frac{\partial J_2}{\partial b} \end{bmatrix} \\ \\
\nabla^2 J_2 = \begin{bmatrix} \frac{\partial^2 J_2}{\partial\alpha_k\partial\alpha_l} & \frac{\partial^2 J_2}{\partial\alpha_k\partial b} \\ \frac{\partial^2 J_2}{\partial b\partial\alpha_l} & \frac{\partial^2 J_2}{\partial b^2} \end{bmatrix}
\end{gather*}
目标函数$J$之gradient及Hessian如下:
\begin{gather}
\nabla J = \nabla J_1 + \nabla J_2 \\
H = \nabla^2 J = \nabla^2 J_1 + \nabla^2 J_2
\end{gather}
Part Ⅳ
现以二分类为例进行算法实施:
\begin{equation*}
\begin{cases}
h(x) \geq 0 & y = 1 & \text{正例} \\
h(x) < 0 & y = -1 & \text{负例}
\end{cases}
\end{equation*}

代码实现:

  1 # Smooth Support Vector Machine之实现
  2 
  3 import numpy
  4 from matplotlib import pyplot as plt
  5 
  6 
  7 def spiral_point(val, center=(0, 0)):
  8     rn = 0.4 * (105 - val) / 104
  9     an = numpy.pi * (val - 1) / 25
 10     
 11     x0 = center[0] + rn * numpy.sin(an)
 12     y0 = center[1] + rn * numpy.cos(an)
 13     z0 = -1
 14     x1 = center[0] - rn * numpy.sin(an)
 15     y1 = center[1] - rn * numpy.cos(an)
 16     z1 = 1
 17     
 18     return (x0, y0, z0), (x1, y1, z1)
 19 
 20 
 21 def spiral_data(valList):
 22     dataList = list(spiral_point(val) for val in valList)
 23     data0 = numpy.array(list(item[0] for item in dataList))
 24     data1 = numpy.array(list(item[1] for item in dataList))
 25     return data0, data1
 26     
 27 
 28 # 生成训练数据集
 29 trainingValList = numpy.arange(1, 101, 1)
 30 trainingData0, trainingData1 = spiral_data(trainingValList)
 31 trainingSet = numpy.vstack((trainingData0, trainingData1))
 32 # 生成测试数据集
 33 testValList = numpy.arange(1.5, 101.5, 1)
 34 testData0, testData1 = spiral_data(testValList)
 35 testSet = numpy.vstack((testData0, testData1))
 36 
 37 
 38 class SSVM(object):
 39     
 40     def __init__(self, trainingSet, c=1, mu=1, beta=100):
 41         self.__trainingSet = trainingSet                     # 训练集数据
 42         self.__c = c                                         # 误差项权重
 43         self.__mu = mu                                       # gaussian kernel参数
 44         self.__beta = beta                                   # 光滑化参数
 45         
 46         self.__A, self.__D = self.__get_AD()
 47         
 48         
 49     def get_cls(self, x, alpha, b):
 50         A, D = self.__A, self.__D
 51         mu = self.__mu
 52         
 53         x = numpy.array(x).reshape((-1, 1))
 54         KAx = self.__get_KAx(A, x, mu)
 55         clsVal = self.__calc_hVal(KAx, D, alpha, b)
 56         if clsVal > 0:
 57             return 1
 58         elif clsVal == 0:
 59             return 0
 60         else:
 61             return -1
 62             
 63             
 64     def get_accuracy(self, dataSet, alpha, b):
 65         ‘‘‘
 66         正确率计算
 67         ‘‘‘
 68         rightCnt = 0
 69         for row in dataSet:
 70             clsVal = self.get_cls(row[:2], alpha, b)
 71             if clsVal == row[2]:
 72                 rightCnt += 1
 73         accuracy = rightCnt / dataSet.shape[0]
 74         return accuracy
 75         
 76         
 77     def optimize(self, maxIter=100, epsilon=1.e-9):
 78         ‘‘‘
 79         maxIter: 最大迭代次数
 80         epsilon: 收敛判据, 梯度趋于0则收敛
 81         ‘‘‘
 82         A, D = self.__A, self.__D
 83         c = self.__c
 84         mu = self.__mu
 85         beta = self.__beta
 86 
 87         alpha, b = self.__init_alpha_b((A.shape[1], 1))
 88         KAA = self.__get_KAA(A, mu)
 89         
 90         JVal = self.__calc_JVal(KAA, D, c, beta, alpha, b)
 91         grad = self.__calc_grad(KAA, D, c, beta, alpha, b)
 92         Hess = self.__calc_Hess(KAA, D, c, beta, alpha, b)
 93         
 94         for i in range(maxIter):
 95             if self.__converged1(grad, epsilon):
 96                 return alpha, b, True
 97 
 98             dCurr = -numpy.matmul(numpy.linalg.inv(Hess), grad)
 99             ALPHA = self.__calc_ALPHA_by_ArmijoRule(alpha, b, JVal, grad, dCurr, KAA, D, c, beta)
100 
101             delta = ALPHA * dCurr
102             alphaNew = alpha + delta[:-1, :]
103             bNew = b + delta[-1, -1]
104             JValNew = self.__calc_JVal(KAA, D, c, beta, alphaNew, bNew)
105             if self.__converged2(delta, JValNew - JVal, epsilon):
106                 return alphaNew, bNew, True
107                 
108             alpha, b, JVal = alphaNew, bNew, JValNew
109             grad = self.__calc_grad(KAA, D, c, beta, alpha, b)
110             Hess = self.__calc_Hess(KAA, D, c, beta, alpha, b)
111         else:
112             if self.__converged1(grad, epsilon):
113                 return alpha, b, True
114         return alpha, b, False
115         
116         
117     def __converged2(self, delta, JValDelta, epsilon):
118         val1 = numpy.linalg.norm(delta)
119         val2 = numpy.linalg.norm(JValDelta)
120         if val1 <= epsilon or val2 <= epsilon:
121             return True
122         return False
123         
124     
125     def __calc_ALPHA_by_ArmijoRule(self, alphaCurr, bCurr, JCurr, gCurr, dCurr, KAA, D, c, beta, C=1.e-4, v=0.5):
126         i = 0
127         ALPHA = v ** i
128         delta = ALPHA * dCurr
129         alphaNext = alphaCurr + delta[:-1, :]
130         bNext = bCurr + delta[-1, -1]
131         JNext = self.__calc_JVal(KAA, D, c, beta, alphaNext, bNext)
132         while True:
133             if JNext <= JCurr + C * ALPHA * numpy.matmul(dCurr.T, gCurr)[0, 0]: break
134             i += 1
135             ALPHA = v ** i
136             delta = ALPHA * dCurr
137             alphaNext = alphaCurr + delta[:-1, :]
138             bNext = bCurr + delta[-1, -1]
139             JNext = self.__calc_JVal(KAA, D, c, beta, alphaNext, bNext)
140         return ALPHA
141     
142     
143     def __converged1(self, grad, epsilon):
144         if numpy.linalg.norm(grad) <= epsilon:
145             return True
146         return False
147     
148         
149     def __p(self, x, beta):
150         val = numpy.log(numpy.exp(beta * x) + 1) / beta
151         return val
152         
153         
154     def __s(self, x, beta):
155         val = 1 / (numpy.exp(-beta * x) + 1)
156         return val
157         
158         
159     def __d(self, x, beta):
160         term = numpy.exp(beta * x)
161         val = beta * term / (term + 1) ** 2
162         return val
163         
164         
165     def __calc_Hess(self, KAA, D, c, beta, alpha, b):
166         Hess_J1 = self.__calc_Hess_J1(alpha)
167         Hess_J2 = self.__calc_Hess_J2(KAA, D, c, beta, alpha, b)
168         Hess = Hess_J1 + Hess_J2
169         return Hess
170         
171         
172     def __calc_Hess_J2(self, KAA, D, c, beta, alpha, b):
173         Hess_J2 = numpy.zeros((KAA.shape[0] + 1, KAA.shape[0] + 1))
174         Y = numpy.matmul(D, numpy.ones((D.shape[0], 1)))
175         YY = numpy.matmul(Y, Y.T)
176         KAAYY = KAA * YY
177         
178         z = 1 - numpy.matmul(KAAYY, alpha) - Y * b
179         p = numpy.array(list(self.__p(z[i, 0], beta) for i in range(z.shape[0]))).reshape((-1, 1))
180         s = numpy.array(list(self.__s(z[i, 0], beta) for i in range(z.shape[0]))).reshape((-1, 1))
181         d = numpy.array(list(self.__d(z[i, 0], beta) for i in range(z.shape[0]))).reshape((-1, 1))
182         term = s * s + p * d
183         
184         for k in range(Hess_J2.shape[0] - 1):
185             for l in range(k + 1):
186                 val = c * Y[k, 0] * Y[l, 0] * numpy.sum(KAA[:, k:k+1] * KAA[:, l:l+1] * term)
187                 Hess_J2[k, l] = Hess_J2[l, k] = val
188             val = c * Y[k, 0] * numpy.sum(KAA[:, k:k+1] * term)
189             Hess_J2[k, -1] = Hess_J2[-1, k] = val
190         val = c * numpy.sum(term)
191         Hess_J2[-1, -1] = val
192         return Hess_J2
193         
194         
195     def __calc_Hess_J1(self, alpha):
196         I = numpy.identity(alpha.shape[0])
197         term = numpy.hstack((I, numpy.zeros((I.shape[0], 1))))
198         Hess_J1 = numpy.vstack((term, numpy.zeros((1, term.shape[1]))))
199         return Hess_J1
200         
201     
202     def __calc_grad(self, KAA, D, c, beta, alpha, b):
203         grad_J1 = self.__calc_grad_J1(alpha)
204         grad_J2 = self.__calc_grad_J2(KAA, D, c, beta, alpha, b)
205         grad = grad_J1 + grad_J2
206         return grad
207         
208         
209     def __calc_grad_J2(self, KAA, D, c, beta, alpha, b):
210         grad_J2 = numpy.zeros((KAA.shape[0] + 1, 1))
211         Y = numpy.matmul(D, numpy.ones((D.shape[0], 1)))
212         YY = numpy.matmul(Y, Y.T)
213         KAAYY = KAA * YY
214         
215         z = 1 - numpy.matmul(KAAYY, alpha) - Y * b
216         p = numpy.array(list(self.__p(z[i, 0], beta) for i in range(z.shape[0]))).reshape((-1, 1))
217         s = numpy.array(list(self.__s(z[i, 0], beta) for i in range(z.shape[0]))).reshape((-1, 1))
218         term = p * s
219         
220         for k in range(grad_J2.shape[0] - 1):
221             val = -c * Y[k, 0] * numpy.sum(Y * KAA[:, k:k+1] * term)
222             grad_J2[k, 0] = val
223         grad_J2[-1, 0] = -c * numpy.sum(Y * term)
224         return grad_J2
225         
226     
227     def __calc_grad_J1(self, alpha):
228         grad_J1 = numpy.vstack((alpha, [[0]]))
229         return grad_J1
230     
231     
232     def __calc_JVal(self, KAA, D, c, beta, alpha, b):
233         J1 = self.__calc_J1(alpha)
234         J2 = self.__calc_J2(KAA, D, c, beta, alpha, b)
235         JVal = J1 + J2
236         return JVal
237         
238         
239     def __calc_J2(self, KAA, D, c, beta, alpha, b):
240         tmpOne = numpy.ones((KAA.shape[0], 1))
241         x = tmpOne - numpy.matmul(numpy.matmul(numpy.matmul(D, KAA), D), alpha) - numpy.matmul(D, tmpOne) * b
242         p = numpy.array(list(self.__p(x[i, 0], beta) for i in range(x.shape[0])))
243         J2 = numpy.sum(p * p) * c / 2
244         return J2
245         
246     
247     def __calc_J1(self, alpha):
248         J1 = numpy.sum(alpha * alpha) / 2
249         return J1
250         
251     
252     def __get_KAA(self, A, mu):
253         KAA = numpy.zeros((A.shape[1], A.shape[1]))
254         for rowIdx in range(KAA.shape[0]):
255             for colIdx in range(rowIdx + 1):
256                 x1 = A[:, rowIdx:rowIdx+1]
257                 x2 = A[:, colIdx:colIdx+1]
258                 val = self.__calc_gaussian(x1, x2, mu)
259                 KAA[rowIdx, colIdx] = KAA[colIdx, rowIdx] = val
260         return KAA
261     
262     
263     def __init_alpha_b(self, shape):
264         ‘‘‘
265         alpha, b之初始化
266         ‘‘‘
267         alpha, b = numpy.zeros(shape), 0
268         return alpha, b
269         
270         
271     def __get_KAx(self, A, x, mu):
272         KAx = numpy.zeros((A.shape[1], 1))
273         for rowIdx in range(KAx.shape[0]):
274             x1 = A[:, rowIdx:rowIdx+1]
275             val = self.__calc_gaussian(x1, x, mu)
276             KAx[rowIdx, 0] = val
277         return KAx
278         
279         
280     def __calc_hVal(self, KAx, D, alpha, b):
281         hVal = numpy.matmul(numpy.matmul(alpha.T, D), KAx)[0, 0] + b
282         return hVal
283         
284         
285     def __calc_gaussian(self, x1, x2, mu):
286         val = numpy.exp(-mu * numpy.linalg.norm(x1 - x2) ** 2)
287         # val = numpy.sum(x1 * x2)
288         return val
289         
290         
291     def __get_AD(self):
292         A = self.__trainingSet[:, :2].T
293         D = numpy.diag(self.__trainingSet[:, 2])
294         return A, D
295         
296 
297 class SpiralPlot(object):
298     
299     def spiral_data_plot(self, trainingData0, trainingData1, testData0, testData1):
300         fig = plt.figure(figsize=(5, 5))
301         ax1 = plt.subplot()
302         ax1.scatter(trainingData1[:, 0], trainingData1[:, 1], c="red", marker="o", s=10, label="training data - Positive")
303         ax1.scatter(trainingData0[:, 0], trainingData0[:, 1], c="blue", marker="o", s=10, label="training data - Negative")
304         ax1.scatter(testData1[:, 0], testData1[:, 1], c="red", marker="x", s=10, label="test data - Positive")
305         ax1.scatter(testData0[:, 0], testData0[:, 1], c="blue", marker="x", s=10, label="test data - Negative")
306         ax1.set(xlim=(-0.5, 0.5), ylim=(-0.5, 0.5), xlabel="$x_1$", ylabel="$x_2$")
307         plt.legend(fontsize="x-small")
308         fig.tight_layout()
309         fig.savefig("spiral.png", dpi=100)
310         plt.close()
311         
312         
313     def spiral_pred_plot(self, trainingData0, trainingData1, testData0, testData1, ssvmObj, alpha, b):
314         x = numpy.linspace(-0.5, 0.5, 500)
315         y = numpy.linspace(-0.5, 0.5, 500)
316         x, y = numpy.meshgrid(x, y)
317         z = numpy.zeros(shape=x.shape)
318         for rowIdx in range(x.shape[0]):
319             for colIdx in range(x.shape[1]):
320                 z[rowIdx, colIdx] = ssvmObj.get_cls((x[rowIdx, colIdx], y[rowIdx, colIdx]), alpha, b)
321         cls2color = {-1: "blue", 0: "white", 1: "red"}
322         
323         fig = plt.figure(figsize=(5, 5))
324         ax1 = plt.subplot()
325         ax1.contourf(x, y, z, levels=[-1.5, -0.5, 0.5, 1.5], colors=["blue", "white", "red"], alpha=0.3)
326         ax1.scatter(trainingData1[:, 0], trainingData1[:, 1], c="red", marker="o", s=10, label="training data - Positive")
327         ax1.scatter(trainingData0[:, 0], trainingData0[:, 1], c="blue", marker="o", s=10, label="training data - Negative")
328         ax1.scatter(testData1[:, 0], testData1[:, 1], c="red", marker="x", s=10, label="test data - Positive")
329         ax1.scatter(testData0[:, 0], testData0[:, 1], c="blue", marker="x", s=10, label="test data - Negative")
330         ax1.set(xlim=(-0.5, 0.5), ylim=(-0.5, 0.5), xlabel="$x_1$", ylabel="$x_2$")
331         plt.legend(loc="upper left", fontsize="x-small")
332         fig.tight_layout()
333         fig.savefig("pred.png", dpi=100)
334         plt.close()
335 
336 
337 
338 if __name__ == "__main__":
339     ssvmObj = SSVM(trainingSet, c=0.1, mu=250, beta=100)
340     alpha, b, tab = ssvmObj.optimize()
341     accuracy1 = ssvmObj.get_accuracy(trainingSet, alpha, b)
342     print("Accuracy on trainingSet is {}%".format(accuracy1 * 100))
343     accuracy2 = ssvmObj.get_accuracy(testSet, alpha, b)
344     print("Accuracy on testSet is {}%".format(accuracy2 * 100))
345     
346     spObj = SpiralPlot()
347     spObj.spiral_data_plot(trainingData0, trainingData1, testData0, testData1)
348     spObj.spiral_pred_plot(trainingData0, trainingData1, testData0, testData1, ssvmObj, alpha, b)

View Code

笔者所用训练集、测试集数据分布如下:
技术图片

很显然, 此数据集非线性可分.

结果展示:
此模型在训练集、测试集上的准确率均达到100%.
使用建议:
①. 通过核函数映射至高维后, 支持向量可能存在很多个;
②. SSVM本质上求解的仍然是原问题, 此时$\alpha$仅作为变数变换引入, 无需满足KKT条件中对偶可行性;
③. 数据集是否需要归一化, 应按实际情况予以考虑.
参考文档:
Yuh-Jye Lee and O. L. Mangasarian. "SSVM: A Smooth Support Vector Machine for Classification", Computational Optimization and Applications, 20, (2001) 5-22.

Smooth Support Vector Machine - Python实现

标签：== closed spi matrix event ref ret pac 技术

原文地址：https://www.cnblogs.com/xxhbdk/p/12275567.html

踩

(0)

评论一句话评论（0）

分享档案

更多>

2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)

周排行