原空间中最小二乘支持向量机的新算法

Journal of Systems Engineering and Electronics ›› 2009, Vol. 31 ›› Issue (1): 142-145,237.

原空间中最小二乘支持向量机的新算法

赵永平, 孙健国

南京航空航天大学能源与动力学院, 江苏, 南京, 210016

收稿日期:2007-09-06 修回日期:2008-03-26 出版日期:2009-01-20 发布日期:2010-01-03
作者简介:赵永平(1982- ),男,博士研究生,主要研究方向为机器学习,控制理论和控制方法.E-mail:Y.P.zhao@nuaa.edu.cn
基金资助:
国家自然科学基金资助课题(50576033)

New method for least squares support vector machine in the primal

ZHAO Yong-ping, SUN Jian-guo

Coll. of Energy and Power Engineering, Nanjing Univ. of Aeronautics and Astronautics, Nanjing 210016, China

Received:2007-09-06 Revised:2008-03-26 Online:2009-01-20 Published:2010-01-03

摘要/Abstract

摘要： 为了解决原空间中最小二乘支持向量机的解缺乏稀疏性的缺点,提出了Pruning法、MFCV法和IMFCV法并对BDFS法进行了修改和运用。对一个不含有奇异点的系统而言,Pruning法、BDFS法和MFCV法在一定程度上都能实现原空间中最小二乘支持向量机解的稀疏性。BDFS法无论是训练时间还是预测时间都比Pruning法短;和MFCV法比起来,虽然BDFS法的训练时间短,但比MFCV的预测时间长。对一个含有奇异点的系统而言,Pruning法几乎失去了效用;虽然BDFS和MFCV法的训练时间都比IMFCV法的训练时间短,但IMFCV法能成功抑制奇异点从而缩短预测时间。

Abstract: In order to solve the shortcoming of lacking sparseness in the solution of least squares support vector machine in the primal,the pruning method,multi-fold cross validation(MFCV) method and improved multi-fold cross validation(IMFCV) method are proposed and the backward deletion feature selection(BDFS) method is modified and applied.These methods all realize the sparseness of the least squares support vector machine to a certain degree in the primal for a system without outliers.The predicted time of BDFS is shorter than the pruning’s;although the training time of BDFS is shorter than MFCV’s,its predicted time is longer.In the face of a system with outliers,the pruning almost loses effectiveness,and the training time of BDFS and MFCV are both shorter than IMFCV’s,but IMFCV can oppress outliers and reduces the predicted time remarkably.

中图分类号:

TP301

赵永平, 孙健国. 原空间中最小二乘支持向量机的新算法[J]. Journal of Systems Engineering and Electronics, 2009, 31(1): 142-145,237.

ZHAO Yong-ping, SUN Jian-guo. New method for least squares support vector machine in the primal[J]. Journal of Systems Engineering and Electronics, 2009, 31(1): 142-145,237.

参考文献

[1] Vapnik V N.The nature of statistical theory[M].New York:Springer-Verlag,1995.
[2] Polat K,Günes S,Arslan A.A cascade learning system for classification of diabetes disease:generalized discriminant analysis and least square support vector machine[J].Expert Systems with Applications,2008,34(1):482-487.
[3] Hua Z S,Wang Y,Xu X Y,et al.Predicting corporate financial distress based on integration of support vector machine and logistic regression[J].Expert Systems with Applications,2007,33 (2):434-440.
[4] Chen X R,Jing Z L,Xiao G.Nonlinear fusion for face recognit ion using fuzzy integral[J].Communicationsin Nonlinear Science and Numerical Simulation,2007,12:823-831.
[5] Huang C L,Chen M C,Wang C J.Credit scoring with a data mining approach based on support vector machines[J].Expert Systems with Applications,2007,33(4):847-856.
[6] Suykens J A K,Vandewalle J.Least squares support vector machines classifiers[J].Neural Processing Letters,1999,9 (3):293-300.
[7] Suykens J A K,Brabanter J D,Lukas L.Weighted least squares support vector machine:robustness and sparse approximation[J].Neurocomputing,2002,48(1):85-105.
[8] Olivier C.Training a support vector machine in the primal[J].Neural Computation,2007,19 (5):1155-1178.
[9] Abe S.Sparse least squares support vector training in the reduced empirical feature space[J].Pattern Analysis ＆ Applications,2007,10(3):203-214.
[10] Xiong H L,Swamy M N S,Ahmad M O.Optimizing the kernel in the empirical feature space[J].IEEE Transactions on Neural Networks,2005,16(2).-460-474.
[11] Suykens J A K,Lukas L,Vandewalle J.Sparse approximation using least squares support vector machines[C]//Proc.of IEEE International Symposium on Circuits and Systems,2000:757-760.
[12] Kruif B J,Vries J A.Pruning error minimization in least squares support vector machines[J].IEEE Transactions on Neural Networks,2003,14(3):696-702.
[13] Hoegaerts L,Suykens J A K,Vandewalle J,et al.A comparison of pruning algorithms for sparse least squares support vector machines[C]//Proc.of International Conference on Neural Information Processing,2004:1247-1253.
[14] Espinoza M,Suykens J A K,Moor B D.Fixed-size least squares support vector machines:a large scale application in electrical load forecasting[J].Computational Management Science,2006,3(2):113-129.
[15] 赵永平,孙健国.一类非平坦函数的多核最小二乘支持向量机的鲁棒回归算法[J].信息与控制,2008,37(2):160-165.
[16] Wang L,Bo L F,Jiao L C.Sparse kernel ridge regression using backward deletion[C]//Proc.of Lecture Notes in Artificial Intelligence,2006:265-374.