Journal of Zhejiang University SCIENCE A
(Monthly)

2006   Vol. 7   No. 11   p. 1942-1947

SVD-LSSVM and its application in chemical pattern classification

TAO Shao-hui, CHEN De-zhao^†‡, HU Wang-ming

(Department of Chemical Engineering, Zhejiang University, Hangzhou 310027, China)
^‡ Corresponding Author
^†E-mail: dzc@zju.edu.cn
Received May 28, 2006 revision accepted July 28, 2006

Abstract: Pattern classification is an important field in machine learning; least squares support vector machine (LSSVM) is a powerful tool for pattern classification. A new version of LSSVM, SVD-LSSVM, to save time of selecting hyper parameters for LSSVM is proposed. SVD-LSSVM is trained through singular value decomposition (SVD) of kernel matrix. Cross validation time of selecting hyper parameters can be saved because a new hyper parameter, singular value contribution rate (SVCR), replaces the penalty factor of LSSVM. Several UCI benchmarking data and the Olive classification problem were used to test SVD-LSSVM. The result showed that SVD-LSSVM has good performance in classification and saves time for cross validation.

Key words: Pattern classification, Structural risk minimization, Least squares support vector machine (LSSVM), Hyper parameter selection, Cross validation, Singular value decomposition (SVD)
doi:10.1631/jzus.2006.A1942             CLC number: TP183

References:

[1] Blake, C.L., Merz, C.J., 1998. UCI Repository of Machine Learning Database. Http://www.ics.uci.edu/~mlearn/ mlrepository.html. Dept. of Information and Computer Science, University of California, Irvine, CA.

[2] Golub, G.H., van Loan, C.F., 1989. Matrix Computations. Gene Johns Hopkins University Press.

[3] Hopke, P.K., Massart, D.L., 1993. Reference data sets for chemometrical methods testing. Chemometrics and Intelligent Laboratory Systems, 19(1):35-41.

[4] Pelckmans, K., de Brabanter, J., Suykens, J.A.K., de Moor, B., 2005. The differogram: Nonparametric noise variance estimation and its use for model. Neurocomputing, Special Issue on Signal Processing, 69(1-3):100-122.

[5] Pelckmans, K., Suykens, J.A.K., de Moor, B., 2006. Additive regularization trade-off: Fusion of training and validation levels in kernel methods. Machine Learning, 62(3):217- 252.

[6] Suykens, J.A.K., Vandewalle, J., 1999a. Least squares support vector machine classifiers. Neural Processing Letters, 9(3):293-300.

[7] Suykens, J.A.K., Vandewalle, J., 1999b. Multiclass Least Squares Support Vector Machines. Intl. Joint Conference on Neural Networks, IJCNN’99, Washington, D.C.

[8] van Gestel, T., Suykens, J.A.K., Baesens, B., Stijn, V., Vanthienen, J., Dedene, G., Bart, D.M., Vandewalle, J., 2004. Benchmarking least squares support vector machine classifiers. Machine Learning, 54(1):5-32.