基于有序編碼的核極限學(xué)習(xí)順序回歸模型
doi: 10.11999/JEIT170765 cstr: 32379.14.JEIT170765
基金項(xiàng)目:
國家自然科學(xué)基金(71110107026, 71331005, 91546201, 11671379, 111331012)，中國科學(xué)院大學(xué)資助項(xiàng)目(Y55202LY00)
Ordered Code-based Kernel Extreme Learning Machine for Ordinal Regression
Funds:
The National Natural Science Foundation of China (71110107026, 71331005, 91546201, 11671379, 111331012), The Grant of University of Chinese Academy of Sciences (Y55202LY00)
-
摘要: 順序回歸是機(jī)器學(xué)習(xí)領(lǐng)域中介于分類和回歸之間的有監(jiān)督問題。在實(shí)際中，許多帶有序關(guān)系標(biāo)簽的問題都可以被建模成順序回歸問題，因此順序回歸受到眾多學(xué)者的關(guān)注?；跇O限學(xué)習(xí)機(jī)(ELM)的算法能有效避免因迭代過程陷入的局部最優(yōu)解，減少訓(xùn)練時(shí)間，但基于極限學(xué)習(xí)機(jī)的算法在順序回歸問題上的研究較少。該文將核極限學(xué)習(xí)機(jī)與糾錯(cuò)輸出編碼相結(jié)合，提出了一種基于有序編碼的核極限學(xué)習(xí)順序回歸模型。該模型有效解決了如何在順序回歸中取得良好的特征映射以及如何避免傳統(tǒng)極限學(xué)習(xí)機(jī)中隱層節(jié)點(diǎn)個(gè)數(shù)依賴于人工設(shè)置的問題。為驗(yàn)證提出模型的有效性，該文在多個(gè)順序回歸數(shù)據(jù)集上進(jìn)行了測試，測試結(jié)果表明，相比于傳統(tǒng)ELM模型，該文提出的模型在準(zhǔn)確率上平均提升了10.8%，在數(shù)據(jù)集上預(yù)測表現(xiàn)最優(yōu)，而且獲得了最短的訓(xùn)練時(shí)間，從而驗(yàn)證了模型的有效性。
-
關(guān)鍵詞:
- 糾錯(cuò)輸出編碼 /
- 順序回歸 /
- 極限學(xué)習(xí)機(jī) /
- 核函數(shù)
Abstract: Ordinal regression is one of the supervised learning issues, which resides between classification and regression in machine learning fields. There exist many real problems in practice, which can be modeled as ordinal regression problems due to the ordering information between labels. Therefore ordinal regression has received increasing interest by many researchers recently. The Extreme Learning Machine (ELM)-based algorithms are easy to train without iterative algorithm and they can avoid the local optimal solution; meanwhile they reduce the training time compared with other learning algorithms. However, the ELM-based algorithms which are applied to ordinal regression have not been exploited much. This paper proposes a new ordered code-based kernel extreme learning ordinal regression machine to fill this gap, which combines the kernel ELM and error correcting output codes effectively. The model overcomes the problems of how to get high quality feature mappings in ordinal regression and how to avoid setting the number of hidden nodes by manual. To validate the effectiveness of this model, numerous experiments are conducted on a lot of datasets. The experimental results show that the model can improve the accuracy by 10.8% on average compared with traditional ELM-based algorithms and achieve the state- of-the-art performance with the least time. -
NAKOV P, RITTER A, ROSENTHAL S, et al. SemEval- 2016 task 4: Sentiment analysis in Twitter[C]. International Workshop on Semantic Evaluation, San Diego, USA, 2016: 1-18. doi: 10.18653/v1/S16-1028. TIAN Q, CHEN S, and TAN X. Comparative study among three strategies of incorporating spatial structures to ordinal image regression[J]. Neurocomputing, 2014, 136: 152-161. doi: 10.1016/j.neucom.2014.01.017. CORRENTE S, DOUMPOS M, GRECO S, et al. Multiple criteria hierarchy process for sorting problems based on ordinal regression with additive value functions[J]. Annals of Operations Research, 2017, 251(1/2): 117-139. doi: 10.1007/ s10479-015-1898-1. GUTIRREZ P A, PREZ-ORTIZ M, SANCHEZ- MONEDERO J, et al. Ordinal regression methods: Survey and experimental study[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(1): 127-146. doi: 10.1109/ TKDE.2015.2457911. HUANG G B, ZHU Q Y, and SIEW C K. Extreme learning machine: Theory and applications[J]. Neurocomputing, 2006, 70(1): 489-501. doi: 10.1016/j.neucom.2005.12.126. RAJASEKARAN S and PAI G A V. Neural Networks, Fuzzy Systems and Evolutionary Algorithms: Synthesis and Applications[M]. Haryana, India: Rajkamal Electric Press, 2017: 151-168. CHU W and KEERTHI S S. Support vector ordinal regression[J]. Neural Computation, 2007, 19(3): 792-815. doi: 10.1162/neco.2007.19.3.792. HUANG G B, ZHOU H, DING X, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(2): 513-529. doi: 10.1109/TSMCB. 2011.2168604. UCAR A, DEMIR Y, and GZELI C. A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering[J]. Neural Computing and Applications, 2016, 27(1): 131-142. doi: 10.1007/s00521-014-1569-1. 徐濤, 郭威, 呂宗磊. 基于快速極限學(xué)習(xí)機(jī)和差分進(jìn)化的機(jī)場噪聲預(yù)測模型[J]. 電子與信息學(xué)報(bào), 2016, 38(6): 1512-1518. doi: 10.11999/JEIT150986. XU Tao, GUO Wei, and L Zonglei. Prediction model of airport noise based on fast extreme learning machine and differential evolution[J]. Journal of Electronics Information Technology, 2016, 38(6): 1512-1518. doi: 10.11999/JEIT 150986. GOODFELLOW I, BENGIO Y, and COURVILLE A. Deep Learning[M]. Massachusetts, USA, MIT Press, 2016: 165-480. doi: 10.1038/nature14539. DENG W Y, ZHENG Q H, LIAN S, et al. Ordinal extreme learning machine[J]. Neurocomputing, 2010, 74(1): 447-456. doi: 10.1016/j.neucom.2010.08.022. RICCARDI A, FERNNDEZ-NAVARRO F, and CARLONI S. Cost-sensitive AdaBoost algorithm for ordinal regression based on extreme learning machine[J]. IEEE Transactions on Cybernetics, 2014, 44(10): 1898-1909. doi: 10.1109/TCYB. 2014.2299291. HORNIK K, STINCHCOMBE M, and WHITE H. Multilayer feedforward networks are universal approximators[J]. Neural Networks, 1989, 2(5): 359-366. doi: 10.1016/0893-6080(89) 90020-8. HUANG G B and BABRI H A. Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions[J]. IEEE Transactions on Neural Networks, 1998, 9(1): 224-229. doi: 10.1109/72.655045. HUANG G B, CHEN L, and SIEW C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes[J]. IEEE Transactions on Neural Networks, 2006, 17(4): 879-892. doi: 10.1109/TNN. 2006.875977. HUANG G B. Learning capability and storage capacity of two-hidden-layer feedforward networks[J]. IEEE Transactions on Neural Networks, 2003, 14(2): 274-281. doi: 10.1109/TNN.2003.809401. BARTLETT P L. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network[J]. IEEE Transactions on Information Theory, 1998, 44(2): 525-536. doi: 10.1109/18.661502. TANG J, DENG C, and HUANG G B. Extreme learning machine for multilayer perceptron[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 809-821. doi: 10.1109/TNNLS.2015.2424995. HOERL A E and KENNARD R W. Ridge regression: Biased estimation for nonorthogonal problems[J]. Technometrics, 1970, 12(1): 55-67. doi: 10.1080/00401706.1970.10488634. ALLWEIN E L, SCHAPIRE R E, and SINGER Y. Reducing multiclass to binary: A unifying approach for margin classifiers[J]. Journal of Machine Learning Research, 2000, 1(12): 113-141. doi: 10.1162/15324430152733133. 雷蕾, 王曉丹, 羅璽, 等. ECOC多類分類研究綜述[J]. 電子學(xué)報(bào), 2014, 42(9): 1794-1800. doi: 10.3969/j.issn.0372-2112. 2014.09.020. LEI Lei, WANG Xiaodan, LUO Xi, et al. An overview of multi-classification based on error-correcting output codes[J]. Acta Electronica Sinica, 2014, 42(9): 1794-1800. doi: 10.3969 /j.issn.0372-2112.2014.09.020. HUANG G, HUANG G B, SONG S, et al. Trends in extreme learning machines: A review[J]. Neural Networks, 2015, 61: 32-48. doi: 10.1016/j.neunet.2014.10.001. LIU Q, HE Q, and SHI Z. Extreme support vector machine classifier[C]. 12th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Osaka, Japan, 2008: 222-233. doi: 10.1007/978-3-540-68125-0_21. FRNAY B and VERLEYSEN M. Using SVMs with randomised feature spaces: an extreme learning approach[C]. European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, 2010: 315-320. HUANG G B, DING X, and ZHOU H. Optimization method based extreme learning machine for classification[J]. Neurocomputing, 2010, 74(1): 155-163. doi: 10.1016/j.neucom. 2010.02.019. CHU W and GHAHRAMANI Z. Gaussian processes for ordinal regression[J]. Journal of Machine Learning Research, 2005, 6(7): 1019-1041. BACCIANELLA S, ESULI A, and SEBASTIANI F. Evaluation measures for ordinal regression[C]. The Ninth International Conference on Intelligent Systems Design and Applications, Pisa, Italy, 2009: 283-287. doi: 10.1109/ISDA. 2009.230. -
計(jì)量
- 文章訪問數(shù): 1610
- HTML全文瀏覽量: 169
- PDF下載量: 175
- 被引次數(shù): 0
下載:
