融合改進(jìn)二元螢火蟲(chóng)算法和互補(bǔ)性測(cè)度的集成剪枝方法
doi: 10.11999/JEIT170984 cstr: 32379.14.JEIT170984
-
①(合肥工業(yè)大學(xué)管理學(xué)院 合肥 230009) ②(過(guò)程優(yōu)化與智能決策教育部重點(diǎn)實(shí)驗(yàn)室 合肥 230009) ③(湖州師范學(xué)院商學(xué)院 湖州 313000) ④(安徽財(cái)經(jīng)大學(xué)管理科學(xué)與工程學(xué)院 蚌埠 233030)
基金項(xiàng)目:
國(guó)家自然科學(xué)基金(91546108, 71271071, 71490725, 71301041)，國(guó)家重點(diǎn)研發(fā)計(jì)劃(2016YFF0202604)，過(guò)程優(yōu)化與智能決策教育部重點(diǎn)實(shí)驗(yàn)室開(kāi)放課題
Improved Binary Glowworm Swarm Optimization Combined with Complementarity Measure for Ensemble Pruning
-
ZHU Xuhui①② NI Zhiwei①② NI Liping①② JIN Feifei①② CHENG Meiying③ LI Jingming④
Funds:
The National Natural Science Foundation of China (91546108, 71271071, 71490725, 71301041), The National Key Research and Development Plan (2016YFF0202604), Open Research Fund Program of Key Laboratory of Process Optimization and Intelligent Decision-making
-
摘要: 差異性和平均精度是提高分類(lèi)器集成性能的兩個(gè)重要指標(biāo)。增加差異性勢(shì)必會(huì)降低平均精度，增大平均精度一定會(huì)減小差異性。故在差異性和平均精度之間存在一個(gè)平衡狀態(tài)，使得集成性能最優(yōu)。為了尋找該平衡狀態(tài)，該文提出融合改進(jìn)二元螢火蟲(chóng)算法和互補(bǔ)性測(cè)度的集成剪枝方法。首先，采用bootstrap抽樣方法獨(dú)立訓(xùn)練出多個(gè)基分類(lèi)器，構(gòu)建原始基分類(lèi)器池。其次，采用互補(bǔ)性測(cè)度對(duì)原始基分類(lèi)器池進(jìn)行預(yù)剪枝。接著，通過(guò)改進(jìn)螢火蟲(chóng)的移動(dòng)方式和搜索過(guò)程，引入重新初始化機(jī)制和跳躍行為，提出改進(jìn)二元螢火蟲(chóng)算法。最后，采用改進(jìn)二元螢火蟲(chóng)算法對(duì)預(yù)剪枝后的基分類(lèi)器，進(jìn)行進(jìn)一步剪枝，選擇出集成性能最優(yōu)的基分類(lèi)器子集合。在5個(gè)UCI數(shù)據(jù)集上的實(shí)驗(yàn)結(jié)果表明，較其他方法，使用較少的基分類(lèi)器，獲得了更優(yōu)的集成性能，具有良好的有效性和顯著性。
-
關(guān)鍵詞:
- 螢火蟲(chóng)算法 /
- 互補(bǔ)性測(cè)度 /
- 集成剪枝
Abstract: The key to the success of an ensemble system are the diversity and the average accuracy of base classifiers. The increase of diversity among base classifiers will lead to the decrease of the average accuracy, and vice versa. So there exists a tradeoff between the diversity and the average accuracy, which makes the ensemble perform the best with respect to ensemble pruning. To find the tradeoff, Improved Binary Glowworm Swarm Optimization combined with Complementarity measure for Ensemble Pruning (IBGSOCEP) is proposed. Firstly, an initial pool of classifiers is constructed through training independently some base classifiers using bootstrap sampling. Secondly, the classifiers in the initial pool are pre-pruned using complementarity measure. Thirdly, Improved Binary Glowworm Swarm Optimization (IBGSO) is proposed by improving moving way, searching processes of glowworm, introducing re-initialization, and leaping behaviors. Finally, the optimal sub-ensemble is achieved from the base classifiers after pre-pruning using IBGSO. Experimental results on 5 UCI datasets demonstrate that IBGSODSEN can achieve better results than other approaches with less number of base classifiers, and that its effectiveness and significance. -
BASHBAGHI S, GRANGER E, SABOURIN R, et al. Dynamic ensembles of exemplar-SVMs for still-to-video face recognition[J]. Pattern Recognition, 2017, 69(C): 61-81. doi: 10.1016/j.patcog.2017.04.014. LIU Jiachen, MIAO Qiguang, CAO Ying, et al. Ensemble one-class classifiers based on hybrid diversity generation and pruning[J]. Journal of Electronics & Information Technology, 2015, 37(2): 386-393. doi: 10.11999/JEIT140161. [3] LI Kai, XING Junliang, HU Weiming, et al. D2C: Deep cumulatively and comparatively learning for human age estimation[J]. Pattern Recognition, 2017, 66(6): 95-105. doi: 10.1016/j.patcog.2017.01.007. [4] LU Huijuan, AN Chunlin, ZHENG Enhui, et al. Dissimilarity based ensemble of extreme learning machine for gene expression data classification[J]. Neurocomputing, 2014, 128(5): 22-30. doi: 10.1016/j.neucom.2013.02.052. YANG Chun, YIN Xucheng, HAO Hongwei, et al. Classifier ensemble with diversity: Effectiveness analysis and ensemble optimization[J]. Acta Automatica Sinica, 2014, 40(4): 660-674. doi: 10.3724/SP.J.1004.2014.00660. [6] YKHLEF H and BOUCHAFFRA D. An efficient ensemble pruning approach based on simple coalitional games[J]. Information Fusion, 2017, 34(C): 28-42. doi: 10.1016/j.inffus. 2016.06.003. [7] ZHOU Zhihua, WU Jianxin, and TANG Wei. Ensembling neural networks: many could be better than all[J]. Artificial Intelligence, 2002, 137(1): 239-263. doi: 10.1016/S0004- 3702(02)00190-X. [8] MARTÍNEZ-MUÑOZ G, HERNANDEZ LOBATO D, and SUAREZ A. An analysis of ensemble pruning techniques based on ordered aggregation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 245-259. doi: 10.1109/TPAMI.2008.78. [9] GUO Li and BOUKIR S. Margin-based ordered aggregation for ensemble pruning[J]. Pattern Recognition Letters, 2013, 34(6): 603-609. doi: 10.1016/j.patrec.2013.01.003. [10] DAI Qun, ZHANG Ting, and LIU Ningzhong. A new reverse reduce-error ensemble pruning algorithm[J]. Applied Soft Computing, 2015, 28(3): 237-249. doi: 10.1016/j.asoc.2014. 10.045. [11] ROKACH L. Collective-agreement-based pruning of ensembles[J]. Computational Statistics and Data Analysis, 2009, 53(4): 1015-1026. doi: 10.1016/j.csda.2008.12.001. [12] LAZAREVIC A and OBRADOVIC Z. Effective pruning of neural network classifier ensembles[C]. International Joint Conference on Neural Networks, Washington DC, USA, 2001: 796-801. doi: 10.1109/IJCNN.2001.939461. [13] GIACINTO G, ROLI F, and FUMERA G. Design of effective multiple classifier systems by clustering of classifiers[C]. International Conference on Pattern Recognition, Barcelona, Spain, 2000: 160-163. doi: 10.1109/ICPR.2000.906039. [14] BAKKER B and HESKES T. Clustering ensembles of neural network models[J]. Neural Network, 2003, 16(2): 261-269. doi: 10.1016/S0893-6080(02)00187-9. [15] ZHOU Hongfang, ZHAO Xuehan, and WANG Xiao. An effective ensemble pruning algorithm based on frequent patterns[J]. Knowledge-Based Systems, 2014, 56(C): 79-85. doi: 10.1016/j.knosys.2013. 10.024. [16] CAVALCANTI G D C, OLIVEIRA L S, NOURA T J M, et al. Combining diversity measures for ensemble pruning[J]. Pattern Recognition Letters, 2016, 74(C): 38-45. doi: 10.1016 /j.patrec.2016.01.029. NI Zhiwei, ZHANG Chen, and NI Liping. Haze forecast method of selective ensemble based on glowworm swarm optimization algorithm[J]. Pattern Recognition and Artificial Intelligence, 2016, 29(2): 143-153. doi: 10.16451/j.cnki. issn1003-6059.201602006. [18] MARINAKI M and MARINAKI Y. A glowworm swarm optimization algorithm for the vehicle routing problem with stochastic demands[J]. Expert Systems with Applications, 2016, 46(C): 145-163. doi: 10.1016/j.eswa.2015.10.012. [19] BREIMAN L. Bagging predictors[J]. Machine Learning, 1996, 24(2): 123-140. doi: 10.1023/A:1018054314350. [20] SINGHAL P K, NARESH R, and SHARMA V. Binary fish swarm algorithm for profit-based unit commitment problem in competitive electricity market with ramp rate constraints [J]. IET Generation, Transmission & Distribution, 2015, 9(13): 1697-1707. doi: 10.1049/iet-gtd.2015.0201. -
計(jì)量
- 文章訪問(wèn)數(shù): 1253
- HTML全文瀏覽量: 238
- PDF下載量: 56
- 被引次數(shù): 0
下載:
