可遷移測度準(zhǔn)則下的協(xié)變量偏移修正多源集成方法

楊興明; 吳克偉; 孫永宣; 謝昭

doi:10.11999/JEIT150323

可遷移測度準(zhǔn)則下的協(xié)變量偏移修正多源集成方法

doi: 10.11999/JEIT150323 cstr: 32379.14.JEIT150323

基金項(xiàng)目:

國家自然科學(xué)基金(60905005, 61273237)

計(jì)量
- 文章訪問數(shù): 1386
- HTML全文瀏覽量: 185
- PDF下載量: 673
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2015-03-17
- 修回日期: 2015-08-13
- 刊出日期: 2015-12-19

Modified Covariate-shift Multi-source Ensemble Method in Transferability Metric

Funds:

The National Natural Science Foundation of China (60905005, 61273237)

摘要

摘要: 遷移學(xué)習(xí)通過充分利用源域共享知識(shí)，實(shí)現(xiàn)對目標(biāo)域的小樣本問題求解，然而，對訓(xùn)練和測試樣本分布差異測度仍然是該領(lǐng)域的主要挑戰(zhàn)。該文針對多源遷移學(xué)習(xí)算法中，由于源域選擇和源域輔助樣本選擇不當(dāng)引起的負(fù)遷移問題進(jìn)行研究，提出一種可遷移測度準(zhǔn)則下的協(xié)變量偏移修正多源集成方法。首先，根據(jù)源域和目標(biāo)域之間的協(xié)變量偏移原則，利用聯(lián)合概率的密度估計(jì)，定義輔助樣本的可遷移測度，驗(yàn)證目標(biāo)域和源域在數(shù)據(jù)空間中標(biāo)記分布的一致性。其次，在多源域選擇階段，引入非遷移判別過程，提高了源域知識(shí)的遷移準(zhǔn)確性。最后，在Caltech 256數(shù)據(jù)集中，驗(yàn)證了Gist特征知識(shí)表示和遷移的有效性，分析了多種條件下的輔助樣本選擇和源域選擇的有效性。實(shí)驗(yàn)結(jié)果表明所提算法可有效降低負(fù)遷移現(xiàn)象的發(fā)生，獲得更好的遷移學(xué)習(xí)性能
- 集成學(xué)習(xí) /
- 遷移學(xué)習(xí) /
- 協(xié)變量偏移 /
- 圖像分類
Abstract: Transfer learning usually focuses on dealing with small training set in target domain by sharing knowledge generated from source ones, in which one main challenge is divergence metric of distributed samples between training and test data. In order to deal with negative transfer problem caused by improper auxiliary sample selections in source domains, this paper presents a modified covariate-shift multi-source ensemble method with transferability criterion. Firstly, transferability metric of auxiliary samples is defined by joint density estimation in accordance with co-variant transfer principles from source to target, so that the coherency of data distributions is verified. After that, whether transfer learning occurs or not should be determined after evaluating transferability metric in different sources to boost accuracy. Finally, experiments on Caltech256 using GIST demonstrate effectiveness and efficiency in the proposed approach and discussions of performance under diverse selections from auxiliary samples and source domains are presented as well. Experimental results show that the proposed method can sufficiently hold back negative transfer for better learnability in transfer style.
- Ensemble learning /
- Transfer learning /
- Covariate shift /
- Image classification

HTML全文

參考文獻(xiàn)(32)

Tommasi T, OrabonaF, and Caputo B. Learning categories from few examples with multi model knowledge transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 928941.

Yao Y and Doretto G. Boosting for transfer learning with multiple sources[C]. Proceedings of Computer Vision and Pattern Recognition, San Francisco, 2010: 18551862.

Long M S, Wang J M, Ding G G, et al.. Adaptation regularization a general framework for transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5): 10761089.

Lin D, An X, and Zhang J. Double-bootstrapping source data selection for instance-based transfer learning[J]. Pattern Recognition Letters, 2013, 34(11): 12791285.

Kuzborskij I, Orabona F, and Caputo B. From N to N+1: multiclass transfer incremental learning[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 33583365.

Zhu Y, Chen Y Q, Lu Z Q, et al.. Heterogeneous transfer learning for image classification[C]. Proceedings of AAAI Conference on Artificial Intelligence, San Francisco, 2011:　13041309.

張倩, 李明, 王雪松, 等. 一種面向多源領(lǐng)域的實(shí)例遷移學(xué)習(xí)[J]. 自動(dòng)化學(xué)報(bào), 2014, 40(6): 1176-1183.

Zhang Qian, Li Ming, Wang Xue-song, et al.. Instance-based transfer learning for multi-source domains[J]. Acta Automatica Sinica, 2014, 40(6): 11761183.

Pang J, Huang Q, Yan S, et al.. Transferring boosted detectors towards viewpoint and scene adaptiveness[J]. IEEE Transactions on Image Processing, 2011, 20(5): 13881400.

Li G, Qin L, Huang Q, et al.. Treat samples differently: object tracking with semi-supervised online CovBoost[C]. Proceedings of International Conference on Computer Vision, Barcelona, 2011: 627634.

Qi G J, Aggarwal C, Rui Y, et al.. Towards cross-category knowledge propagation for learning visual concepts[C]. Proceedings of Computer Vision and Pattern Recognition, Colorado Springs, 2011: 897904.

Chu W S, Torre F D, and CohnJ F. Selective transfer machine for personalized facial action unit detection[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 35153522.

Yang S Z, Hou C P, Zhang C S, et al.. Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning[J]. Neural Computing and Applications, 2013, 23(2): 541559.

方耀寧, 郭云飛, 丁雪濤, 等. 一種基于標(biāo)簽遷移學(xué)習(xí)的改進(jìn)正則化奇異值分解推薦算法[J]. 電子與信息學(xué)報(bào), 2013, 35(12): 30463050.

Fang Yao-ning, Guo Yun-fei, Ding Xue-tao, et al.. An improved regularized singular value decomposition recommender algorithm based on tag transfer learning[J]. Journal of Electronics Information Technology, 2013, 35(12): 30463050.

Gopalan R. Learning cross-domain information transfer for location recognition and clustering[C]. Proceedings of Computer Vision and Pattern Recognition, Portland, 2013: 731738.

Luo Y, Liu T L, Tao D C, et al.. Decomposition-based transfer distance metric learning for image classification[J]. IEEE Transactions on Image Processing, 2014, 23(9): 37893801.

Long M S, Wang J M, Ding G G, et al.. Transfer learning with graph co-regularization[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(7): 18051818.

洪佳明, 印鑒, 黃云, 等. TrSVM: 一種基于領(lǐng)域相似性的遷移學(xué)習(xí)算法[J]. 計(jì)算機(jī)研究與發(fā)展, 2011, 48(10): 18231830.

Hong Jia-ming, Yin Jian, Huang Yun, et al.. TrSVM: a transfer learning algorithm using domain similarity[J]. Journal of Computer Research and Development, 2011, 48(10): 18231830.

Seah C W, Tsang I W, and Ong Y S. Transfer ordinal label learning[J]. IEEE Transactions on Neural Networkand Learning System, 2013, 24(11): 18631876.

許敏, 王士同, 史熒中. 一種新的面向遷移學(xué)習(xí)的L2核分類器[J]. 電子與信息學(xué)報(bào), 2013, 35(9): 2059-2065.

Xu Min, Wang Shi-tong, and Shi Ying-zhong. A novel transfer-learning-oriented L2 kernel classifier[J]. Journal of Electronics Information Technology, 2013, 35(9): 20592065.

Long M S, Wang J M, Ding G G, et al.. Transfer feature learning with joint distribution adaptation[C]. Proceedings of International Conference on Computer Vision, Sydney, 2013: 22002207.

Patricia N and Caputo B. Learning to learn, from transfer learning to domain adaptation: a unifying perspective[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 14421449.

Zhang B, Wang Y, Wang Y, et al.. Stable learning in coding space for multi-class decoding and its extension for multi-class hypothesis transfer learning[C]. Proceedings of Computer Vision and Pattern Recognition, Columbus, 2014: 10751081.

Gretton A, Smola A, Huang J, et al.. Covariate Shift by Kernel Mean Matching[M]. Cambridge: MIT Press, 2009: 131160.

Huang P P, Wang G, and Qin S Y. Boosting for transfer learning from multiple data sources[J]. Pattern Recognition Letters, 2012, 33(5): 568579.

Nie Q F, Jin L Z, and Fei S M. Probability estimation for multi-class classification using AdaBoost[J]. Pattern Recognition, 2014, 47(12): 39313940.

Sugiyama M, Krauledat M, and Mller K R. Covariate shift adaptation by importance weighted cross validation[J]. The Journal of Machine Learning Research, 2007, 8(1): 9851005.

Choi M J, Lim J J, Torralba A, et al.. Exploiting hierarchical context on a large database of object categories[C]. IEEE Conference on Computer Vision Pattern Recognition, San Frencisco, CA, 2010: 129136.

Han Y and Liu G. Biologically inspired task oriented gist model for scene classification[J]. Computer Vision and Image Understanding, 2013, 117(1): 7695.

相關(guān)文章

施引文獻(xiàn)

資源附件(0)

訪問統(tǒng)計(jì)