帶虛警抑制的基于歸一化殘差的野值檢測方法

汝小虎; 柳征; 姜文利; 黃知濤

doi:10.11999/JEIT150469

帶虛警抑制的基于歸一化殘差的野值檢測方法

doi: 10.11999/JEIT150469 cstr: 32379.14.JEIT150469

計(jì)量
- 文章訪問數(shù): 1393
- HTML全文瀏覽量: 133
- PDF下載量: 425
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2015-04-22
- 修回日期: 2015-09-01
- 刊出日期: 2015-12-19

Normalized Residual-based Outlier Detection with False-alarm Probability Controlling

摘要

摘要: 野值檢測，或稱異常值檢測是模式識別和知識發(fā)現(xiàn)中一個(gè)重要的問題。以往的野值檢測方法難以有效地抑制虛警概率，針對這一問題，該文提出一種帶監(jiān)督情形下基于歸一化殘差(Normalized Residual, NR)的野值檢測方法。首先利用訓(xùn)練樣本計(jì)算待考查模式的NR值，其次比較NR值與野值檢測門限的相對大小，從而判斷待考查模式是否為野值。該文理論上推導(dǎo)了野值門限與虛警概率之間的關(guān)系表達(dá)式，以此為依據(jù)設(shè)置檢測門限，可實(shí)現(xiàn)在少量訓(xùn)練樣本情況下仍能抑制虛警率的目的。計(jì)算機(jī)仿真和實(shí)測數(shù)據(jù)測試驗(yàn)證了所提方法在野值檢測和虛警抑制方面的優(yōu)越性能。
- 模式識別 /
- 監(jiān)督 /
- 野值檢測 /
- 虛警概率 /
- 歸一化殘差
Abstract: Outlier detection, also called anomaly detection, is an important issue in pattern recognition and knowledge discovery. Previous outlier detection methods can not effectively control the false-alarm probability. To solve the problem, a supervised method based on Normalized Residual (NR) is proposed. Using the training patterns, it first calculates the NR value of the query pattern, which is compared with a predefined detection threshold to determine whether the pattern is an outlier. In this paper, the relationship between the threshold and false-alarm probability is theoretically derived, based on which an appropriate threshold can be chosen. In this way, the desired false-alarm probability can be obtained even when few training patterns are available. Simulations and measured data experiments validate the superior performance of the proposed method on outlier detection and false-alarm probability controlling.
- Pattern recognition /
- Supervised /
- Outlier detection /
- False-alarm probability /
- Normalized Residual (NR)

HTML全文

參考文獻(xiàn)(20)

Hawkins D. Identification of Outliers[M]. London: Chapman and Hall, 1980: Chapter 1-2.

Liu J, Wan J, Zheng H, et al.. A method of specific emitter verification based on CSDA and SVDD[C]. Proceedings of the IEEE 2nd International Conference on Computer Science and Network Technology, Changchun, China, 2012: 562-565.

Miller D J and Browning J. A mixture model and EM-based algorithm for class discovery, robust classification, and outlier rejection in mixed labeled/unlabeled data sets[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(11): 1468-1483.

Westerweel J and Scarano F. Universal outlier detection for PIV data[J]. Experiments in Fluids, 2005, 39(6): 1096-1100.

Duncan J, Dabiri D, Hove J, et al.. Universal outlier detection for Particle Image Velocimetry (PIV) and Particle Tracking Velocimetry (PTV) data[J]. Measurement Science and Technology, 2010, 21(5): 57002-57006.

Wu S and Wang S R. Information-theoretic outlier detection for large-scale categorical data[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(3): 589-602.

Li Z G, Baseman R J, Zhu Y D, et al.. A unified framework for outlier detection in trace data analysis[J]. IEEE Transactions on Semiconductor Manufacturing, 2014, 27(1): 95-103.

Albanese A, Pal S K, and Petrosino A. Rough sets, kernel set, and spatiotemporal outlier detection[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(1): 194-207.

Ghosh A K and Chaudhuri P. On maximum depth and related classifiers[J]. Scandinavian Journal of Statistics, 2005, 32(2): 327-350.

Ru X H, Liu Z, and Jiang W L. Normalized residual-based outlier detection[C]. Proceedings of the IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Guilin, China, 2014: 190-193.

Nattorn B, Arthorn L, and Krung S. Outlier detection score based on ordered distance difference[C]. Proceedings of the IEEE International Computer Science and Engineering Conference (ICSEC), Nakhon Pathom, Thailand, 2013: 157-162.

Zhao M and Saligrama V. Anomaly detection with score functions based on nearest neighbor graphs[J]. Advances in Neural Information Processing Systems, 2009, 22(1): 2250-2258.

Qian J and Saligrama V. New statistic in p-value estimation for anomaly detection[C]. Proceedings of the IEEE Statistical Signal Processing Workshop (SSP), Ann Arbor, Michigan, USA, 2012: 393-396.

Chen Y T, Qian J, and Saligrama V. A new one-class SVM for anomaly detection[C]. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada, 2013: 3567-3571.

Breunig M M, Kriegel H-P, Ng R T, et al.. LOF: identifying density-based local outliers[C]. Proceedings of the ACM SIGMOD International Conference on Management of Data, New York, USA, 2000: 93-104.

Sch?lkopf B, Platt J C, Shawe-Taylor J C, et al.. Estimating the support of a high-dimensional distribution[J]. Neural Computation, 2001, 13(7): 1443-1471.

Furlani M, Tuia D, Munoz-Mari J, et al.. Discovering single classes in remote sensing images with active learning[C]. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Munich, Germany, 2012: 7341-7344.

Jumutc V and Suykens J. Multi-class supervised novelty detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(12): 2510-2523.

葉浩歡, 柳征, 姜文利. 考慮多普勒效應(yīng)的脈沖無意調(diào)制特征比較[J]. 電子與信息學(xué)報(bào), 2012, 34(11): 2654-2659.

Ye H H, Liu Z, and Jiang W L. A comparison of unintentional modulation on pulse features with the consideration of Doppler effect[J]. Journal of Electronics Information Technology, 2012, 34(11): 2654-2659.

相關(guān)文章

施引文獻(xiàn)

資源附件(0)

訪問統(tǒng)計(jì)