采用自適應預篩選的遙感圖像目標開集檢測研究

黨思航; 李曉哲; 夏召強; 蔣曉悅; 桂術(shù)亮; 馮曉毅

doi:10.11999/JEIT231426

采用自適應預篩選的遙感圖像目標開集檢測研究

doi: 10.11999/JEIT231426 cstr: 32379.14.JEIT231426

1.
西北工業(yè)大學上海閔行協(xié)同創(chuàng)新中心上海 201108
2.
西北工業(yè)大學電子信息學院西安 710072
3.
重慶郵電大學通信與信息工程學院重慶 400065

基金項目: 國家自然科學基金(62201461, 62301101)，上海市2022年度“科技創(chuàng)新行動計劃”啟明星培育(揚帆專項)項目(22YF1452100)，陜西省科技廳秦創(chuàng)原項目(QCYRCXM-2022-325)，陜西省重點研發(fā)計劃(2023-ZDLGY-16, 2023-ZDLGY-44, 2023-ZDLGY-12, 2021-ZDLGY15-01, 2021-ZDLGY09-04, 2021GY-004, 2022-ZDLGY06-07)，重慶市博士“直通車”科研項目(sl202100000315)

詳細信息

作者簡介:
黨思航：男，副教授，研究方向為雷達目標識別、增量學習

李曉哲：男，碩士，研究方向為目標檢測、開集識別

夏召強：男，副教授，研究方向為圖像處理、計算機視覺

蔣曉悅：女，副教授，研究方向為圖像處理、計算機視覺

桂術(shù)亮：男，講師，研究方向為雷達信號處理、目標檢測

馮曉毅：女，教授，研究方向為圖像處理、計算機視覺

通訊作者:
夏召強　zxia@nwpu.edu.cn

中圖分類號: TP75
計量
- 文章訪問數(shù): 492
- HTML全文瀏覽量: 271
- PDF下載量: 81
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2023-12-02
- 修回日期: 2024-07-04
- 網(wǎng)絡出版日期: 2024-07-25
- 刊出日期: 2024-10-30

Research on Open-Set Object Detection in Remote Sensing Images Based on Adaptive Pre-Screening

1.
Collaborative Innovation Center of NPU, Shanghai, Northwestern Polytechnical University, Shanghai 201108, China
2.
School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, China
3.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Funds: The National Natural Science Foundation of China(62201461, 62301101), Shanghai Sailing Program (22YF1452100), The QINCHUANGYUAN Program (QCYRCXM-2022-325), The Key Research and Development Program of Shaanxi (2023-ZDLGY-16, 2023-ZDLGY-44, 2023-ZDLGY-12, 2021-ZDLGY15-01, 2021-ZDLGY09-04, 2021GY-004, 2022-ZDLGY06-07), Chongqing Doctoral Direct Train Research Project (sl202100000315)

摘要

摘要: 開放動態(tài)環(huán)境下目標類別不斷豐富，遙感目標檢測問題不能局限于已知類目標的鑒別，還需要對未知類目標做出有效判決。該文設計一種基于自適應預篩選的遙感開集目標檢測網(wǎng)絡，首先，提出面向目標候選框的自適應預篩選模塊，依據(jù)篩選出的候選框坐標得到具有豐富語義信息和空間特征的查詢傳遞至解碼器。然后，結(jié)合原始圖像中目標邊緣信息提出一種偽標簽選取方法，并以開集判決為目的構(gòu)造損失函數(shù)，提高網(wǎng)絡對未知新類特征的學習能力。最后，采用MAR20飛機目標識別數(shù)據(jù)集模擬不同的開放動態(tài)遙感目標檢測環(huán)境，通過廣泛的對比實驗和消融實驗，驗證了該文方法能夠?qū)崿F(xiàn)對已知類目標的可靠檢測和未知類目標的有效檢出。
- 遙感目標檢測 /
- 候選框篩選 /
- 開集識別
Abstract: In open, dynamic environments where the range of object categories continually expands, the challenge of remote sensing object detection is to detect a known set of object categories while simultaneously identifying unknown objects. To this end, a remote sensing open-set object detection network based on adaptive pre-screening is proposed. Firstly, an adaptive pre-screening module is proposed for object region proposals. Based on the coordinates of the selected region proposals, queries with rich semantic information and spatial features are generated and passed to the decoder. Subsequently, a pseudo-label selection method is devised based on object edge information, and loss functions are constructed with the aim of open set classification to enhance the network’s ability to learn knowledge of unknown classes. Finally, the Military Aircraft Recognition (MAR20) dataset is used to simulate various dynamic environments. Extensive comparative experiments and ablation experiments show that the proposed method can achieve reliable detection of known and unknown objects.
- Remote sensing object detection /
- Region proposal selection /
- Open set recognition

HTML全文

圖 1 網(wǎng)絡總體結(jié)構(gòu)

下載: 全尺寸圖片幻燈片

圖 2 自適應預篩選模塊

下載: 全尺寸圖片幻燈片

圖 3 MAR20數(shù)據(jù)集部分圖像展示

下載: 全尺寸圖片幻燈片

圖 4 MAR20測試集示例圖像定性結(jié)果

下載: 全尺寸圖片幻燈片

1 基于圖像邊緣信息的偽標簽選取算法

輸入：當前迭代$ t $條件下：對應特征圖$ \boldsymbol{A} $；經(jīng)過DDETR匹配機制剩余的預測候選框$ {{\boldsymbol}}_{i}^{{\mathrm{F}}} $；基于圖像邊緣信息生成的候選框$ {{\boldsymbol}}_{j}^{{\mathrm{E}}} $；損失存儲隊　列$ {L}_{m} $；微調(diào)參數(shù)$ {\lambda }_{p} $和$ {\lambda }_{n} $；權(quán)重更新迭代次數(shù)$ {T}_{w} $；權(quán)重值$ {w}_{1} $和$ {w}_{2} $；偽標簽個數(shù)$ u $
輸出：當前迭代$ t $條件下：圖像的偽標簽
1. while train do:
2. 式(1)初步得到基于卷積特征的目標置信度得分$ F\left({{\boldsymbol}}_{i}^{{\mathrm{F}}}\right) $；
3. 式(3)得到基于圖像底層邊緣信息的目標置信度得分S$ \left({{\boldsymbol}}_{i}^{\mathrm{{E}}}\right) $；
4. if $ t\mathrm{\%}{T}_{w}==0 $ then:
5. 使用式(7)和$ {L}_{m} $計算$ \Delta l $；
6. 使用式(8)計算$ \Delta w $；
7. 使用式(5)更新權(quán)重值$ {w}_{1} $和$ {w}_{2} $；
8. end if
9. 使用式(4)得到剩余的預測候選框$ {{\boldsymbol}}_{i}^{{\mathrm{F}}} $的最終目標置信度分數(shù)$ {F}_{i}^{{\mathrm{new}}} $；
10. 對$ {F}_{i}^{\mathrm{n}\mathrm{e}\mathrm{w}} $從大到小排序，選取前$ {u} $個候選框標記“未知類”。

下載: 導出CSV

表 1 MAR20數(shù)據(jù)集圖像數(shù)量分布情況

A1	A2	A3	A4	A5	A6	A7	A8	A9	A10
168	16	150	70	247	31	100	142	146	146

A11	A12	A13	A14	A15	A16	A17	A18	A19	A20
86	66	212	252	108	265	173	37	129	130
含多類	1017
總計	3842

下載: 導出CSV

表 2 開集目標檢測任務

實驗編號	未知類	目標類別		訓練與測試比例		總計
實驗編號	#已知類+#未知類	已知類	未知類	訓練	測試	總計
任務1	0.75	A1～A5	A6～A20	644	161	805
任務2	0.5	A1～A10	A11～A20	736	185	921
任務3	0.25	A1～A15	A16～A20	764	192	956

下載: 導出CSV

表 3 網(wǎng)絡檢測結(jié)果對比(%)

任務編號	任務1		任務2		任務3
任務編號	已知類mAP	未知類召回率	已知類mAP	未知類召回率	已知類mAP	未知類召回率
Faster-RCNN	73.95	–	77.84	–	88.18	–
YOLOv3	88.02	–	88.40	–	88.86	–
DDETR	84.30	–	87.60	–	88.95	–
OW-DETR	82.52	17.66	87.90	29.68	87.66	30.41
CAT	77.40	21.21	83.78	36.09	85.05	53.42
本文算法	89.09	38.67	90.35	47.17	90.38	61.20

下載: 導出CSV

表 4 模塊驗證實驗結(jié)果(%)

模塊			任務1消融實驗		任務2消融實驗		任務3消融實驗
基準模型	自適應預篩選	基于邊緣信息的偽標簽選取策略	已知類mAP	未知類召回率	已知類mAP	未知類召回率	已知類mAP	未知類召回率
√			82.52	17.66	87.90	29.68	87.66	30.41
√	√		89.34	2.43	90.83	5.96	89.74	13.33
√		√	83.28	45.81	87.54	63.01	87.69	54.80
√	√	√	89.09	38.67	90.35	47.17	90.38	61.20

下載: 導出CSV

參考文獻(24)

[1]	ZAIDI S S A, ANSARI M S, ASLAM A, et al. A survey of modern deep learning based object detection models[J]. Digital Signal Processing, 2022, 126: 103514. doi: 10.1016/j.dsp.2022.103514.
[2]	ZOU Zhengxia, CHEN Keyan, SHI Zhenwei, et al. Object detection in 20 years: A survey[J]. Proceedings of the IEEE, 2023, 111(3): 257–276. doi: 10.1109/JPROC.2023.3238524.
[3]	呂進東, 王彤, 唐曉斌. 基于圖注意力網(wǎng)絡的半監(jiān)督SAR艦船目標檢測[J]. 電子與信息學報, 2023, 45(5): 1541–1549. doi: 10.11999/JEIT220139. Lü Jindong, WANG Tong, and TANG Xiaobin. Semi-supervised SAR ship target detection with graph attention network[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1541–1549. doi: 10.11999/JEIT220139.
[4]	王璽坤, 姜宏旭, 林珂玉. 基于改進型YOLO算法的遙感圖像艦船檢測[J]. 北京航空航天大學學報, 2020, 46(6): 1184–1191. doi: 10.13700/j.bh.1001-5965.2019.0394. WANG Xikun, JIANG Hongxu, and LIN Keyu. Remote sensing image ship detection based on modified YOLO algorithm[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(6): 1184–1191. doi: 10.13700/j.bh.1001-5965.2019.0394.
[5]	AI Jiaqiu, TIAN Ruitian, LUO Qiwu, et al. Multi-scale rotation-invariant Haar-like feature integrated CNN-based ship detection algorithm of multiple-target environment in SAR imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(12): 10070–10087. doi: 10.1109/TGRS.2019.2931308.
[6]	黃玉玲, 陶昕辰, 朱濤, 等. 殘差對抗目標檢測算法的遙感圖像檢測[J]. 電光與控制, 2023, 30(7): 63–67. doi: 10.3969/j.issn.1671-637X.2023.07.011. HUANG Yuling, TAO Xinchen, ZHU Tao, et al. A remote sensing image detection method based on residuals adversarial object detection algorithm[J]. Electronics Optics & Control, 2023, 30(7): 63–67. doi: 10.3969/j.issn.1671-637X.2023.07.011.
[7]	馬梁, 茍于濤, 雷濤, 等. 基于多尺度特征融合的遙感圖像小目標檢測[J]. 光電工程, 2022, 49(4): 210363. doi: 10.12086/oee.2022.210363. MA Liang, GOU Yutao, LEI Tao, et al. Small object detection based on multi-scale feature fusion using remote sensing images[J]. Opto-Electronic Engineering, 2022, 49(4): 210363. doi: 10.12086/oee.2022.210363.
[8]	REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. doi: 10.1109/TPAMI.2016.2577031.
[9]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection[C]. The 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 779–788. doi: 10.1109/CVPR.2016.91.
[10]	REDMON J and FARHADI A. YOLOv3: An incremental improvement[EB/OL]. https://arxiv.org/abs/1804.02767, 2018.
[11]	邵延華, 張鐸, 楚紅雨, 等. 基于深度學習的YOLO目標檢測綜述[J]. 電子與信息學報, 2022, 44(10): 3697–3708. doi: 10.11999/JEIT210790. SHAO Yanhua, ZHANG Duo, CHU Hongyu, et al. A review of YOLO object detection based on deep learning[J]. Journal of Electronics & Information Technology, 2022, 44(10): 3697–3708. doi: 10.11999/JEIT210790.
[12]	CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]. The 16th European Conference on Computer Vision, Glasgow, UK, 2020: 213–229. doi: 10.1007/978-3-030-58452-8_13.
[13]	GENG Chuanxing, HUANG Shengjun, and CHEN Songcan. Recent advances in open set recognition: A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(10): 3614–3631. doi: 10.1109/TPAMI.2020.2981604.
[14]	DANG Sihang, CAO Zongjie, CUI Zongyong, et al. Open set incremental learning for automatic target recognition[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(7): 4445–4456. doi: 10.1109/TGRS.2019.2891266.
[15]	DANG Sihang, XIA Zhaoqiang, JIANG Xiaoyue, et al. Inclusive consistency-based quantitative decision-making framework for incremental automatic target recognition[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5215614. doi: 10.1109/TGRS.2023.3312330.
[16]	JOSEPH K J, KHAN S, KHAN F S, et al. Towards open world object detection[C]. The 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 5826–5836. doi: 10.1109/CVPR46437.2021.00577.
[17]	GUPTA A, NARAYAN S, JOSEPH KJ, et al. OW-DETR: Open-world detection transformer[C]. The 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 9225–9234. doi: 10.1109/CVPR52688.2022.00902.
[18]	MA Shuailei, WANG Yuefeng, WEI Ying, et al. CAT: LoCalization and identification cascade detection transformer for open-world object detection[C]. The 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 19681–19690. doi: 10.1109/CVPR52729.2023.01885.
[19]	UIJLINGS J R R, VAN DE SANDE K E A, GEVERS T, et al. Selective search for object recognition[J]. International Journal of Computer Vision, 2013, 104(2): 154–171. doi: 10.1007/s11263-013-0620-5.
[20]	ZOHAR O, WANG K C, and YEUNG S. PROB: Probabilistic objectness for open world object detection[C]. The 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 11444–11453. doi: 10.1109/CVPR52729.2023.01101.
[21]	CHENG Gong, XIE Xingxing, HAN Junwei, et al. Remote sensing image scene classification meets deep learning: Challenges, methods, benchmarks, and opportunities[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 3735–3756. doi: 10.1109/JSTARS.2020.3005403.
[22]	ZITNICK C L and DOLLáR P. Edge boxes: Locating object proposals from edges[C]. The 13th European Conference on Computer Vision, Zurich, Switzerland, 2014: 391–405. doi: 10.1007/978-3-319-10602-1_26.
[23]	禹文奇, 程塨, 王美君, 等. MAR20: 遙感圖像軍用飛機目標識別數(shù)據(jù)集[J]. 遙感學報, 2023, 27(12): 2688–2696. doi: 10.11834/jrs.20222139. YU Wenqi, CHENG Gong, WANG Meijun, et al. MAR20: A benchmark for military aircraft recognition in remote sensing images[J]. National Remote Sensing Bulletin, 2023, 27(12): 2688–2696. doi: 10.11834/jrs.20222139.
[24]	ZHU Xizhou, SU Weijie, LU Lewei, et al. Deformable DETR: Deformable transformers for end-to-end object detection[C]. 9th International Conference on Learning Representations, 2021.