基于多通道調(diào)頻連續(xù)波毫米波雷達(dá)的微動(dòng)手勢識(shí)別

夏朝陽; 周成龍; 介鈞譽(yù); 周濤; 汪相鋒; 徐豐

doi:10.11999/JEIT190797

基于多通道調(diào)頻連續(xù)波毫米波雷達(dá)的微動(dòng)手勢識(shí)別

doi: 10.11999/JEIT190797 cstr: 32379.14.JEIT190797

復(fù)旦大學(xué)電磁波信息科學(xué)教育部重點(diǎn)實(shí)驗(yàn)室上海 200433

基金項(xiàng)目: 國家自然科學(xué)基金(61822107)

詳細(xì)信息

作者簡介:
夏朝陽：男，1993年生，博士生，研究方向?yàn)槔走_(dá)信號處理、目標(biāo)識(shí)別和深度學(xué)習(xí)

周成龍：男，1995年生，碩士生，研究方向?yàn)槔走_(dá)目標(biāo)識(shí)別和深度學(xué)習(xí)應(yīng)用

介鈞譽(yù)：男，1993年生，碩士生，研究方向?yàn)槔走_(dá)目標(biāo)識(shí)別和深度學(xué)習(xí)應(yīng)用

周濤：男，1996年生，碩士生，研究方向?yàn)槔走_(dá)目標(biāo)識(shí)別和深度學(xué)習(xí)應(yīng)用

汪相鋒：男，1995年生，碩士生，研究方向?yàn)槔走_(dá)目標(biāo)識(shí)別和深度學(xué)習(xí)應(yīng)用

徐豐：男，1982年生，教授，博士生導(dǎo)師，研究方向?yàn)镾AR圖像解譯、電磁散射建模和類腦人工智能

通訊作者:
徐豐　fengxu@fudan.edu.cn

中圖分類號: TN958.94
計(jì)量
- 文章訪問數(shù): 5079
- HTML全文瀏覽量: 1644
- PDF下載量: 294
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2019-10-16
- 修回日期: 2019-11-27
- 網(wǎng)絡(luò)出版日期: 2019-12-09
- 刊出日期: 2020-01-21

Micro-motion Gesture Recognition Based on Multi-channel Frequency Modulated Continuous Wave Millimeter Wave Radar

Key Laboratory for Information Science of Electromagnetic Waves(MoE), Fudan University, Shanghai 200433, China

Funds: The National Natural Science Foundation of China (61822107)

摘要

摘要:
該文提出一種基于多通道調(diào)頻連續(xù)波(FMCW)毫米波雷達(dá)的微動(dòng)手勢識(shí)別方法，并給出一種微動(dòng)手勢特征提取的最優(yōu)雷達(dá)參數(shù)設(shè)計(jì)準(zhǔn)則。通過對手部反射的雷達(dá)回波進(jìn)行時(shí)頻分析處理，估計(jì)目標(biāo)的距離多普勒譜、距離譜、多普勒譜和水平方向角度譜。設(shè)計(jì)固定幀時(shí)間長度拼接的距離-多普勒-時(shí)間圖特征，與距離-時(shí)間特征、多普勒-時(shí)間特征、水平方向角度-時(shí)間圖特征和三者聯(lián)合特征等，分別對7類微動(dòng)手勢進(jìn)行表征。根據(jù)手勢運(yùn)動(dòng)過程振幅和速度差異，進(jìn)行手勢特征捕獲和對齊。利用僅有5層的輕量化卷積神經(jīng)網(wǎng)絡(luò)對微動(dòng)手勢特征進(jìn)行分類。實(shí)驗(yàn)結(jié)果表明，相較其他特征，設(shè)計(jì)的距離-多普勒-時(shí)間圖特征能夠更為準(zhǔn)確地表征微動(dòng)手勢，且對未經(jīng)訓(xùn)練的測試對象具有更好的泛化能力。
- 毫米波雷達(dá) /
- 微動(dòng)手勢識(shí)別 /
- 調(diào)頻連續(xù)波 /
- 卷積神經(jīng)網(wǎng)絡(luò)
Abstract:
A micro-motion gesture recognition method based on multi-channel Frequency Modulated Continuous Wave (FMCW) millimeter wave radar is proposed, and an optimal radar parameter design criterion for feature extraction of micro-motion gestures is presented. The time-frequency analysis process is performed on the radar echo reflected by the hand, and the range Doppler spectrum, the range spectrum, the Doppler spectrum and the horizontal direction angle spectrum of the target are estimated. Then the range-Doppler-time-map feature is designed, range-time-map feature, Doppler-time-map feature, horizontal-angle-time-map feature, and three-joint feature with fixed frame time length are used to characterize the 7 classes micro-motion gestures, respectively. And these gesture features are captured and aligned according to the difference in amplitude and speed of the gesture motion process. Then a five-layer lightweight convolutional neural network is designed to classify the gesture features. The experimental results show that, the range-Doppler-time-map feature designed in this paper characterizes the micro-motion gesture more accurately and has a better generalization ability for untrained test objects compared with other features.
- Millimeter wave radar /
- Micro-motion gesture recognition /
- Frequency Modulated Continuous Wave(FMCW) /
- Convolutional Neural Network(CNN)

HTML全文

圖 1 實(shí)驗(yàn)流程圖

下載: 全尺寸圖片幻燈片

圖 2 7類微動(dòng)手勢的動(dòng)作示意圖與5種特征圖

下載: 全尺寸圖片幻燈片

圖 3 單通道與8通道平均的手勢幀RD圖信噪比對比

下載: 全尺寸圖片幻燈片

表 1 不同時(shí)間長度特征的分類準(zhǔn)確率對比(%)

數(shù)據(jù)集長度	6幀	8幀	10幀	15幀
平均分類準(zhǔn)確率	76.86	91.86	99.14	99.29

下載: 導(dǎo)出CSV

表 2 多種手勢表征方法的對比

特征類型	分類方法	5名訓(xùn)練對象平均分類準(zhǔn)確率(%)
SC-RDTM	單通道CNN	98.28
CA-RDTM	單通道CNN	99.14
CA-DTM	單通道CNN	97.14
CA-RTM	單通道CNN	88.00
HATM	單通道CNN	71.71
CA-RTM, CA-DTM 與HATM聯(lián)合	3通道CNN	93.57

下載: 導(dǎo)出CSV

表 3 多種手勢表征方法的對比

特征類型	分類方法	測試對象A平均分類準(zhǔn)確率(%)	測試對象B平均分類準(zhǔn)確率(%)
SC-RDTM	單通道CNN	86.00	84.29
CA-RDTM	單通道CNN	87.71	85.43
CA-DTM	單通道CNN	84.57	83.43
CA-RTM	單通道CNN	27.14	25.42
HATM	單通道CNN	34.28	30.57
CA-RTM, CA-DTM與HATM聯(lián)合	3通道CNN	65.14	55.71

下載: 導(dǎo)出CSV

表 4 7種微動(dòng)手勢分類的混淆矩陣

		預(yù)測類別
		食指雙擊	食指順時(shí) 針繞圈	食指逆時(shí) 針繞圈	食指拇指分開	食指拇指并攏	拇指在食指上前搓動(dòng)	拇指在食指上后搓動(dòng)	準(zhǔn)確度(%)
真實(shí)類別	食指雙擊	100	0	0	0	0	0	0	100
	食指順時(shí)針繞圈	0	100	0	0	0	0	0	100
	食指逆時(shí)針繞圈	0	0	100	0	0	0	0	100
	食指拇指分開	0	0	0	100	0	0	0	100
	食指拇指并攏	0	0	0	0	98	0	2	98
	拇指在食指上前搓動(dòng)	0	0	0	0	0	100	0	100
	拇指在食指上后搓動(dòng)	0	0	0	0	4	0	96	96
	準(zhǔn)確度(%)	100	100	100	100	96.08	100	97.96	99.14

下載: 導(dǎo)出CSV

表 5 測試對象A 7類微動(dòng)手勢分類的混淆矩陣

		預(yù)測類別
		食指雙擊	食指順時(shí) 針繞圈	食指逆時(shí) 針繞圈	食指拇指分開	食指拇指并攏	拇指在食指上前搓動(dòng)	拇指在食指上后搓動(dòng)	準(zhǔn)確度(%)
真實(shí)類別	食指雙擊	46	3	1	0	0	0	0	92
	食指順時(shí)針繞圈	0	35	15	0	0	0	0	70
	食指逆時(shí)針繞圈	3	13	34	0	0	0	0	68
	食指拇指分開	0	0	0	49	0	1	0	98
	食指拇指并攏	0	0	0	0	46	0	4	92
	拇指在食指上前搓動(dòng)	0	0	0	1	0	49	0	98
	拇指在食指上后搓動(dòng)	0	0	0	0	2	0	48	96
	準(zhǔn)確度(%)	93.88	68.63	68	98	95.83	98	92.31	87.71

下載: 導(dǎo)出CSV

表 6 測試對象B 7類微動(dòng)手勢分類的混淆矩陣

		預(yù)測類別
		食指雙擊	食指順時(shí)針繞圈	食指逆時(shí) 針繞圈	食指拇指分開	食指拇指并攏	拇指在食指上前搓動(dòng)	拇指在食指上后搓動(dòng)	準(zhǔn)確度(%)
真實(shí)類別	食指雙擊	45	2	3	0	0	0	0	90
	食指順時(shí)針繞圈	0	32	18	0	0	0	0	64
	食指逆時(shí)針繞圈	1	16	33	0	0	0	0	66
	食指拇指分開	0	0	0	46	0	4	0	92
	食指拇指并攏	0	0	0	0	48	0	2	96
	拇指在食指上前搓動(dòng)	0	0	0	2	0	48	0	96
	拇指在食指上后搓動(dòng)	0	0	0	0	3	0	47	94
	準(zhǔn)確度(%)	97.83	64	61.11	95.83	94.12	92.31	95.92	85.43

下載: 導(dǎo)出CSV

參考文獻(xiàn)(19)

WAN Qian, LI Yiran, LI Changzhi, et al. Gesture recognition for smart home applications using portable radar sensors[C]. The 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, USA, 2014: 6414–6417.

KHAN F, LEEM S K, and CHO S H. Hand-based gesture recognition for vehicular applications using IR-UWB radar[J]. Sensors, 2017, 17(4): 833. doi: 10.3390/s17040833

XIA Zia, SANG Xinzhu, CHEN Duo, et al. An interactive VR system based on full-body tracking and gesture recognition[C]. The SPIE 10021, Optical Design and Testing VⅡ, Beijing, China 2016: 1002129.

LEE B G and LEE S M. Smart wearable hand device for sign language interpretation system with sensors fusion[J]. IEEE Sensors Journal, 2018, 18(3): 1224–1232. doi: 10.1109/JSEN.2017.2779466

TARANTA Ⅱ E M, SIMONS T K, SUKTHANKAR R, et al. Exploring the benefits of context in 3D gesture recognition for game-based virtual environments[J]. ACM Transactions on Interactive Intelligent Systems, 2015, 5(1): Article No.1.

BONATO V, FERNANDES M M, and MARQUES E. A smart camera with gesture recognition and SLAM capabilities for mobile robots[J]. International Journal of Electronics, 2006, 93(6): 385–401. doi: 10.1080/00207210600565465

孟春寧, 呂建平, 陳萱華. 基于普通紅外攝像機(jī)的手勢識(shí)別[J]. 計(jì)算機(jī)工程與應(yīng)用, 2015, 51(16): 17–22. doi: 10.3778/j.issn.1002-8331.1504-0303

MENG Chunning, Lü Jianping, and CHEN Xuanhua. Gesture recognition based on universal infrared camera[J]. Computer Engineering and Applications, 2015, 51(16): 17–22. doi: 10.3778/j.issn.1002-8331.1504-0303

PLOUFFE G and CRETU A M. Static and dynamic hand gesture recognition in depth data using dynamic time warping[J]. IEEE Transactions on Instrumentation and Measurement, 2016, 65(2): 305–316. doi: 10.1109/TIM.2015.2498560

YANG Qifan, TANG Hao, ZHAO Xuebing, et al. Dolphin: Ultrasonic-based gesture recognition on smartphone platform[C]. The 2014 IEEE 17th International Conference on Computational Science and Engineering, Chengdu, China, 2014: 1461–1468.

KELLOGG B, TALLA V, and GOLLAKOTA S. Bringing gesture recognition to all devices[C]. The 11th Usenix Conference on Networked Systems Design and Implementation, Seattle, USA, 2014: 303–316.

ZOU Yongpan, XIAO Jiang, HAN Jinsong, et al. GRfid: A device-free RFID-based gesture recognition system[J]. IEEE Transactions on Mobile Computing, 2017, 16(2): 381–393. doi: 10.1109/TMC.2016.2549518

ABDELNASSER H, YOUSSEF M, and HARRAS K A. WiGest: A ubiquitous WiFi-based gesture recognition system[C]. Proceedings of IEEE Conference on Computer Communications, Hongkong, China, 2015: 1472–1480.

ZHANG Jiajun, TAO Jinkun, and SHI Zhiguo. Doppler-radar based hand gesture recognition system using convolutional neural networks[C]. The 2017 International Conference on Communications, Signal Processing, and Systems, Singapore, 2019: 1096–1113.

LIEN J, GILLIAN N, KARAGOZLER M, et al. Soli: Ubiquitous gesture sensing with millimeter wave radar[J]. ACM Transactions on Graphics, 2016, 35(4): Article No.142.

WANG Saiwen, SONG Jie, LIEN J, et al. Interacting with Soli: Exploring fine-grained dynamic gesture recognition in the radio-frequency spectrum[C]. The 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, 2016: 851–860.

HAZRA S and SANTRA A. Robust gesture recognition using Millimetric-wave radar system[J]. IEEE Sensors Letters, 2018, 2(4): 7001804.

MALYSA G, WANG Dan, NETSCH L, et al. Hidden Markov model-based gesture recognition with FMCW radar[C]. 2016 IEEE Global Conference on Signal and Information Processing, Washington, USA, 2016: 1017–1021.

王勇, 吳金君, 田增山, 等. 基于FMCW雷達(dá)的多維參數(shù)手勢識(shí)別算法[J]. 電子與信息學(xué)報(bào), 2019, 41(4): 822–829. doi: 10.11999/JEIT180485

WANG YONG, WU Jinjun, TIAN Zengshan, et al. Gesture recognition with multi-dimensional parameter using FMCW radar[J]. Journal of Electronics &Information Technology, 2019, 41(4): 822–829. doi: 10.11999/JEIT180485

劉熠辰, 徐豐. 基于雷達(dá)技術(shù)的手勢識(shí)別[J]. 中國電子科學(xué)研究院學(xué)報(bào), 2016, 11(6): 609–613. doi: 10.3969/j.issn.1673-5692.2016.06.009

LIU Yichen and XU Feng. Gesture recognition based on radar technology[J]. Journal of CAEIT, 2016, 11(6): 609–613. doi: 10.3969/j.issn.1673-5692.2016.06.009

相關(guān)文章

施引文獻(xiàn)

資源附件(0)

訪問統(tǒng)計(jì)