基于霧線先驗(yàn)的時(shí)空關(guān)聯(lián)約束視頻去霧算法

姚婷婷; 梁越; 柳曉鳴; 胡青

doi:10.11999/JEIT190403

基于霧線先驗(yàn)的時(shí)空關(guān)聯(lián)約束視頻去霧算法

doi: 10.11999/JEIT190403 cstr: 32379.14.JEIT190403

大連海事大學(xué)信息科學(xué)技術(shù)學(xué)院大連 116026

基金項(xiàng)目: 中央高?；究蒲袠I(yè)務(wù)費(fèi)專項(xiàng)資金(3132020208)，國家自然科學(xué)基金(31700742)

詳細(xì)信息

作者簡介:
姚婷婷：女，1988年生，講師，研究方向?yàn)橛?jì)算機(jī)視覺與圖像處理等

梁越：男，1996年生，碩士生，研究方向?yàn)殪F天視頻處理

柳曉鳴：男，1959年生，教授，研究方向?yàn)楹Ｉ辖煌娮有畔⑻幚?、雷達(dá)信號處理等

胡青：男，1978年生，教授，研究方向?yàn)楹Ｊ滦畔鬏敗⒆詣幼R別系統(tǒng)等

通訊作者:
姚婷婷　ytt1030@dlmu.edu.cn

¹⁾ SfM算法程序可以從網(wǎng)址: http://ccwu.me/vsfm/獲得
中圖分類號: TN911.73, TP391
計(jì)量
- 文章訪問數(shù): 2051
- HTML全文瀏覽量: 904
- PDF下載量: 114
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2019-06-05
- 修回日期: 2020-06-22
- 網(wǎng)絡(luò)出版日期: 2020-07-17
- 刊出日期: 2020-11-16

Video Dehazing Algorithm via Haze-line Prior with Spatiotemporal Correlation Constraint

College of Information Science and Technology, Dalian Maritime University, Dalian 116026, China

Funds: The Fundamental Research Funds for the Central Universities (3132020208), The National Natural Science Foundation of China (31700742)

摘要

摘要: 現(xiàn)有視頻去霧算法由于缺少對視頻結(jié)構(gòu)關(guān)聯(lián)約束和幀間一致性分析，容易導(dǎo)致連續(xù)幀去霧結(jié)果在顏色和亮度上存在突變，同時(shí)去霧后的前景目標(biāo)邊緣區(qū)域也容易出現(xiàn)退化現(xiàn)象。針對上述問題，該文提出一種基于霧線先驗(yàn)的時(shí)空關(guān)聯(lián)約束視頻去霧算法，通過引入每幀圖像在空間鄰域中具有的結(jié)構(gòu)關(guān)聯(lián)性和時(shí)間鄰域中具有的連續(xù)一致性，提高視頻去霧算法的求解準(zhǔn)確性和魯棒性。算法首先使用暗通道先驗(yàn)估計(jì)每幀圖像的大氣光向量，并結(jié)合霧線先驗(yàn)求取初始透射率圖。然后引入加權(quán)最小二乘邊緣保持平滑濾波器對初始透射率圖進(jìn)行空間平滑，消除奇異點(diǎn)和噪聲對估計(jì)結(jié)果的影響。進(jìn)一步利用相機(jī)參數(shù)刻畫連續(xù)幀間透射率圖的時(shí)序變化規(guī)律，對獨(dú)立求取的每幀透射率圖進(jìn)行時(shí)序關(guān)聯(lián)修正。最后根據(jù)霧圖模型獲得最終的視頻去霧結(jié)果。定性和定量的對比實(shí)驗(yàn)結(jié)果表明，該算法下視頻去霧結(jié)果的幀間過渡更加自然，同時(shí)對每一幀圖像的色彩還原更加準(zhǔn)確，圖像邊緣的細(xì)節(jié)信息顯示也更加豐富。
- 視頻去霧 /
- 空間平滑 /
- 時(shí)序關(guān)聯(lián)約束 /
- 霧線先驗(yàn)
Abstract: Because of the existent video dehazing algorithm lacks the analysis of the video structure correlation constraint and inter-frame consistency, it is easy to cause the dehazing results of continuous frames to have sudden changes in color and brightness. Meanwhile, the edge of foreground target is also prone to degradation. Focus on the aforementioned problems, a novel video dehazing algorithm via haze-line prior with spatiotemporal correlation constraint is proposed, which improves the accuracy and robustness of video dehazing result by bringing the structural relevance and temporal consistency of each frame. Firstly, the dark channel and haze-line prior are utilized to estimate the atmospheric light vector and initial transmission image of each frame. Then a weighted least square edge preserving smoothing filter is introduced to smooth the initial transmission image and eliminate the influence of singularities and noises on the estimated results. Furthermore, the camera parameters are calculated to describe the time series variation of the transmission image between continuous frames, and the independently obtained transmission image of each frame is corrected with temporal correlation constraint. Finally, according to the physical model, the video dehazing results are obtained. The experimental results of qualitative and quantitative comparison show that the proposed algorithm could make the inter-frame transition more smooth, and restore the color of each frame more accurately. Besides, more details are displayed at the edge of the dehazing results.
- Video dehazing /
- Spatial smoothing /
- Temporal correlation constraint /
- Haze-line prior
¹⁾ SfM算法程序可以從網(wǎng)址: http://ccwu.me/vsfm/獲得

HTML全文

圖 1 霧線示意圖

下載: 全尺寸圖片幻燈片

圖 2 本文算法總體框圖

下載: 全尺寸圖片幻燈片

圖 3 同一幀各階段透射率圖對比

下載: 全尺寸圖片幻燈片

圖 4 視頻Ship和Beach連續(xù)幀下去霧結(jié)果對比

下載: 全尺寸圖片幻燈片

圖 5 單幀圖像去霧結(jié)果對比

下載: 全尺寸圖片幻燈片

表 1 各算法在不同評價(jià)指標(biāo)下的性能對比

視頻集	算法	VCM	SSIM	HCC	信息熵	UQI
Bali	文獻(xiàn)[16]算法	47.7398±4.3502	0.6614±0.0459	–0.3173±0.0485	6.8571±0.1641	0.5923±0.0425
	文獻(xiàn)[17]算法	30.1254±5.6277	0.5526±0.0215	–0.2960±0.0326	7.3794±0.0792	0.4669±0.0225
	文獻(xiàn)[19]算法	41.6247±4.9448	0.8693±0.0041	–0.0813±0.0581	7.5121±0.0447	0.8061±0.0086
	文獻(xiàn)[20]算法	37.4402±4.0238	0.6221±0.0214	–0.1968±0.1061	6.5753±0.1445	0.5889±0.0335
	文獻(xiàn)[21]算法	49.7812±4.4782	0.7001±0.1116	–0.0312±0.0639	7.5413±0.0572	0.8819±0.0101
	本文算法	51.3852±6.3223	0.6679±0.0249	–0.2532±0.0251	7.9253±0.1322	0.8938±0.0285
Blenheim	文獻(xiàn)[16]算法	35.5897±2.2001	0.8686±0.0129	0.0667±0.0165	6.5025±0.1967	0.7960±0.0258
	文獻(xiàn)[17]算法	37.5815±1.6224	0.8260±0.0366	0.4688±0.0945	7.0697±0.0903	0.7661±0.0373
	文獻(xiàn)[19]算法	26.0153±1.9259	0.9123±0.0035	0.4406±0.0231	7.0390±0.0716	0.8332±0.0209
	文獻(xiàn)[20]算法	18.2786±2.2769	0.8215±0.0262	0.2759±0.0339	6.4305±0.0602	0.7721±0.0449
	文獻(xiàn)[21]算法	64.9899±1.7827	0.6632±0.0094	0.0377±0.0135	6.2305±0.1637	0.7239±0.0108
	本文算法	40.0056±0.9116	0.9764±0.0022	0.7940±0.0969	7.4379±0.0434	0.9549±0.0131
Playground	文獻(xiàn)[16]算法	50.2886±6.4619	0.8954±0.0129	–0.0276±0.0249	5.9694±0.4580	0.9021±0.0123
	文獻(xiàn)[17]算法	31.9030±8.1211	0.7788±0.0575	0.0115±0.1177	7.4964±0.0911	0.7529±0.0703
	文獻(xiàn)[19]算法	43.6243±3.5659	0.9205±0.0086	0.1664±0.0519	7.2406±0.1060	0.8967±0.0142
	文獻(xiàn)[20]算法	35.5237±3.2426	0.7807±0.0182	–0.1450±0.0595	6.9652±0.0930	0.7728±0.0309
	文獻(xiàn)[21]算法	51.8038±5.3890	0.6917±0.0258	–0.0457±0.0417	6.8567±0.2089	0.8213±0.0204
	本文算法	54.4761±10.9746	0.9546±0.0230	0.0587±0.0506	7.1156±0.1721	0.9379±0.0421
Stele	文獻(xiàn)[16]算法	30.7638±15.7369	0.3649±0.0288	0.4263±0.1234	6.8163±0.0828	0.7349±0.0476
	文獻(xiàn)[17]算法	44.2502±3.1428	0.7392±0.0199	0.0577±0.0443	7.1132±0.0828	0.7665±0.0296
	文獻(xiàn)[19]算法	51.0145±5.4531	0.8529±0.0109	0.3319±0.0846	6.5512±0.1477	0.7671±0.0174
	文獻(xiàn)[20]算法	40.8246±4.8187	0.8214±0.0157	0.2711±0.1034	6.2814±0.1242	0.7805±0.0233
	文獻(xiàn)[21]算法	75.7723±3.8898	0.5875±0.0166	0.0227±0.0419	7.3425±0.1065	0.7881±0.0252
	本文算法	29.9452±3.0779	0.9045±0.0222	0.4791±0.1163	7.9275±0.0933	0.8580±0.0361
Motocycle	文獻(xiàn)[16]算法	35.9520±15.9557	0.6854±0.0389	–0.0101±0.0254	6.9864±0.0507	0.5594±0.0292
	文獻(xiàn)[17]算法	49.8669±5.5066	0.7500±0.0324	0.3960±0.0882	7.4345±0.0583	0.8401±0.0319
	文獻(xiàn)[19]算法	33.4659±11.3533	0.8693±0.0047	0.2797±0.0475	7.0304±0.0646	0.8535±0.0114
	文獻(xiàn)[20]算法	25.9303±7.0241	0.3705±0.0650	–0.3172±0.3156	5.7929±0.3812	0.1670±0.0547
	文獻(xiàn)[21]算法	69.1006±4.9147	0.5231±0.0146	0.0111±0.0299	7.6674±0.1062	0.7442±0.0219
	本文算法	52.0758±8.0932	0.7720±0.0281	0.5653±0.0896	7.2598±0.0992	0.8955±0.0261
Ship	文獻(xiàn)[16]算法	38.8112±11.4819	0.7648±0.1141	0.0505±0.1817	7.5924±0.1369	0.6952±0.1243
	文獻(xiàn)[17]算法	33.9529±3.1727	0.8063±0.0162	0.0147±0.0551	7.5878±0.0791	0.7969±0.0267
	文獻(xiàn)[19]算法	35.1544±10.5172	0.8396±0.0104	0.0313±0.0449	7.5395±0.0431	0.7682±0.0134
	文獻(xiàn)[20]算法	33.8128±3.7508	0.5602±0.0496	–0.3243±0.0212	6.8355±0.1586	0.3912±0.0669
	文獻(xiàn)[21]算法	53.0535±4.7626	0.6343±0.0167	–0.1062±0.0099	6.8996±0.1023	0.7783±0.0097
	本文算法	46.0955±3.6169	0.8365±0.0125	0.2137±0.0863	7.8230±0.0131	0.7985±0.0206
Beach	文獻(xiàn)[16]算法	16.2818±4.7095	0.9786±0.0096	0.7025±0.0864	7.3837±0.0457	0.9702±0.0133
	文獻(xiàn)[17]算法	38.2337±5.4335	0.7396±0.0096	–0.2957±0.0371	7.4769±0.0844	0.6582±0.0159
	文獻(xiàn)[19]算法	9.4401±2.4871	0.8823±0.0050	0.0043±0.0454	7.4627±0.0237	0.8078±0.0062
	文獻(xiàn)[20]算法	22.7816±3.3244	0.4762±0.4821	–0.3919±0.0207	6.4422±0.2008	0.2868±0.0615
	文獻(xiàn)[21]算法	32.7816±8.8543	0.7221±0.0207	0.0181±0.0082	7.4134±0.0851	0.8845±0.0031
	本文算法	28.2158±5.8622	0.9802±0.0154	0.8117±0.0881	7.8108±0.0240	0.9415±0.0197

下載: 導(dǎo)出CSV

表 2 各算法計(jì)算效率對比

算法	文獻(xiàn)[16]算法	文獻(xiàn)[17]算法	文獻(xiàn)[19]算法	文獻(xiàn)[20]算法	文獻(xiàn)[21]算法	本文算法
時(shí)間(s)	182.2654	1.0985	0.1076	1.8250	0.2601	1.0502

下載: 導(dǎo)出CSV

參考文獻(xiàn)(21)

XU Yong, WEN Jie, FEI Lunke, et al. Review of video and image defogging algorithms and related studies on image restoration and enhancement[J]. IEEE Access, 2015, 4: 165–188. doi: 10.1109/ACCESS.2015.2511558

YU Tianhe, MENG Xue, ZHU Ming, et al. An improved multi-scale retinex fog and haze image enhancement method[C]. 2016 International Conference on Information System and Artificial Intelligence, Hong Kong, China, 2016: 557–560. doi: 10.1109/ISAI.2016.0124.

LI Yaning, WANG Junping, GAO Kang, et al. Fast morphological filtering haze removal method from a single image[J]. Journal of Computational Information Systems, 2015, 11(16): 5799–5806. doi: 10.12733/jcis14861

QIAO Tong, REN Jinchang, WANG Zheng, et al. Effective denoising and classification of hyperspectral images using curvelet transform and singular spectrum analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(1): 119–133. doi: 10.1109/TGRS.2016.2598065

黃果, 許黎, 陳慶利, 等. 非局部多尺度分?jǐn)?shù)階微分圖像增強(qiáng)算法研究[J]. 電子與信息學(xué)報(bào), 2019, 41(12): 2972–2979. doi: 10.11999/JEIT190032

HUANG Guo, XU Li, CHEN Qingli, et al. Research on non-local multi-scale fractional differential image enhancement algorithm[J]. Journal of Electronics &Information Technology, 2019, 41(12): 2972–2979. doi: 10.11999/JEIT190032

HE Kaiming, SUN Jian, and TANG Xiaoou. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353. doi: 10.1109/TPAMI.2010.168

楊愛萍, 王南, 龐彥偉, 等. 人工光源條件下夜間霧天圖像建模及去霧[J]. 電子與信息學(xué)報(bào), 2018, 40(6): 1330–1337. doi: 10.11999/JEIT170704

YANG Aiping, WANG Nan, PANG Yanwei, et al. Nighttime haze removal based on new imaging model with artificial light sources[J]. Journal of Electronics &Information Technology, 2018, 40(6): 1330–1337. doi: 10.11999/JEIT170704

江巨浪, 孫偉, 王振東, 等. 基于透射率權(quán)值因子的霧天圖像融合增強(qiáng)算法[J]. 電子與信息學(xué)報(bào), 2018, 40(10): 2388–2394. doi: 10.11999/JEIT171032

JIANG Julang, SUN Wei, WANG Zhendong, et al. Integrated enhancement algorithm for hazy image using transmittance as weighting factor[J]. Journal of Electronics &Information Technology, 2018, 40(10): 2388–2394. doi: 10.11999/JEIT171032

MA Xiao, SHAO Limin, XU Guanlei, et al. Intelligent defogging method based on clustering and dark channel prior[C]. 2018 IEEE 3rd International Conference on Image, Vision and Computing, Chongqing, China, 2018: 149–156. doi: 10.1109/ICIVC.2018.8492842.

楊燕, 王志偉. 基于均值不等關(guān)系優(yōu)化的自適應(yīng)圖像去霧算法[J]. 電子與信息學(xué)報(bào), 2020, 42(3): 755–763. doi: 10.11999/JEIT190368

YANG Yan and WANG Zhiwei. Adaptive image dehazing algorithm based on mean unequal relation optimization[J]. Journal of Electronics &Information Technology, 2020, 42(3): 755–763. doi: 10.11999/JEIT190368

JOHN J and WILSCY M. Enhancement of weather degraded video sequences using wavelet fusion[C]. The 7th IEEE International Conference on Cybernetic Intelligent Systems, London, England, 2008: 1–6. doi: 10.1109/UKRICIS.2008.4798926.

YOON I, KIM S, KIM D, et al. Adaptive defogging with color correction in the HSV color space for consumer surveillance system[J]. IEEE Transactions on Consumer Electronics, 2012, 58(1): 111–116. doi: 10.1109/TCE.2012.6170062

郭璠, 蔡自興, 謝斌. 基于霧氣理論的視頻去霧算法[J]. 電子學(xué)報(bào), 2011, 39(9): 2019–2025.

GUO Fan, CAI Zixing, and XIE Bin. Video defogging algorithm based on fog theory[J]. Acta Electronica Sinica, 2011, 39(9): 2019–2025.

劉海波, 楊杰, 吳正平, 等. 改進(jìn)的基于霧氣理論的視頻去霧[J]. 光學(xué)精密工程, 2016, 24(7): 1789–1798. doi: 10.3788/OPE.20162407.1789

LIU Haibo, YANG Jie, WU Zhengping, et al. Improved video defogging based on fog theory[J]. Optics and Precision Engineering, 2016, 24(7): 1789–1798. doi: 10.3788/OPE.20162407.1789

馬忠麗, 文杰, 郝亮亮. 海面艦船場景的視頻圖像海霧去除算法[J]. 系統(tǒng)工程與電子技術(shù), 2014, 36(9): 1860–1867. doi: 10.3969/j.issn.1001-506X.2014.09.31

MA Zhongli, WEN Jie, and HAO Liangliang. Video image defogging algorithm for surface ship scenes[J]. Systems Engineering and Electronics, 2014, 36(9): 1860–1867. doi: 10.3969/j.issn.1001-506X.2014.09.31

LI Zhuwen, TAN Ping, TAN R T, et al. Simultaneous video defogging and stereo reconstruction[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015: 4988–4997. doi: 10.1109/CVPR.2015.7299133.

BERMAN D, TREIBITZ T, AVIDAN S, et al. Non-local image dehazing[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 2016: 1674–1682. doi: 10.1109/CVPR.2016.185.

KATO T, SHIMIZU I, and PAJDLA T. Selecting image pairs for SfM by introducing jaccard similarity[C]. The 15th IAPR International Conference on Machine Vision Applications (MVA), Nagoya, Japan, 2017: 25–29. doi: 10.23919/MVA.2017.7986764.

CAI Bolun, XU Xiangmin, and TAO Dacheng. Real-time video dehazing based on spatio-temporal MRF[C]. The 17th Pacific-Rim Conference on Multimedia on Advances in Multimedia Information Processing, Cham, 2016: 315–325. doi: 10.1007/978-3-319-48896-7_31.

ZHAO Dong, XU Long, YAN Yihua, et al. Multi-scale optimal fusion model for single image dehazing[J]. Signal Processing: Image Communication, 2019, 74: 253–265. doi: 10.1016/j.image.2019.02.004

YU Teng, SONG Kang, MIAO Pu, et al. Nighttime single image dehazing via pixel-wise alpha blending[J]. IEEE Access, 2019, 7: 114619–114630. doi: 10.1109/ACCESS.2019.2936049

相關(guān)文章

施引文獻(xiàn)

資源附件(0)

訪問統(tǒng)計(jì)