基于深度估计和梯度下降的水下图像恢复与增强

王宁, 贾薇, 陈延政, 魏一, 吴浩峻

doi:10.16411/j.cnki.issn1006-7736.2024.03.001

大连海事大学学报 >

2024 , Vol. 50 >Issue 3: 1 - 12

DOI: https://doi.org/10.16411/j.cnki.issn1006-7736.2024.03.001

基于深度估计和梯度下降的水下图像恢复与增强

展开

(大连海事大学轮机工程学院，辽宁大连 116026)

王宁*（1983 — ），男，博士，教授，博士生导师，E-mail：n.wang@ieee.org

网络出版日期: 2024-03-13

基金资助

国家自然科学基金资助项目（U23A20680；52271306）；国家高层次人才支持计划项目（SQ2022QB00329）；国防基础科研计划一般项目（JCKY2022410C013）；辽宁省“兴辽英才计划”领军人才项目（XLYC2202005）；大连市科技创新基金重大基础研究项目（2023JJ11CG009）；中央高校基本科研业务费专项资金项目（3132023501）

收起

Underwater image restoration and enhancement using depth estimation and gradient descent

Expand

(Marine Engineering College, Dalian Maritime University, Dalian 116026, China)

Online published: 2024-03-13

Fold

摘要

由于水下介质散射和吸收等固有特性，水下图像面临图像模糊、低对比度和颜色失真等多重降质问题，严重影响视觉感知性能。针对上述问题，提出基于深度估计和梯度下降策略的水下图像恢复与增强框架(UIRENet)。首先，借助卷积、非线性激活函数模块，构建深度感知网络，实现对不同退化区域的场景深度感知图，克服场景深度依赖的退化；其次，提出梯度优化策略，优化卷积网络参数，提升深度网络增强性能；最后，结合感知损失、边缘损失和水下色彩恒常损失，形成水下图像增强网络损失函数。通过在UIEB-90、UIEB-M和EUVP数据集上开展综合测试实验，验证了UIRENet框架在降低水下图像模糊度、提升视觉效果方面均显著优于目前典型水下图像增强方法，特别在客观评价指标UIQM上，相比CLAHE、ICM、GC、IBLA、DCP、ULAP、FUnIE-GAN、UGAN和Uformer等方法分别提高0.3700、0.6446、0.5919、1.3081、1.3032、1.1672、0.0593、0.1329和0.0934。

关键词： 水下图像; 图像恢复; 图像增强; 深度估计; 梯度下降策略; 卷积神经网络

本文引用格式

王宁, 贾薇, 陈延政, 魏一, 吴浩峻 . 基于深度估计和梯度下降的水下图像恢复与增强[J]. 大连海事大学学报, 2024 , 50(3) : 1 -12 . DOI: 10.16411/j.cnki.issn1006-7736.2024.03.001

Abstract

Due to inherent scattering and absorption, underwater images inevitably suffer from multiple degradations arising from blurring, low contrast and color distortion, thereby seriously deteriorating visual perception. In this paper, a deep learning-based underwater image restoration and enhancement framework (UIRENet) was proposed by virtue of depth estimation and gradient descent strategy. With the aid of convolutional and nonlinear activation function modules, a deep perception network was constructed to achieve scene depth perception maps for different degradation regions, thereby overcoming the dependence of scene depth degradation. A gradient optimization strategy was further proposed to optimize the parameters of convolutional networks and improve the performance of deep network enhancement. Combined with perceptual, edge and underwater color constancy losses, a comprehensive loss function for underwater image enhancement networks was rationally formed. Comprehensive experiments on the UIEB-90, UIEB-M and EUVP datasets show that the UIRENet framework significantly outperforms typical underwater image enhancement methods in terms of reducing underwater image blurriness and improving visual effects. In particular, comparing to CLAHE, ICM, GC, IBLA, DCP, ULAP, FUnIE-GAN, UGAN and Uformer, the objective evaluation metric UIQM can be promoted by 0.3700, 0.6446, 0.5919, 1.3081,1.3032, 1.1672, 0.0593, 0.1329 and 0.0934, respectively.

Key words： underwater image; image restoration; image enhancement; depth estimation; gradient descent strategy; convolutional neural network

参考文献

[1] WANG N, WANG Y Y, ER M J. Review on deep learning techniques for marine object recognition: Architectures and algorithms[J]. Control Engineering Practice, 2022, 118: 104458.
[2] ZUIDERVELD K. Contrast limited adaptive histogram equalization[J]. Graphic Gems. 1994,35(69): 474-485.
[3] PENG Y T, COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594．
[4] SONG W, WANG Y, HUANG D M, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]// Pacific Rim Conference on Multimedia. Switzerland: Springer, 2018: 678–688. doi:10.1007/978-3-030-00776-8_62.
[5] WANG N, CHEN T K, KONG X J, et al. Underwater attentional generative adversarial networks for image enhancement[J]. IEEE Transactions on Human-Machine Systems, 2023, 53(3): 490-500.
[6] LI C, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038.
[7] FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]//2018 IEEE International Conference on Robotics and Automation(ICRA) . Brisbane, QLD:IEEE, 2018: 7159-7165. doi: 10.1109/ICRA.2018.8460552.
[8] CHEN T K, WANG N, CHEN Y Z, et al. Semantic attention and relative scene depth-guided network for underwater image enhancement[J]. Engineering Applications of Artificial Intelligence, 2023, 123: 106532.
[9] WANG N, CHEN T K, LIU S M, et al. Deep learning-based visual detection of marine organisms: A survey[J]. Neurocomputing, 2023, 532: 1-32.
[10] ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155.
[11] TAI Y, YANG J, LIU X M, et al. Memnet: A persistent memory network for image restoration[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 4539-4547. doi: 10.1109/ICCV.2017.486.
[12] ZHANG Y L, LI K P, LI K, et al. Residual non-local attention networks for image restoration[J]. In International Conference on Learning Representations, 2019:1-18.
[13] ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]// Computer Vision – ECCV 2018: 15th European Conference. Munich：ECCV，2018: 294-310. doi：https://doi.org/10.48550/arXiv.1807.02758.
[14] IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]// 32nd International Conference on Machine Learning. Computer Science, 2015, 1: 448–456. doi: 10.48550/arXiv.1502.03167.
[15] ZAMIR S W, ARORA A, KHAN S, et al. Multi-stage progressive image restoration[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021: 14821-14831. doi: 10.1109/CVPR46437.2021.01458
[16] 黄冬梅, 王龑, 宋巍,等. 不同颜色模型下自适应直方图拉伸的水下图像增强[J]. 中国图象图形学报，2018，23(5): 640-651．
HUANG D M, WANG Y, SONG W, et al. Underwater image enhancement method using adaptive histogram stretching in different color models[J]． Journal of Image and Graphics，2018，23(5): 640-651. (in Chinese)
[17] IQBAL K, SALAM R A, OSMAN A, et al. Underwater image enhancement using an integrated colour model[J]. IAENG International Journal of Computer Science, 2007, 34(2).
[18] IQBAL K, ODETAYO M, JAMES A, et al. Enhancing the low quality images using unsupervised colour correction method[C]//2010 IEEE International Conference on Systems, Man and Cybernetics. Istanbul: IEEE, 2010: 1703-1709. doi: 10.1109/ICSMC.2010.5642311
[19] 郭继昌, 茹丽, 郭春乐. 基于梯度通道和优化透射率的水下图像复原[J]. 吉林大学学报, 2020, 50(4): 1435-1442.
GUO J C，RU L，GUO C L. Underwater image restoration based on gradient channel and optimized transmission[J]. Journal of Jilin University, 2020, 50(4):1435-1442. (in Chinese)
[20] HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341-2353.
[21] DREWS P, NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]//2013 IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 825-830. doi: 10.1109/ICCVW.2013.113.
[22] GUO Y C, LI H Y, ZHUANG P X. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2019, 45(3): 862-870.
[23] LI J, SKINNER K A, EUSTICE R M, et al. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation letters, 2017, 3(1): 387-394.
[24] ISLAM M J, XIA Y Y, SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227-3234.
[25] 袁红春,赵华龙,高凯.基于多阶段协同处理的水下图像增强[J]. 激光与光电子学进展, 2024, 61(8): 0810003.
YUAN H C, ZHAO H L, GAO K, Underwater image enhancement based on multi-stage collaborative processing[J]. Laser & Optoelectronics Progress, 2024, 61(8): 0810003. (in Chinese)
[26] CHEN T K, WANG N, WANG R F, et al. One-stage CNN detector-based benthonic organisms detection with limited training dataset[J]. Neural Networks, 2021, 144: 247-259.
[27] MOU C, WANG Q, ZHANG J. Deep generalized unfolding networks for image restoration[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE， 2022: 17399-17410. doi: 10.1109/CVPR52688.2022.01688
[28] KINGMA D P, BA J. Adam: A method for stochastic optimization[C]. 2015 International Conference on Learning Representations. [S.l.]: Computer Science,2015: 1-15. doi:10.48550/arXiv.1412.6980.
[29] LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2019, 29: 4376-4389.
[30] YANG M, SOWMYA A. An underwater color image quality evaluation metric[J]. IEEE Transactions on Image Processing, 2015, 24(12): 6062-6071.
[31] PANETTA K, GAO C, AGAIAN S. Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2015, 41(3): 541-551.
[32] FARID H. Blind inverse gamma correction[J]. IEEE Transactions on Image Processing, 2001, 10(10): 1428-1433.
[33] FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]//2018 IEEE International Conference on Robotics and Automation(ICRA). Brisbane: IEEE, 2018: 7159-7165.
[34] WANG Z D, CUN X D, BAO J M, et al. Uformer: A general u-shaped transformer for image restoration[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans：IEEE，2022: 17683-17693. doi: 10.1109/CVPR52688.2022.01716.

Options

文章导航

模态框（Modal）标题

摘要

本文引用格式

Abstract

参考文献