Underwater image restoration and enhancement using depth estimation and gradient descent

WANG Ning, JIA Wei, CHEN Yanzheng, WEI Yi, WU Haojun

doi:10.16411/j.cnki.issn1006-7736.2024.03.001

Journal of Dalian Maritime University >

2024 , Vol. 50 >Issue 3: 1 - 12

DOI: https://doi.org/10.16411/j.cnki.issn1006-7736.2024.03.001

Underwater image restoration and enhancement using depth estimation and gradient descent

Expand

(Marine Engineering College, Dalian Maritime University, Dalian 116026, China)

Online published: 2024-03-13

Fold

Abstract

Due to inherent scattering and absorption, underwater images inevitably suffer from multiple degradations arising from blurring, low contrast and color distortion, thereby seriously deteriorating visual perception. In this paper, a deep learning-based underwater image restoration and enhancement framework (UIRENet) was proposed by virtue of depth estimation and gradient descent strategy. With the aid of convolutional and nonlinear activation function modules, a deep perception network was constructed to achieve scene depth perception maps for different degradation regions, thereby overcoming the dependence of scene depth degradation. A gradient optimization strategy was further proposed to optimize the parameters of convolutional networks and improve the performance of deep network enhancement. Combined with perceptual, edge and underwater color constancy losses, a comprehensive loss function for underwater image enhancement networks was rationally formed. Comprehensive experiments on the UIEB-90, UIEB-M and EUVP datasets show that the UIRENet framework significantly outperforms typical underwater image enhancement methods in terms of reducing underwater image blurriness and improving visual effects. In particular, comparing to CLAHE, ICM, GC, IBLA, DCP, ULAP, FUnIE-GAN, UGAN and Uformer, the objective evaluation metric UIQM can be promoted by 0.3700, 0.6446, 0.5919, 1.3081,1.3032, 1.1672, 0.0593, 0.1329 and 0.0934, respectively.

Key words： underwater image; image restoration; image enhancement; depth estimation; gradient descent strategy; convolutional neural network

Cite this article

WANG Ning, JIA Wei, CHEN Yanzheng, WEI Yi, WU Haojun . Underwater image restoration and enhancement using depth estimation and gradient descent[J]. Journal of Dalian Maritime University, 2024 , 50(3) : 1 -12 . DOI: 10.16411/j.cnki.issn1006-7736.2024.03.001

References

[1] WANG N, WANG Y Y, ER M J. Review on deep learning techniques for marine object recognition: Architectures and algorithms[J]. Control Engineering Practice, 2022, 118: 104458.
[2] ZUIDERVELD K. Contrast limited adaptive histogram equalization[J]. Graphic Gems. 1994,35(69): 474-485.
[3] PENG Y T, COSMAN P C. Underwater image restoration based on image blurriness and light absorption[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594．
[4] SONG W, WANG Y, HUANG D M, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration[C]// Pacific Rim Conference on Multimedia. Switzerland: Springer, 2018: 678–688. doi:10.1007/978-3-030-00776-8_62.
[5] WANG N, CHEN T K, KONG X J, et al. Underwater attentional generative adversarial networks for image enhancement[J]. IEEE Transactions on Human-Machine Systems, 2023, 53(3): 490-500.
[6] LI C, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98: 107038.
[7] FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]//2018 IEEE International Conference on Robotics and Automation(ICRA) . Brisbane, QLD:IEEE, 2018: 7159-7165. doi: 10.1109/ICRA.2018.8460552.
[8] CHEN T K, WANG N, CHEN Y Z, et al. Semantic attention and relative scene depth-guided network for underwater image enhancement[J]. Engineering Applications of Artificial Intelligence, 2023, 123: 106532.
[9] WANG N, CHEN T K, LIU S M, et al. Deep learning-based visual detection of marine organisms: A survey[J]. Neurocomputing, 2023, 532: 1-32.
[10] ZHANG K, ZUO W M, CHEN Y J, et al. Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155.
[11] TAI Y, YANG J, LIU X M, et al. Memnet: A persistent memory network for image restoration[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 4539-4547. doi: 10.1109/ICCV.2017.486.
[12] ZHANG Y L, LI K P, LI K, et al. Residual non-local attention networks for image restoration[J]. In International Conference on Learning Representations, 2019:1-18.
[13] ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]// Computer Vision – ECCV 2018: 15th European Conference. Munich：ECCV，2018: 294-310. doi：https://doi.org/10.48550/arXiv.1807.02758.
[14] IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C]// 32nd International Conference on Machine Learning. Computer Science, 2015, 1: 448–456. doi: 10.48550/arXiv.1502.03167.
[15] ZAMIR S W, ARORA A, KHAN S, et al. Multi-stage progressive image restoration[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021: 14821-14831. doi: 10.1109/CVPR46437.2021.01458
[16] 黄冬梅, 王龑, 宋巍,等. 不同颜色模型下自适应直方图拉伸的水下图像增强[J]. 中国图象图形学报，2018，23(5): 640-651．
HUANG D M, WANG Y, SONG W, et al. Underwater image enhancement method using adaptive histogram stretching in different color models[J]． Journal of Image and Graphics，2018，23(5): 640-651. (in Chinese)
[17] IQBAL K, SALAM R A, OSMAN A, et al. Underwater image enhancement using an integrated colour model[J]. IAENG International Journal of Computer Science, 2007, 34(2).
[18] IQBAL K, ODETAYO M, JAMES A, et al. Enhancing the low quality images using unsupervised colour correction method[C]//2010 IEEE International Conference on Systems, Man and Cybernetics. Istanbul: IEEE, 2010: 1703-1709. doi: 10.1109/ICSMC.2010.5642311
[19] 郭继昌, 茹丽, 郭春乐. 基于梯度通道和优化透射率的水下图像复原[J]. 吉林大学学报, 2020, 50(4): 1435-1442.
GUO J C，RU L，GUO C L. Underwater image restoration based on gradient channel and optimized transmission[J]. Journal of Jilin University, 2020, 50(4):1435-1442. (in Chinese)
[20] HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341-2353.
[21] DREWS P, NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]//2013 IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 825-830. doi: 10.1109/ICCVW.2013.113.
[22] GUO Y C, LI H Y, ZHUANG P X. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2019, 45(3): 862-870.
[23] LI J, SKINNER K A, EUSTICE R M, et al. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation letters, 2017, 3(1): 387-394.
[24] ISLAM M J, XIA Y Y, SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227-3234.
[25] 袁红春,赵华龙,高凯.基于多阶段协同处理的水下图像增强[J]. 激光与光电子学进展, 2024, 61(8): 0810003.
YUAN H C, ZHAO H L, GAO K, Underwater image enhancement based on multi-stage collaborative processing[J]. Laser & Optoelectronics Progress, 2024, 61(8): 0810003. (in Chinese)
[26] CHEN T K, WANG N, WANG R F, et al. One-stage CNN detector-based benthonic organisms detection with limited training dataset[J]. Neural Networks, 2021, 144: 247-259.
[27] MOU C, WANG Q, ZHANG J. Deep generalized unfolding networks for image restoration[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE， 2022: 17399-17410. doi: 10.1109/CVPR52688.2022.01688
[28] KINGMA D P, BA J. Adam: A method for stochastic optimization[C]. 2015 International Conference on Learning Representations. [S.l.]: Computer Science,2015: 1-15. doi:10.48550/arXiv.1412.6980.
[29] LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2019, 29: 4376-4389.
[30] YANG M, SOWMYA A. An underwater color image quality evaluation metric[J]. IEEE Transactions on Image Processing, 2015, 24(12): 6062-6071.
[31] PANETTA K, GAO C, AGAIAN S. Human-visual-system-inspired underwater image quality measures[J]. IEEE Journal of Oceanic Engineering, 2015, 41(3): 541-551.
[32] FARID H. Blind inverse gamma correction[J]. IEEE Transactions on Image Processing, 2001, 10(10): 1428-1433.
[33] FABBRI C, ISLAM M J, SATTAR J. Enhancing underwater imagery using generative adversarial networks[C]//2018 IEEE International Conference on Robotics and Automation(ICRA). Brisbane: IEEE, 2018: 7159-7165.
[34] WANG Z D, CUN X D, BAO J M, et al. Uformer: A general u-shaped transformer for image restoration[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans：IEEE，2022: 17683-17693. doi: 10.1109/CVPR52688.2022.01716.

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References