Electronic image stabilization technology of shipborne camera based on sea-sky line detection

  • SHAN Xiong-fei ,
  • ZHAO De-peng ,
  • PAN Ming-yang ,
  • WANG De-qiang
Expand
  • (Navigation College,Dalian Maritime University, Dalian 116026,China)

Received date: 2020-04-30

  Revised date: 2020-06-05

  Online published: 2020-06-05

Abstract

In the background of sea and sky, due to the complex sea conditions and the ship’s six-degree-of-freedom motion, it is difficult for the shipborne camera to obtain stable video sequence, which is not conducive to the detection of sea targets. Aiming at this problem, an electronic image stabilization algorithm based on sea-sky line (SSL) detection was proposed by fusing data of shipbrone cameras and inertial sensors. The algorithm consists of three steps, namely SSL estimation, SSL detection and point-line model (PLM). Firstly, the camera motion attitude model was constructed to estimate the candidate region of SSL in the image; then, the edge detection and Hough transform were applied to extract the optimal SSL in the candidate region; finally, the PLM was applied to stabilize the original video in the area near the SSL, and the stable video sequence was obtained. In the experiment, the real ship data set matched with the camera’s motion attitude was constructed, and the effectiveness of the SSL detection algorithm was verified by comparative experiments. The experiment results show that the image stabilization effect is significantly better than other algorithms, which provides reliable technical support for the visual development of unmanned ships.

Cite this article

SHAN Xiong-fei , ZHAO De-peng , PAN Ming-yang , WANG De-qiang . Electronic image stabilization technology of shipborne camera based on sea-sky line detection[J]. Journal of Dalian Maritime University, 2020 , 46(3) : 76 -86 . DOI: 10.16411/j.cnki.issn1006-7736.2020.03.009

References

[1]邱家涛.电子稳像算法和视觉跟踪算法研究[D].西安:西安电子科技大学,2013. [2]QIU J T.A study of electronic image stabilization algorithms and visual tracking algorithm[D]. Xi an: Xidian University, 2013. (in Chinese) [3] HSIAO Jen-Pin, HSU Cheng-Chung, SHIH Tzu-Chiang.The real-time video stabilization for the rescue robot[C]. 2009 ICCAS-SICE. Fukuoka, Japan, 2009:4364-4369. [4] SHEN H, PAN Q, CHENG Y M, et al.Fast video stabilization algorithm for UAV[C]. 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems. Shanghai, China, 2009: 542-546. [5] LIU Kexiang, QIAN Jie, YANG Ruike.Block Matching Algorithm Based on RANSAC Algorithm[C]. 2010 International Conference on Image Analysis and Signal Processing. Zhejiang, China, 2010:223-227. [6] LEE Ken-Yi, CHUANG Yung-Yu, CHEN Bing-Yu, et al.Video stabilization using robust feature trajectories[C]. 2009 IEEE 12th International Conference on Computer Vision. Kyoto, Japan, 2009:1397-1404. [7]WANG C T, KIM J H, BYUN K Y, et al.Robust digital image stabilization using the Kalman filter[J].IEEE Transactions Consumer Electronics, 2009, 55(1):6-14 [8] BATTIATO Sebastiano, GALLO Giovanni, PUGLISI Giovanni, et al.SIFT features tracking for video stabilization[C]. // 14th International Conference on Image Analysis and Processing (ICIAP 2007). Modena, Italy, 2007:825-830. [9] CHEN Chao-Ho, CHEN Tsing-Yi, HU Wu-Chih, et al.Video Stabilization for Fast Moving Camera Based on Feature Point Classification[C]. 2015 Third International Conference on Robot, Vision and Signal Processing (RVSP). Taiwan, China, 2015:10-13. [10] HU W C, CHEN C H, SU Y J, et al.Feature-based real-time video stabilization for vehicle video recorder system[J].Multimedia Tools and Applications, 2018, 77(5):5107-5127 [11] HU W C, CHEN C H, CHEN T Y, et al.Real-time video stabilization for fast-moving vehicle cameras[J].Multimedia Tools and Applications, 2018, 77(1):1237-1260 [12] HU W C, CHEN C H, CHEN C M, et al.[J]. . 2014:343-353.[J].Intelligent Data analysis and its Applications, 2014, 1(297):343-353 [13] HU W C, CHEN C H, CHEN T Y, et al.[J]. . 2015, 30:164-180.[J].Journal of Visual Communication and Image Representation, 2015, 30:164-180 [14] LING Q, DENG S B, LI F, et al.[J]. . 2018, 28(3):561-572.[J].IEEE Transactions on Circuits and Systems for Video Technology, 2018, 28(3):561-572 [15] ZHAO M D, DENG S B, LING Q.[J]. . 2019, 7:13422-13432.[J].IEEE Access, 2019, 7:13422-13432 [16] ZHAO M D, LING Q.A Robust Traffic Video Stabilization Method Assisted by Foreground Feature Trajectories[J]. IEEE Access, 2019, 7:42921-42933 [17] LIU S C, YUAN L, TAN P, et al.Spatially Smooth Optical Flow for Video Stabilization[C]. 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA. 2014:4209-4216. [18] LIU S C, TAN P, YUAN L, et al.Minimum Latency Online Video Stabilization[C]. 14th European Conference on Computer Vision (ECCV). Amsterdam, NETHERLANDS. 2016:800-815. [19] REN Z H, LI J J, LIU S C, et al.Meshflow video denoising[C]. 2017 IEEE International Conference on Image Processing (ICIP). Beijing, China, 2017:2966-2970. [20]郭禹.航海学[M].大连:大连海事大学出版社, 2005:19-20. [21]GUO Yu.Navigation [M]. Dalian: Maritime University Press, 2005:19-20. (in Chinese) [22]Leandro A F, Fernandes M, Oliveira.Oliveira. .n[J]. 2008, 41(1)299-314.[J]. Pattern Recognition, 2008, 41(1):299-314 [23] SHI J B, TOMASI C.Good Features to Track[C]. IEEE Conference on Computer Vision and Pattern Recognition. Seattle, 1994:593-600. [24] SATOPAA V, ALBRECHT J, IRWIN D, et al.Finding a " Kneedle" in a Haystack: Detecting Knee Points in System Behavior[C]. 2011 31st International Conference on Distributed Computing Systems Workshops. Minneapolis, MN, USA. 2011:166-171. [25] FEFILATYEV S, GOLDGOF D, SHREVE M, et al.[J]. , 2012, 54:1-12.[J].Ocean Engineering, 2012, 54:1-12 [26] ZHANG Y, LI Q Z, ZANG F N.Ship detection for visual maritime surveillance from non-stationary platforms[J].Ocean Engineering, 2017, 141:53-63 [27] GRUNDMANN M, KWATRA V, ESSA I.Auto-directed video stabilization with robust L1 optimal camera paths[C]. IEEE Conference Computer Vision Pattern Recognition (CVPR 2011). 2011:225–232.
Outlines

/