Driving mechanism and path design of integrating port multimodal knowledge graphs with large models&nbsp;

LI Taoying, ZHANG Yijia, ZENG Qingcheng

doi:10.16411/j.cnki.issn1006-7736.2026.02.002

Journal of Dalian Maritime University >

2026 , Vol. 52 >Issue 2: 10 - 21

DOI: https://doi.org/10.16411/j.cnki.issn1006-7736.2026.02.002

Driving mechanism and path design of integrating port multimodal knowledge graphs with large models

Expand

(a. School of Maritime Economics and Management; b. Information Science and Technology College, Dalian Maritime University, Dalian 116026, China)

Online published: 2026-07-13

Fold

Abstract

Under the background of port intelligence upgrades, the dynamic complexity of business scenarios and the multi-source heterogeneity of data have exposed gaps in the “perception-cognition-decision” chain of current port intelligence systems. Knowledge graphs excel at symbolic reasoning but struggle with handling multi-modal semantics and dynamic knowledge fusion. Large models possess powerful natural language capabilities but face challenges in achieving accurate and reliable decision-making reasoning due to insufficient interpretability and lack of domain knowledge. This situation of “separation of perception and cognition, and disconnection of generation and reasoning” makes it impossible for a single technology to meet the port’s demands for highly reliable, interpretable, and adaptive intelligent decision-making. Therefore, integrating multi-modal knowledge graphs and large models has become a key path for driving the evolution of port intelligence systems. This paper systematically reviewed the related work on artificial intelligence, knowledge graphs, and large models in port intelligence, analyzed the driving mechanism of their integration under the dual perspective of industrial demand and technology integration, and constructed hierarchical technical and functional frameworks. Meanwhile, based on port engineering practice and development status, this paper identified and discussed key technical challenges faced in achieving deep integration, aiming to provide theoretical basis and practical guidance for related research and engineering applications.

Key words： port intelligentization; multimodal knowledge graphs; large models

Cite this article

LI Taoying, ZHANG Yijia, ZENG Qingcheng . Driving mechanism and path design of integrating port multimodal knowledge graphs with large models [J]. Journal of Dalian Maritime University, 2026 , 52(2) : 10 -21 . DOI: 10.16411/j.cnki.issn1006-7736.2026.02.002

References

［1］WANG P, HU Q Y, MEI Q, et al. Intelligent port logistics: a spatiotemporal knowledge graph and AI-agent framework for berth allocation［J］. Advanced Engineering Informatics, 2025, 68:103633.

［2］薛桂香, 王辉, 周卫峰, 等. 基于知识图谱和时空扩散图卷积网络的港口交通流量预测［J］. 计算机应用, 2024, 44(9):2952-2957.

XUE G X, WANG H, ZHOU W F, et al. Port traffic flow prediction based on knowledge graph and spatio-temporal diffusion graph convolutional network［J］. Journal of Computer Applications, 2024, 44(9):2952-2957. (in Chinese)

［3］ZHANG H, GUO J N, GUO S S, et al. An optimization study on dynamic berth allocation based on vessel arrival time prediction［J］. Ocean Engineering, 2025, 332:121321.

［4］CHU Z, YAN R, WANG S A. Vessel turnaround time prediction: a machine learning approach［J］. Ocean & Coastal Management, 2024, 249:107021.

［5］ZHANG Y Z, SU Z Q, LI J K, et al. Doing shipping well with predictions: machine learning-based port congestion analysis［J］. Maritime Policy & Management, 2026,53（2）: 263-284.

［6］KUO S Y, LIN P C, HUANG X R, et al. Cad-transformer: a CNN-transformer hybrid framework for automatic appearance defect classification of shipping containers［J］. IEEE Transactions on Instrumentation and Measurement, 2025, 74:1-21.

［7］WANG H Q, YAN R, AU M H, et al. Federated learning for green shipping optimization and management［J］. Advanced Engineering Informatics, 2023, 56:101994.

［8］JIN J H, MA M Y, JIN H, et al. Container terminal daily gate in and gate out forecasting using machine learning methods［J］. Transport Policy, 2023, 132:163-174.

［9］LIU S L, HUANG L. Application of machine learning in port throughput prediction［C］// Fourth International Conference on Computer Vision, Application, and Algorithm (CVAA 2024).［S.l.:s.n.］, 2025, 13486:691-696.

［10］HUANG H X, YAN Q L, YANG Y, et al. Spatial classification model of port facilities and energy reserve prediction based on deep learning for port management―a case study of Ningbo［J］. Ocean & Coastal Management, 2024, 258:107413.

［11］CUONG T N, KIM H S, LONG L N B, et al. Deep learning-enhanced quantum optimization for integrated job scheduling in container terminals［J］. Engineering Applications of Artificial Intelligence, 2025, 148: 110431.

［12］LV Y Q, ZOU M K, LI J, et al. Dynamic berth allocation under uncertainties based on deep reinforcement learning towards resilient ports［J］. Ocean & Coastal Management, 2024, 252:107113.

［13］LI R F, ZHANG X Y, JIANG L L, et al. An adaptive heuristic algorithm based on reinforcement learning for ship scheduling optimization problem［J］. Ocean & Coastal Management, 2022, 230:106375.

［14］LEE W H, CHO S W. Reinforcement learning approach for outbound container stacking in container terminals［J］. Computers & Industrial Engineering, 2025, 204: 111069.

［15］RUB B, CACACE J, RODRIGUEZ J, et al. VESSELimg: a large UAV-based vessel image dataset for port surveillance［C］//2024 International Conference on Unmanned Aircraft Systems (ICUAS). Chania:IEEE, 2024:76-83.

［16］TIENIN B W, CUI G, NANA Y A T, et al. FedRS-net: a federated learning approach for collaborative multi-modal maritime analytics［C］//2024 27th International Conference on Information Fusion (FUSION). Venice:IEEE, 2024.doi:10.23919/FUSION59988.2024.10706321.

［17］MORANDO E, DAFFIN F C, STAHL T, et al. Multi-sensor data fusion to enhance maritime situational awareness［C］// 2023 IEEE International Geoscience and Remote Sensing Symposium. Pasadena:IEEE, 2023: 6829-6831.

［18］GUPTA S, SAINI N, KUNDU S, et al. Synergizing vision and language in remote sensing: a multimodal approach for enhanced disaster classification in emergency response systems［C］//2024 IEEE International Geoscience and Remote Sensing Symposium. Athens: IEEE, 2024:3278-3281.

［19］YANG W Q, BAO X Y, ZHENG Y, et al. A digital twin framework for large comprehensive ports and a case study of Qingdao Port［J］. The International Journal of Advanced Manufacturing Technology, 2024, 131(11): 5571-5588.

［20］KLAR R, FREDRIKSSON A, ANGELAXIS V. Digital twins for ports: derived from smart city and supply chain twinning experience［J］. IEEE Access, 2023, 11: 71777-71799.

［21］ZHANG X Y, LIU C Y, XU Y, et al. A knowledge graph-based inspection items recommendation method for port state control inspection of LNG carriers［J］. Ocean Engineering, 2024, 313:119434.

［22］ZHU R C, HU X F, BAI Y P, et al. Risk analysis of terrorist attacks on LNG storage tanks at ports［J］. Safety Science, 2021, 137:105192.

［23］LIU S, WANG W Y, ZHONG S P, et al. A graph-based approach for integrating massive data in container terminals with application to scheduling problem［J］. International Journal of Production Research, 2024, 62(16):5945-5965.

［24］尹秀腾, 李响, 刘国辉, 等. 面向港口突发事件的多层网络知识图谱构建［J］. 信息工程大学学报, 2025, 26(2): 189-195.

YIN X T, LI X, LIU G H, et al. Construction of a multi-layer network knowledge graph for port emergency events ［J］. Journal of Information Engineering University, 2025, 26(2):189-195. (in Chinese)

［25］崔明月, 白晓勇, 王清哲. 基于知识图谱的航海信息个性化推荐应用研究［J］. 舰船科学技术, 2024, 46(4): 152-157.

CUI M Y, BAI X Y, WANG Q Z. Research on personalized recommendation of navigation information based on knowledge graph［J］. Ship Science and Technology, 2024, 46(4):152-157. (in Chinese)

［26］CHEN N Y, YANG A R, WU H, et al. SEMINT: an LLM-empowered long-term vessel trajectory prediction framework［J］. International Journal of Geographical Information Science, 2025,39(9):1938-1972.

［27］DIN M U, AKRAM W, BAKHT A B, et al. Maritime mission planning for unmanned surface vessel using large language model［C］//2025 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR). Palermo: IEEE, 2025:1-6.

［28］娄云洁, 艾明飞, 庄术洁, 等. DK-Port：基于大语言模型和强化学习的港口自动驾驶仿真环境构建与验证［J］. 智能科学与技术学报, 2025, 7(1):98-113.

LOU Y J, AI M F, ZHUANG S J, et al. DK-Port: construction and validation of port autonomous driving simulation environment based on large language models and reinforcement learning［J］. Chinese Journal of Intelligent Science and Technology, 2025, 7(1):98-113. (in Chinese)

［29］LV C C, SONG N, NIE J, et al. MS-LIP: Multiscale semantic information integration with large language models for marine prediction［J］. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63:1-15.

［30］LAN L, WANG F X, ZHENG X T, et al. Efficient prompt tuning of large vision-language model for fine-grained ship classification［J］. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63:1-10.

［31］郭漩, 李博浩, 虞鹃秀, 等. 大语言模型增强的海运网络社团发现可视分析方法[J/OL]. 计算机辅助设计与图形学学报, 2025. （2025-06-20）[2025-08-19]. https://link.cnki.net/urlid/11.2925.TP.20250620.0902.002. GUO X, LI B H, YU J X, et al. Large language model enhanced visual analysis method for community detection in maritime networks[J/OL]. Journal of Computer Aided Design & Computer Graphics. 2025. （2025-06-20）[2025-08-19]. https://link.cnki.net/urlid/11.2925.TP.20250620.0902.002. (in Chinese)

［32］KIM Y, LEE K, HAN Y. Exploring large language models (LLMs) based chatbots for pump maintenance in ships［J］. Ships and Offshore Structures, 2025: 1-19.

［33］杨玉林, 胡伟, 贾晓, 等. 基于大模型技术的港口设备运维助手智能体系统［J］. 港口科技, 2025(2): 1-5.

YANG Y L, HU W, JIA X, et al. An intelligent-agent system for port equipment O&M based on large-model technology［J］. Port Science & Technology, 2025(2): 1-5. (in Chinese)

Options

Outlines

模态框（Modal）标题

Abstract

Cite this article

References