Intelligent horizon detection and vessel classification based on deep learning in marine environments

Abstract

The interest in remote-controlled and autonomous ships for marine industries has increased significantly over the past few years. The keys to successful autonomous ships are safe vessel navigation and collision avoidance. Therefore, intelligent situational awareness in marine environments is a critical part of autonomous navigation and is necessary for compliance with the law. Intelligent situational awareness in marine environments consists of three main steps: horizon detection, object detection and tracking, and object classification. Among these, horizon detection is the most essential for situational awareness because its outcome can greatly affect the performance of other tasks and the entire system. Additionally, autonomous ships must classify other types of ships to comply with maritime laws. Therefore, in this thesis, we explore a method to efficiently utilize the superior power of a convolutional neural network (CNN) for horizon detection and ship classification, which are essential tasks for autonomous navigation in marine environments. We first propose two methods for identifying the horizon accurately by utilizing a CNN. The first is a method that accurately detects the horizon by combining a multi-scale approach with a CNN. We analyzed the effectiveness of CNNs for the horizon detection task and confirmed that horizon edge detection utilizing a CNN is effective for improving the accuracy of horizon detection. Although the proposed method has shown reliable overall performance for horizon detection, the performance decreases when edges cannot be detected accurately. Therefore, the second proposed method does not rely on edge detection, but instead utilizes semantic segmentation based on a pyramid scene parsing network to estimate the boundary of the sea. Scene segmentation allows the proposed method to identify the horizon, regardless of the presence of edge information in the input image. We compared the performance of the proposed method to that of state-of-the-art methods on the largest publicly available database containing complex maritime scenes and the experimental results demonstrate that the proposed methods can identify the horizon more accurately than comparable state-of-the-art methods. We also analyzed the performance of recent deep neural networks in the field of ship classification. We selected three CNNs (VGGNet, Inception v3, and ResNet) and utilized the largest publicly available ship database, which consists of 26 superclasses and approximately 240,000 images, to compare network performances on vessel classification tasks. The experimental results demonstrate that ResNet achieve the best performance in terms of accuracy of ship classification. Although state-of-the-art CNNs achieve better performance than previous CNNs, there is still significant room for improvement.