AI in Maritime Security: Applications, Challenges, Future Directions, and Key Data Sources

1. Introduction

The vastness of the world’s oceans and seas presents both a vital conduit for global trade and a complex frontier for national and international security. With more than 80% of international trade conducted through maritime routes, and up to 99% of the global data transmission taking place through undersea cables, maritime security has become an indispensable component of global stability [1]. While maritime security is of critical importance, it is facing an increasing number of security threats deriving from illicit activities in waters, competition for natural resources, and geopolitical rivalries. The most pressing security threats today include drug trafficking [2], human smuggling [3], unauthorised fishing [4], espionage and spy vessels [5], oil spills and other environmental violations [6].

The increasing sophistication and scale of these illegal activities in the waters pose a variety of threats that are extremely difficult to monitor, predict, and mitigate for Coast Guard and other law enforcement agencies using traditional surveillance methods. This is because, the traditional surveillance methods, mainly relying on in-person observation, coastal radars, patrol vessels, satellite imagery and aerial reconnaissance, suffer from high costs, limited coverage and delayed response. Moreover, the sheer amount of maritime data from AIS (Automatic Identification Systems), SAR (Synthetic Aperture Radar), and drone footage is not feasible to monitor manually in a timely manner [7].

The recent developments in artificial intelligence (AI) and deep learning techniques present a promising solution for automated intelligent systems to monitor and detect maritime security threats and act promptly [7]. Applications of deep learning, a specialised subfield of machine learning, are emerging as a scalable transformative force in maritime domain awareness and threat detection that enables the processing of vast amounts of multimodal data coming from various sources with high accuracy.

Deep learning focuses on learning patterns in large datasets using hierarchical structures known as artificial neural networks (ANNs). These networks, inspired by the human brain, are implemented using layers of interconnected nodes where each layer transforms the input data in increasingly abstract ways. ANNs come in different types, for example, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory (LSTM), just to name a few. Each type has its pros and cons and the choice depends on the application in hand. For example, a CNN can detect vessels from satellite and drone imagery [8,9,10], and recurrent neural networks (RNNs) can be used to analyse temporal patterns in a video feed showing vessel movements to detect anomalies [11]. Various object detection algorithms such as “You Only Look Once" (YOLO), RetinaNet, Vision Transformers (ViT), and DeepSORT, enable real-time detection of illegal fishing boats, unflagged ships, or vessels engaged in smuggling [12,13,14,15,16].

In a typical deep learning model, such as a CNN for image data or an RNN for sequential or video data, the network is trained using large datasets and various optimisation algorithms to maximise the prediction accuracy. As the network processes data through multiple hidden layers, it automatically learns relevant features, for example, edges, shapes, or motion patterns. This ability to self-learn complex, nonlinear relationships makes deep learning a powerful tool for maritime security tasks such as detecting suspicious vessels or an object on the sea surface [3,16]. In addition to threat detection, deep learning can also be used for a number of related objectives, e.g., optimising patrolling routes for coastguards, and simulating potential threat scenarios for training purposes.

1.1. Maritime Security Threats and Illegal Activities

Modern maritime threats are multifaceted, often transnational, and highly dynamic. In the following, we review the most pressing threats:

1.1.1. Illegal Migration and Border Crossings

A number of maritime borders including the Mediterranean Sea and the English Channel have seen a rise in illegal migrations using overcrowded and unsafe ships [17]. The lack of an effective and real-time surveillance systems makes it difficult for authorities to intervene promptly which often leads to humanitarian crises and political issues.

1.1.2. Drug Trafficking and Human Smuggling

Organised crime groups are increasingly using maritime routes in many parts of the world. These organised crime groups are using sea routes to smuggle illegal drugs and migrants. Often, small and unregistered boats are used to avoid detection and exploit the limitations of manually operated surveillance systems. These unlawful activities not only put human lives at risk but also support other criminal networks [2].

1.1.3. Illegal Fishing

Illegal, Unreported, and Unregulated (IUU) fishing activities in the coastal and deep sea are a threat to fish stocks and a challenge to food security. These activities are costing the economy billions of USD annually [18]. These activities often occur at places in the sea which are protected or disputed, making detection and law enforcement a challenge. The illegal fishing boats use sophisticated evasion techniques to deceive or completely avoid the tracking and monitoring systems [19].

1.1.4. Environmental Threats and Marine Pollution

Environmental threats and marine pollution have become a major concern in today’s industrialised era. Oil spills and illegal dumping of waste products and hazardous materials into seawaters represent a critical environmental concern. These activities not only damage marine ecosystems but also threaten coastal economies and public health. Since these activities are often carried out far from shore, it becomes a challenge to detect such activities using traditional observation methods [20,21].

1.2. Limitations of Traditional Surveillance Approaches

Maritime security and monitoring have been relying on traditional mechanisms comprising coastal radar systems, satellite imagery, patrolling vessels of coastal guard agencies, AIS, and human observations and intelligence. These approaches worked well over time, but are not sufficient with the rise of sophistication criminals are using to threaten. The main limitations of traditional systems are:

• Coverage Gaps: Traditional surveillance systems offer limited coverage, particularly in the deep sea where patrolling infrastructure is insufficient.
• Evasion Tactics: Various malicious actors working in crime groups leverage technology to manipulate AIS data, operate without transponders, or exploit blind spots in satellite coverage.
• Data Overload and Latency: The volume and speed of data generated by various sensors mounted in the sea and satellite systems are too much to be processed by human operators causing delayed responses to threats or even missing them completely.

1.3. Scope and Objectives of the Paper

This survey paper aims to provide a comprehensive overview of the state-of-the-art deep learning techniques applied to maritime security. The objectives are threefold:

Firstly, we explore applications of deep learning in detecting and mitigating maritime threats, including illegal trafficking, unauthorised fishing, environmental violations, and other anomalies at sea. Secondly, we focused on the challenges and limitations associated with deploying deep learning models in operational maritime contexts, such as data scarcity, model generalisation, real-time performance, and sensor fusion. Finally, we identify future research directions and technological advancements that can enhance the robustness and scalability of deep learning systems for maritime security.

The rest of the paper is organised as follows: Section 2 reviews the key deep learning techniques and data sources used for object and anomaly detection and tracking. Section 3 discusses deep learning techniques for maritime surveillance and situational awareness. Section 4 outlines examples of maritime security applications, and Section 5 includes key data sources for the training and testing of deep learning methods in maritime domain. These datasets are either publicly available or available on request from the concerned authors. Section 6 outlines the main challenges and future direction of deep learning in maritime security. Finally, a discussion and conclusions are given in Section 7.

2. Deep Learning for Maritime Object Detection and Tracking

Maritime security is a critical global concern that demands advanced technological solutions to strengthen surveillance and response capabilities. In maritime research, deep learning has emerged as a promising tool for enhancing maritime object detection and tracking. Deep learning techniques enable automated analysis of extensive maritime datasets, integrating sensor data, unmanned aerial/surface/underwater vehicles (UxVs) and satellite imagery, and video feeds to detect ships, vessels, and anomalies in real-time with outstanding accuracy [22,23]. This capability not only enhances situational awareness but also supports timely decision-making crucial for maritime operations, security, and environmental protection [24].

This section explores deep learning applications on problems such as vessel detection, anomaly detection in vessel behaviour, and tracking of maritime objects.

2.1. Vessel Detection

Deep learning application of vessel detection is an important research area. Here, researchers have developed deep learning techniques using several types of data; for example, optical imagery from satellite, drones, and surface vehicles; SAR, radar and AIS data.

2.1.1. Satellite Imagery

Optical satellite imagery (RGB) captures visible light and is effective in detecting vessels based on their visual appearance and characteristics. It provides detailed spatial information and can differentiate between different types of vessels. SAR is also equally valuable for vessel detection, especially in adverse weather conditions and during nighttime. It operates by transmitting microwave signals and capturing reflections, which are then processed to generate images. SAR can detect vessels based on their radar cross-section, allowing for all-weather surveillance and it provides valuable data for maritime surveillance due to its wide coverage and all-weather capabilities [25]. Generally, optical imagery captures visual details, while SAR is effective in detecting vessels regardless of weather conditions or lighting [26].

Figure 1 illustrates how SAR and optical imagery may be achieved through satellites during daylight and nighttime, dark, or unclear visual conditions. SAR uses radio waves, which can penetrate through clouds and operate both in day and nighttimes. This makes SAR ideal for consistent ship detection under all weather and lighting conditions. On the other hand, optical imagery (RGB) relies on sunlight and visible spectrum. It provides high-resolution, colour images, but performance drops in cloudy or nighttime conditions.

2.1.2. Aerial Imagery

Aerial platforms, such as drones, unmanned aerial vehicles (UAVs), and other aircrafts offer high-resolution imagery suitable for detailed vessel detection and tracking [27]. These platforms enhance spatial resolution and temporal coverage, enabling real-time surveillance over vast maritime areas [28].

2.1.3. Surface Imagery

Images obtained from unmanned surface vehicles (USVs) and coastguard ships also play an important role in detecting ships or situations that may pose a risk to maritime safety (e.g., oil spills).

2.1.4. Radar Data

Radar systems play a crucial role in maritime surveillance by detecting vessels through radio waves [29]. Radar data integration with other sensor data improves detection accuracy and allows comprehensive situational awareness in maritime environments [30].

2.1.5. AIS Data

An AIS (Figure 2) is required to autonomously transmit and receive essential navigational and safety-related data that may include the identity, type, position, direction, and speed of the vessel to manage maritime traffic effectively and safely, even when the vessels are not within visual range of each other. Furthermore, AIS also facilitates seamless data exchange between ships and ports, thereby enhancing maritime situational awareness and operational safety [31].

Deep learning techniques have been rigorously developed that use AIS data, combined with imagery and radar, to enhance vessel tracking and identification capabilities for enhanced maritime surveillance [19,32].

2.1.6. Integration of Data from Different Sources

In deep learning techniques, multimodal data fusion approaches (as illustrated in Figure 3) for efficient maritime security can be used where satellite, UxV, radar, and AIS data are combined [33]. It enables insights to be drawn from multiple data sources, resulting in more accurate and valid predictions.

2.1.7. Challenges in Vessel Detection

Vessel detection in maritime security presents several challenges, including the accurate identification of small objects, adverse weather conditions, cluttered backgrounds, and reflections on the sea surface [34]. These factors can significantly reduce the performance of detection algorithms [35]. Effectively addressing these issues is essential to enhance the robustness and reliability of deep learning-based maritime surveillance systems.

2.2. Anomaly Detection in Vessel Behaviour

Anomaly detection in vessel behaviour – distinguishing between legitimate deviations and high-risk anomalies – has emerged as a critical component of maritime security, particularly given the growing volume of global maritime traffic and the sophistication of illicit activities. As highlighted in [36], the detection of abnormal vessel movements is essential to identify potential threats such as smuggling, illegal fishing, and piracy. Traditionally, rule-based systems were used to define what constitutes “normal" behaviour, but such methods often lack the flexibility to adapt to dynamic maritime environments [37]. Recent advances in deep learning have opened new avenues for modelling vessel trajectories and behavioural patterns with improved accuracy and robustness [38].

It is important to mention that, in the problem of anomaly detection using deep learning, the relevant datasets may be imbalanced since anomalous events are rare (minority class) as compared to normal operations. This may lead to bias in deep learning models towards majority class which is normal behaviours. To mitigate these issues, several strategies have been employed. For example, data augmentation is performed to generate synthetic samples for the minority class using techniques like synthetic minority over-sampling technique (SMOTE) [39]. Otherwise, in case of data augmentation deems infeasible, several other techniques may be used. For example, isolation forests (IF) are also effective in identifying anomalies without requiring balanced datasets. These algorithms isolate anomalies based on their distinct characteristics, making them suitable for sparse and imbalanced data scenarios [40]. Autoencoders and variational autoencoders (VAEs) learn representations of normal behaviour and can detect deviations indicative of anomalies. These models are particularly useful when labelled anomaly data is scarce [41]. Cost-sensitive learning assigns higher misclassification costs to minority classes to help models focus on correctly identifying anomalies.

Implementing these strategies enhances the capability of maritime anomaly detection systems to accurately identify rare events, thereby improving maritime safety and security.

2.2.1. Analysing AIS Data for Unusual Patterns

Unusual vessel behaviours may include abrupt speed variations, deviations from established routes, and loitering. In this regard, AIS data may be leveraged to detect anomalous vessel behaviours for maritime surveillance and security. These behavioural anomalies often signal potential threats, including illegal fishing, smuggling, or unauthorised maritime activities.

Speed anomalies, characterised by sudden accelerations or decelerations, can indicate evasive manoeuvres or mechanical issues. Route deviations, where vessels diverge from customary maritime paths, may suggest illicit intentions or navigational errors. Loitering behaviour, defined by prolonged stationary periods or repetitive movements within confined areas, is frequently associated with suspicious activities, especially when exhibited by cargo or tanker ships, which typically maintain consistent transit patterns [42].

To effectively identify these anomalies, researchers have developed various computational methods. For instance, [43] introduced GeoTrackNet, a deep learning model that learns probabilistic representations of AIS tracks to detect deviations from normal maritime behaviour. The authors [44] developed a semi-supervised deep learning approach for ship trajectory classification based on AIS data. They utilised both kinematic and static information of AIS messages to extract vessel trajectories for the classification task. Similarly, [45] employed RNN to model vessel trajectories, enabling the detection of outlier movements indicative of anomalous behaviour. These advanced methodologies underscore the importance of leveraging AIS data for real-time anomaly detection.

2.2.2. Combining AIS with Contextual Information

While AIS data provides essential insights into vessel movements, its standalone application may lead to false positives, as it does not account for external factors influencing vessel behaviour [46]. Incorporating contextual data – such as, meteorological conditions and port activities – allows for a more nuanced understanding of maritime operations, thereby improving anomaly detection efficacy [47]. For example, weather conditions significantly impact vessel navigation and behaviour. Adverse weather events, such as storms or high winds, can cause deviations in vessel speed and course, which, if unaccounted for, may be misclassified as anomalous. The authors in [48] investigated the integration of tide and weather data into vessel track prediction models and found that while the inclusion of such data did not universally enhance predictive accuracy, it provided valuable context for interpreting vessel movements, particularly in complex waterways.

Furthermore, port activity data offer another layer of contextual information that can refine anomaly detection. Vessels may exhibit a typical behaviour, such as loitering or sudden speed changes, due to port congestion, scheduling delays, or operational requirements. The authors in [49] developed metrics for assessing vessel and port efficiency by validating AIS data against port activity records, demonstrating that contextualising AIS data with port operations can elucidate reasons behind seemingly anomalous vessel behaviours [47]. The authors in [50] proposed a framework that incorporates environmental and contextual data to verify anomalies detected in AIS data, thereby filtering out false positives caused by benign factors such as weather-induced deviations.

2.2.3. Using Sequence-Based Models

In the context of maritime anomaly detection, sequence-based models such as RNN, LSTM, Gated Recurrent Units (GRU), and Transformers have demonstrated significant efficacy in modelling the temporal dynamics inherent in AIS data [51]. These models are adept at capturing complex temporal dependencies, thereby facilitating the identification of anomalous vessel behaviours. For instance, [52] employed an encoder-decoder LSTM architecture to predict vessel trajectories, demonstrating improved performance over baseline models in capturing the spatiotemporal patterns of maritime movements.

Transformers, characterised by their self-attention mechanisms, have emerged as powerful tools for time-series analysis, particularly in capturing global dependencies within sequences. The authors in [48] utilised a Transformer model to predict vessel tracks in inland waterways, incorporating environmental factors such as tide and weather data to improve predictive performance.

Furthermore, hybrid models that integrate Transformers with variational autoencoders (VAEs) have been proposed to leverage the strengths of both architectures. Such models have demonstrated superior performance in unsupervised anomaly detection by effectively reconstructing normal vessel trajectories and identifying deviations indicative of anomalous behaviour [53].

3. Deep Learning for Maritime Surveillance and Situational Awareness

Surveillance and situational awareness, in oceans with increasing complex maritime activities, are challenges in the presence of traditional surveillance methods. Advanced deep learning techniques, using diverse maritime data sources such as UxV videos, satellite imagery, radar signals, and AIS data, have helped to develop automated, scalable, and highly accurate systems.

3.1. Maritime Image and Video Analysis

Deep learning has dramatically improved ship and object classification in maritime imagery. Modern CNN-based detectors (YOLO, Single Shot MultiBox Detector – SSD, RCNN, etc.) learn feature hierarchies from raw images. Two-stage detectors (e.g., Faster RCNN family) achieve high accuracy by generating region proposals then classifying them, while one-stage detectors (YOLO, SSD) fuse detection and classification in one pass for real-time performance. For example, recent surveys note that YOLOv3/v4/v5 have been widely used for maritime ship detection and counting. In practical tests, YOLO-based detectors can reliably find multiple vessel types (e.g., bulk cargo, container, fishing, general cargo, passenger) with high accuracy [54].

Object classification must cope with challenging conditions. Studies report that lighting changes, sun glare, waves and reflections often obscure vessels and cause false positives [34,35]. For instance, the authors in [54] observe that night/day illumination variance and water reflections “obscure the outlines of vessels and lead to false positives, significantly reducing the effectiveness of image capture and analysis". Consequently, state-of-the-art methods augment raw CNNs with attention modules or multi-scale features. One recent system YOLOv5-ASC adds an “attention-based receptive field enhancement" module and deformable convolutions to focus on multi-scale vessel features, yielding improved detection accuracy on remotesensing maritime images [10]. Another research in [9] introduced a multi-level, multi-head attention architecture on YOLOv8 to fuse EO (electro-optical) and infrared (IR) imagery; this scale-sensitive attention approach explicitly models different scales and lighting, giving a more robust detection of ships of various sizes.

3.1.1. Event and Activity Recognition

Beyond static objects, deep learning models are used to recognise events (e.g., collisions, loitering) and anomalous behaviour. Early work on maritime anomaly detection often relied on AIS trajectories, but recent research explores vision-based methods. For example, a graph-based anomaly detector was developed in [55] that tracks vessels via CCTV (using YOLOv7+StrongSORT) and constructs trajectory graphs to distinguish normal vs. anomalous motion . This approach identifies deviations in learned “normal" paths, flagging abnormal maneuvers (e.g., erratic course changes) that may indicate accidents or illicit activity.

3.1.2. Scene Understanding and Context

Deep neural networks are also used for semantic segmentation of the maritime scene, aiding contextual analysis (e.g., separating water, sky, land, and obstacles). For example, the research in [56] proposes a CNN encoder–decoder with superpixel refinement to segment water, sky, and obstacles for USV navigation. Their model (trained on publicly available maritime datasets) delivers precise obstacle masks even on complex backgrounds. Semantic segmentation yields a richer “understanding" of the scene – beyond classifying ships, it identifies navigable water vs. hazards – which can assist path planning and collision avoidance.

3.1.3. Video Surveillance for Tracking and Anomaly

Video analytics further extends image-based models with temporal processing. Deep trackers (e.g., DeepSORT) can maintain identity over frames after CNN detection, enabling counting and speed estimation. For example, a real-time port monitoring system combined YOLO detection with Kalman-tracking to count vessel types on land-based cameras [54]. Anomaly detection in video often uses deep autoencoders or graph models on trajectories as noted above [55]. For activity recognition (e.g., identifying suspicious boarding, search-and-rescue), some studies adapt human action networks (e.g., 3D-CNNs, LSTMs) to maritime scenes, although this remains underexplored. Case studies from port security exercises show CNN outputs overlaid on video feeds (bounding boxes, class labels) to alert operators about potential collisions or unauthorised vessels. In summary, deep video surveillance systems integrate detectors and trackers to continuously monitor maritime zones, with the goal of raising alerts on unusual or unsafe events.

Table 1 provides a comparative overview of state-of-the-art deep learning models applied to maritime image and video analysis in the context of surveillance and situational awareness. It summarises the model architecture, intended application domain, key strengths, known limitations, and the reported performance metrics.

3.2. Fusion of Multi-Sensor Data

Maritime surveillance increasingly relies on fusing heterogeneous sensors. No single sensor is sufficient in all conditions; for example, Electro-Optical (EO) cameras provide detailed imagery but struggle at night or in fog, whereas thermal cameras and radar are effective in poor light. Radar+vision fusion can track objects beyond visual range and refine classification (radar velocity plus camera shape). Hence, to exploit complementary strengths, modern systems fuse AIS, radar, EO/IR cameras, and even acoustic or LiDAR sensors via deep learning [63]. The general principle is that multi-modal fusion improves robustness: as one review notes, “no single sensor can guarantee sufficient reliability or accuracy in all different situations. Therefore, sensor fusion, which combines data from different sensors, is used to provide complementary information about the surrounding environment" [64]. The empirical results of [63] confirm the gains: RGB+IR fusion CNN significantly outperformed single-sensor model alone in detecting ships under varying weather. In GMvA, spatiotemporal attention across AIS+CCTV improved target association by 10–20% over baselines [65].

Overall, deep fusion architectures (multi-stream CNNs with attention or GNNs) are crucial for leveraging AIS, radar, EO/IR, acoustic, and LiDAR together. These systems yield a more complete maritime picture, reducing false alarms and filling in sensor gaps.

3.2.1. Fusion Architecture

Deep learning enables fusion at different stages. Fusion approaches are typically categorised as early (data/pixel-level), middle (feature-level) or late (decision-level). For example, [63] implemented three CNN architectures for RGB+IR fusion: early fusion (stacking raw images), feature fusion (merging CNN feature maps), and late fusion (combining separate detector outputs). Attention mechanisms can also learn to weight modalities. One maritime study [63] fused radar rangeDoppler maps with optical images via a deep network: region proposals from radar and EO were combined in a CNN for final detection, improving accuracy under clutter. Graph neural networks have been applied to fuse AIS and video: a graph learning-driven multi-vessel association model (GMvA) integrated AIS time-series with CCTV detections, using spatiotemporal graph attention to associate tracks [65]. By learning a graph embedding of vessel trajectories, GMvA achieved more reliable matching in high-traffic scenarios than either sensor alone. Similarly, multi-head attention modules have been designed to merge EO and IR features in YOLO, making detection sensitive to multi-scale objects [9,10].

3.2.2. Sensor-Specific Examples

AIS data broadcast by vessels provide useful information such as identity, type and route information. Fusing AIS with imagery yields richer situational awareness. For instance, [66] fused YOLOv5-based camera detections with AIS data: the visual system estimated a ship’s bearing and distance from a monocular camera, then matched these with AIS-reported positions to confirm vessel identity. This hybrid approach corrected misclassifications under poor visibility and achieved 5% association accuracy on real data. In another research [67], MIT’s AUV Lab released a dataset (“Philos" series) combining AIS, radar and video of a ferry; deep learning was used to project AIS fixes onto radar imagery, aiding object tracking.

In the maritime literature, radar and LiDAR fusion with vision have also been studied. Radar excels at long-range, all-weather detection; [63] developed a CNN that combines EO and radar proposals to classify vessels, showing decision-level fusion (CNN on merged proposals) outperforms single-modality detection. LiDAR is less common on ships but used on USVs; it provides 3D structure that complements camera colour. Acoustic sensors (buoys or hydrophones) are used for passive surveillance of fast vessels or submarines. A recent platform (OpenEar™ buoys) uses deep sound classification to detect propeller noise; network outputs are sent in real time to a cloud system that correlates tracks with AIS and radar for full Maritime Domain Awareness (MDA) [13].

Table 2 categorises the main fusion strategies used in maritime surveillance, detailing sensor combinations, deep learning techniques employed, application contexts, and key performance highlights as reported in recent literature.

3.3. Maritime Domain Awareness Systems Using Deep Learning

Maritime Domain Awareness (MDA) refers to the integrated understanding of vessel activities for security and safety. Modern MDA platforms tightly couple deep learning analytics with operational decision support. Architecturally, an MDA system ingests streams from satellites, sensors, AIS, and social/OSINT data into a processing pipeline. Deep learning models are run onboard or in the cloud to detect/track vessels, classify behaviours, and flag anomalies in real time [64]. For example, USV platforms have sensor suites (cameras, LiDAR, sonar, radar) feeding a CNN+SLAM module that outputs a continuous situational map. These outputs (e.g., bounding boxes on video, classification labels, ship trajectories) are fused with AIS and Geographic Information System (GIS) databases to yield a unified maritime picture. In addition, real-time analysis is critical. Systems often use edge computing: lightweight CNNs (e.g., Tiny YOLO variants) run on embedded GPUs to detect objects at high frame rates, while heavier analysis (graph association, behaviour classification) can run on shore or cloud servers. Low-latency fusion and messaging frameworks (e.g., Data Distribution Service–DDS, Robot Operating System–ROS) synchronise multi-sensor data as in [69] so that deep algorithms use consistent inputs. An example platform (“eM/S Salama USV") collects synchronised RGB, thermal, stereo, and LiDAR data, storing it for model training and processing it in situ for navigational awareness.

3.3.1. Decision Support and Visualisation

The final stage is to present deep learning outputs to human operators. Automated alerts (e.g., collision warnings, no-AIS vessel detected) are generated when models detect critical events or anomalies. These are visualised on geo-referenced displays: e.g., live maps showing all tracked vessels with colour-coded statuses. Operators can click on a contact to view the latest camera feed, object classification (ship type), and behaviour history. Some systems overlay CNN detections on video (highlight vessel, show type, and confidence), or generate heatmaps of dense traffic. Importantly, explainability is considered: saliency maps or simple rule-checking (e.g., ”vessel exhibited 3-knot speed change in 10s") are used to justify alerts. For instance, a NATO naval system (VATOZ) integrates video analysis by displaying bounding boxes and text labels from deep learning models on a common operational picture. By integrating deep outputs into user-friendly dashboards and track management Graphical User Interface (GUIs) and MDA platforms ensure that situational insights are actionable by controllers and maritime law enforcement [70]. Empirical case studies show benefit: enhanced MDA trials (e.g., AIS+satellite+UAV) report that deep learning reduced ”contacts of interest" from hundreds of targets to a concise short list requiring human review. An experimental deployment of AI acoustic buoys achieved the detection of fast boats with 90% of precision, sending alerts to a command center [13]. In summary, deep learning enriches MDA architectures by automating perception and providing decision support, while interactive visualisation of CNN results helps operators monitor complex maritime domains.

4. Deep Learning for Specific Maritime Security Applications

Maritime literature presents several applications of deep learning techniques in numerous problem areas of maritime security challenges such as vessel detection, piracy prevention, illegal fishing, smuggling, and anomaly detection in ship behaviour. This section presents deep learning techniques applied to specific maritime security domains, highlighting their effectiveness.

4.1. Illegal Fishing Detection

In high seas and nationally controlled oceans, IUU fishing is a global pervasive problem causing the decline of ever increasing fish stocks and marine ecosystems disruption – undermining the ecological balance of the oceans [18]. The United Nations’ Food and Agriculture Organisation (FAO) estimates up to 26 million tons of IUU fishing worth USD 23 billion [71].

In the maritime literature, deep learning techniques have been applied to existing data on legal and illegal fishing activities to classify and identify vessels, detect suspicious behaviour, and predict potential violations of fishing regulations. Usually, spatiotemporal (fishing events over time and in different geographic locations) data with historical records of fishing vessel trajectories are analysed for fishing behaviour classification to detect patterns indicative of illegal activities. AIS and SAR imagery data are also widely used for the same purpose. The following are some of the exemplary research works that utilise deep learning techniques.

The study [19] proposed the so-called BiLSTM-CNN-Attention – a hybrid deep learning method of BiLSTM, CNN, and attention mechanism – to classify fishing vessels including the ones catching fish illegally using AIS data in China’s offshore waters from 2018 to 2022. When compared with other machine learning techniques, such as support vector machines, random forest, XGBoost, and BiGRU, the authors found the proposed method has enhanced generalisation ability. In terms of accuracy, the proposed method was 18% and 10% more accurate than the base models and deep learning methods respectively. BiLSTM-CNN-Attention is superior than others because it utilises BiLSTM for capturing both past and future context in the sequential data, CNN for extracting local patterns and hierarchical features from input sequences, and attention mechanism for focusing on key features to perform the classification task more effectively. However, the model failed to solve data bias problem and misclassified stow-net vessels and gillnetters as illegal fishing trawlers due to large trawler data.

Using SAR data, the authors in [4] attempted to classify fishing vessels with focus on small vessels with minor inter-class differences, using a deep learning technique which they called FishNet. It is a novel combination of four different modules: multipath feature extraction (MUL) – as DenseNet inspired module – for feature extraction from SAR image, feature fusion (FF) – a CNN-based module – for combining the extracted features from different paths to create a comprehensive representation, multilevel feature aggregation (MFA) – a top-down pathway – for aggregating features at multiple levels of the deep learning network, parallel channel and spatial attention (PCSA) – an attention mechanism – allowing the model to focus on the most relevant parts of the image, thereby improving classification performance. In comparison with 27 other deep learning models, FishNet achieved the highest classification performance in terms of accuracy, precision, recall, and F1-score. Overall, FishNet achieved the highest accuracy of 89.79%, which is 6.77% higher than the second-best method. On the other hand, the study faced challenges of small dataset with imbalanced class labels and longer training time of FishNet due to deeper and wider network structure compared to other simpler models leveraged in the experiments.

In the absence of an AIS system on some boats (“dark vessels"), combating illegal fishing through small fishing boats, especially at nighttime, is a challenge. The authors in [15] employed Stacked-YOLOv5 model – small-target detection layer – to effectively detect lit fishing boats using satellite imagery from Luojia1-01 (LJ-01) with over 96% accuracy. Here, the authors effectively combined single-band nighttime light images with stretching methods to form a triband image for improved feature extraction and detection performance. At the same time, the use of radiation correction and masking of offshore oil and gas platforms also enhanced the quality of the input data. However, the sample dataset for target detection is relatively small and the presence of lights from non-fishing vessels may introduce noise and affect detection accuracy. Additionally, the use of single-band images may pose difficulties in feature extraction.

4.2. Piracy and Armed Robbery Prevention

Piracy attacks and armed ship robberies are some of the most significant security threats to the shipping industry, leading to consequences for global economic activities. Using data on piracy attacks between 1994 and 2017, the study [72] suggests that small vessels, open registry vessels, vessels at berth or anchor, at night, in territorial waters and port areas are more vulnerable to piracy attached and robbery than others. Any proactive approach by the Master and the crew may improve the situational awareness, enhance surveillance, and detect threats early to determine the appropriate response in time-sensitive stressful conditions.

When it comes to leveraging deep learning techniques for the detection and prevention of piracy and robbery threats, this area poses a significant research gap. Due to the lack of data – be it AIS, SAR, or any other form – there is an immense demand for research efforts [73].

Researchers in [73] performed spatiotemporal pattern mining to propose a novel dataset that contains maritime piracy incidents by merging data from three sources: ASAM (Anti-Shipping Activity Messages), IMO GISIS (International Maritime Organisation - Global Integrated Shipping Information System), and IMB (International Maritime Bureau). A comprehensive framework for analysing spatiotemporal, time-series, and clustering data was developed. They visualised and analysed piracy incidents over the past three decades, developed the fast adaptive dynamic time warping (FADTW) method for uncovering hidden temporal and spatial-temporal patterns, and a density-based clustering technique DBSCAN (density-based spatial clustering of applications with noise) for extracting spatial distribution patterns and identifying high-risk areas. It is a robust dataset but still focuses on high-risk areas, potentially overlooking other regions. Additionally, the inherent complexity of spatiotemporal patterns may require further refinement and validation.

Focusing on the Straits of Malacca and Singapore waters, the authors in [74] analysed piracy and armed robbery reports, obtained from GISIS, IMO, and Annual Reports on Piracy and Armed Robbery from the Regional Cooperation Agreement on Combating Piracy and Armed Robbery against Ships in Asia (ReCAAP), using BERTopic – an advanced natural language processing (NLP)-based transformer model. BERTopic generated 13 main topics from the incident reports, which were clustered into three categories: ship’s risk, geographical constraints, and crew and authorities’ responses. The research identified several key factors influencing piracy and armed robbery, including ship type, geographical constraints, crew response, and authorities’ actions. Even though BERTopic is an effective technique, its application in maritime piracy research is relatively new, and further studies are needed to confirm its reliability.

4.3. Smuggling and Trafficking Detection

The volume of maritime transportation carrying the global economy today facilitates non-negligible issues of the illicit movement of goods and people, often across international borders, using sea routes – raising growing concerns of maritime security breach, human rights abuses, and the breach of international law.

The authors in [3] used YOLOv10s for detecting dark vessels using Sentinel-1 SAR, Sentinel-2 optical imagery, and spatiotemporal AIS data. The goal of the research was to optimise YOLOv10s for detecting small ships by removing unnecessary Conv and C2f layers from the network architecture. At the same time, they used AIS matchmaking techniques for cross-referencing image-based detections with AIS data. In the case of dark vessels, the approach helps distinguish suspicious activities by vessels that are visible in image but are not broadcasting AIS signals. When compared with state-of-the-art YOLO models across different types of satellite imagery, YOLOv10s achieved superior performance with accuracy ${\mathrm{AP}}_{50}$ = 0.8588, ${\mathrm{AP}}_{50-95}$ = 0.6631, precision=0.9370, recall = 0.9381, and specificity = 0.9869. Another major contribution of this research is their curated custom dataset HS3-S2, combining six open-source single-modality datasets, which helps improve detection capability.

The ship-to-ship exchange of illicit goods is another form of smuggling. To detect such activity, [75] utilised AIS data and PlanetScope satellite SAR imagery to train and test the deep learning model YOLOv8m. According to the authors, a ship-to-ship transfer between any two vessels identified in a satellite imagery can be referred to as smuggling when AIS signals from less than two vessels are available. With this approach, the research identified 400 of such events in the Kerch Strait between 2021 and 2023, with F1 accuracy of 97%.

The authors in [75] critically evaluated the use of AIS data for the suspicious movement of any ship, and proposed to fuse information from more than one source. The study developed an information fusion framework MSIF-SSTR which leverages high-speed radar trajectories and the corresponding meteorological data, specially at nighttime and poor weather conditions, as the potential of smuggling activities is high in such conditions. It is a decision-level fusion where the temporal features extracted by the TCN network are input into LSTM for determining if the ship trajectory suggests to be suspicious. Proposed model achieved a classification accuracy of over 94% on the collected dataset.

To address the problem of dealing with small inflatable smuggling boats, [2] employed advanced deep learning models such as YOLOv2, YOLOv3, and Faster R-CNN, alongside feature extraction models such as GoogLeNet, ResNet18, ResNet50, and ResNet101. The authors collected IR thermal images of small inflatable boats and people on the Elblag and Bug rivers in Poland in different weather conditions. The experiment suggested that, amongst other deep learning algorithms, Faster R-CNN with ResNet101 achieved the highest detection rate but required significant processing time. On the limitations, the authors highlighted challenges such as detection capability reducing in varying environmental conditions and the limitations of data sources catering variety of scenarios.

4.4. Maritime Environmental Monitoring

Scientists have widely used AI and deep learning techniques to detect marine pollution, monitor marine life, and improve our understanding of marine ecosystems, using data from sensors, satellites, and other sources.

Oil spill incidents have devastating effects on the marine ecosystem. Accurate and timely detection of oil spills helps protect and minimise harmful effects on the aquatic biome. There exists extensive literature that explores the deep learning application of oil spills, using computer vision and image data. For instance, [76] employed a variety of deep learning algorithms including UNet, BiSeNetV2, and DeepLabV3+, alongside attention mechanisms such as Squeeze-and-Excitation (SE), Convolutional Block Attention Module (CBAM), and Simple Attention Module (SimAM) on optical images from Sentinel-2 MSI, Landsat-8 OLI, and Landsat-9 OLI2. The experiments suggested that UNet with CBAM was more accurate than others in detecting oil slicks. While the model performed well on these common slicks, it struggled to fully capture thick crude and emulsified oil slicks, which are underrepresented in the given dataset limited to 143 optical images only.

The study [6] used CNN, Multilayer Perceptron (MLP), and U-Net models for oil spill classification and segmentation using Sentinel-1 SAR imagery. They trained and tested these models with 685 Sentinel-1 SAR images in dual-polarisation (VV, VH) mode. The results suggested that a CNN with six convolutional layers, 32 filters, and two hidden layers achieved 99% classification accuracy. U-Net with Focal Loss and Intersection over Union (IoU) metric achieved 96% IoU for training and 90% for validation. Overall, the two-stage CNN and U-Net framework achieved an overall accuracy of 95% and an IoU of 90%. On research limitations, the authors inferred that the look-alikes in SAR images can cause false positives, thus increase in data variability may improve detection accuracy.

The authors in [21] developed a comprehensive pipeline for detecting and classifying oil spills using aerial imagery and deep learning. Collecting RGB image data from various sources, including Kaggle, GitHub, and the Korean Coast Guard, the authors trained and tested deep learning model with ResNet101V1c backbone and DaNet segmentor for image segmentation and classification. The model achieved a mean Intersection over Union (mIoU) of 72.49% and an accuracy of 94.22%. Furthermore, the research used a conditional Generative Adversarial Network (GAN) model to generate synthetic images and annotations and achieved improved model accuracy by 2.56% and balanced the contribution proportions of different oil types. However, the research indicated a challenge in detecting silver and brown oils due to their resemblance to natural elements and imbalances in the training dataset.

Marine ecosystems prevention conservation is another critical research area, and deep learning techniques have effectively served the purpose in the existing literature. The authors in [77] used deep learning models such as ResNet, DenseNet, Inception, and Inception-ResNet for automated classification of deep-sea biota using remotely operated vehicle (ROV) imagery. According to findings of this research, Inception-ResNet achieved a mean classification accuracy of 65% with AUC scores exceeding 0.8 for each class.

4.5. Safety, Search and Rescue Operations

Safety, search and rescue operations in the challenging maritime environments require prompt and accurate decision-making. In this regard, advanced deep learning techniques have been trained and tested by complex data sources such as satellite imagery, aerial surveillance, sonar, and vessel tracking systems.

The authors in [16] developed a sea surface object detection system, so-called Sw-YoloX, which combines transformer and anchor-free mechanisms to improve detection accuracy. It integrates convolutional block attention module (CBAM) and atrous spatial pyramid pooling (ASPP) to address two critical issues: slow convergence rate with high training overhead of transformer architecture and inefficient feature extraction of CNN. With this approach, the authors were able to reduce computational cost and increase detection accuracy. Additionally, they used data augmentation, multi-scale training, and self-training classifier to enhance performance. The model achieved F1 accuracy of 78%, mAP 54, and recall 72 when detecting objects in complex and undulating sea surface environments.

The study [78] attempted to detect a person in open water during search and rescue operations using UAV images and YOLOv4 with MobileNetV3Small backbone. With the image frames from videos captured by UAVs over lakes and seas in Turku, Finland, the authors trained and tested the deep learning model. The experimental analyses suggest that lowering IoU threshold (e.g., 0.10) may result in better detection metrics, with more true positives (TPs) and fewer false negatives (FNs). Additionally, bounding-box size and multiple consecutive frames have impact on detection accuracy, in terms of improved precision and recall, and computational cost.

Another deep learning-based framework for efficient search and rescue operations has been proposed in [79]. Here, the authors first pre-processed UAV images to identify potential victim locations using simple features and then used a pre-trained CNN to verify the presence of victims in the identified region. The two-step process improved the efficiency of detecting person in water and achieved 95% accuracy, with a true positive rate (P_TP) of 0.9778 and a false alarm rate (P_FA) of 0.0769.

Table 3 provides a comparative overview of state-of-the-art deep learning models employed on specific maritime security applications. It summarises the model architecture, intended application domain, key strengths, known limitations, and the reported performance metrics.

5. Key Sources for Deep Learning in Maritime Domain

In this section, some of the key datasets suitable for deep learning research in the maritime domain are presented. These datasets can be obtained directly from a publicly accessible online venue or by requesting the authors, as stated in their research publications. The datasets cover research problems such as vessel detection, classification, behaviour analysis, oil spill detection, security threat detection, and SAR imagery.

When using these datasets, in the context of deep learning for maritime security, key considerations should be taken into account, including annotation quality, diversity of vessel types, and environmental variability. It is observed that that many earlier datasets are limited in scale or completeness; for example, comprising as few as 14 videos [80]. More recent datasets such as Deepdive [77], SeaDronesSee [81], and SAR-HumanDetection-FinlandProper [78] attempt to address these limitations by offering broader coverage. However, significant gaps persist: few large-scale datasets combine EO and IR modalities, and acoustic data remains particularly scarce. While AIS data is widely available (e.g., through platforms like AISHub [82]), there is a lack of publicly labeled datasets for anomaly detection or vessel behaviour classification.

Table 4 presents computer vision datasets and Table 5 lists the datasets according their specific application in maritime domain.

6. Challenges and Future Directions

The domain of maritime security maintains several challenges that pose potential obstacles in the meaningful implementation and deployment of deep learning techniques in real-world use cases. It has been observed, from the existing literature, that the issue of data robustness and availability is the foremost challenge when it comes to training and testing deep learning techniques. Contrarily, in various other domains, such as healthcare, finance, environment, and government data, easy availability of open access public datasets facilitates the experimentation and expansion of research in variety of problem areas. On the other hand, because of specificity and selectivity of the maritime field, the access to maritime affairs and data is limited – especially maritime security where data confidentiality and security matters – curbing large-scale deep learning research. To address the data scarcity problem, researchers have attempted to generate synthetic data to expand existing datasets. To this end, data augmentation techniques, simulation environments to generate artificial data based on selected maritime security scenarios, and generative adversarial networks (GANs) have been explored; however, ensuring realism and diversity for real-life or practical deep learning applications is crucial.

Apart from data availability challenges, there are issues with the quality of available datasets, such as, scarcity of labeled data, class imbalance (e.g., unequal representation of security scenarios), and bias (e.g., regional bias and common scenarios vastly outnumbering critical security events). This leads to an adverse impact on model generalisability and biased deep learning models that struggle to detect anomalies or rare threats effectively. Therefore, model performance degradation and lack of robustness in unseen conditions is inevitable because maritime environments are inherently dynamic. There are several natural and man-made variables; for example, variations in weather and sea state, vessel appearance and forged status, and crafted perturbations in input data. The deep learning models trained in specific contexts often fail to generalise across new or adverse conditions, leading to reduced accuracy in the field.

Additionally, because of the criticality of the maritime security matters where timely decision is paramount to planning and executing response, the real-time application of deep learning models demands onboard deployed on maritime platforms such as ships, drones, or autonomous vehicles (aerial, surface, and underwater). The advanced deep learning architectures require substantial computational resources, forming a key bottleneck for real-time onboard processing. It is observed in existing literature that several deep learning solutions proposed for maritime security are observed to be complex (often combining multiple deep learning architectures) and computationally heavy. The limited onboard computing capability, processing power, and bandwidth-constrained environments require optimised models that operate under limited resources. To this end, researchers have proposed approaches such as model pruning, optimisation, and lightweight architectures (e.g., MobileNet, EfficientNet)that reduce the computational footprint of deep learning models without significantly compromising performance.

Another critical challenge is algorithmic bias. From the existing literature, it is obvious that deep learning models are trained on biased datasets, leading to an AI model that may not be fully trustworthy when taken into account in the decision-making process by the maritime security agencies. A biased algorithm may unfairly target specific vessel types, regions, or scenarios, leading to discriminatory outcomes that can introduce systemic inequities in maritime law enforcement and significant operational and legal consequences. Here comes the role of explainable AI (XAI) which help stakeholders understand and validate the model outputs, fostering greater acceptance and accountability. Additionally, the techniques, such as saliency maps, class activation mappings (CAM), layer-wise relevance propagation (LRP), and SHAP, help visualise and interpret the deep learning results by identifying which features or regions of an input contributed most to a prediction.

It is obvious from the existing maritime literature that the advanced deep learning techniques have proven their potential. The latest deep learning architectures have been widely leveraged in maritime security research, including, deep learning object detection/classification techniques in YOLO group; feature learning techniques in ResNet, EfficientNet, DenseNet, Vision Transformer (ViT) groups; and natural language processing techniques in BERT, XLNet, FLAN-T5 groups; and various techniques for multimodal data fusion. For future research, the avenues for combining these architectures are yet to be explored for robust and effective deep learning-based maritime security applications. Another potential research area would be integrating knowledge graphs or symbolic reasoning for enhancing situational awareness and contextual understanding in maritime security scenarios. Lastly, because of the criticality of outcomes of decisions taken by the security operators, it is important to develop XAI models that can effectively interpretable by human users. Based on the review of existing literature performed in this research, this particular area highlights a clear gap in the literature, making it a promising avenue for future research.

7. Discussion and Conclusion

The smooth functioning of international affairs through sea ensures global stability and economic sustainability. It heavily depends on secure and peaceful maritime domains; therefore, safeguarding maritime assets and borders is a critical mission. Especially, in the strategic maritime regions like the Mediterranean Sea, Gulf of Aden and the English Channel that are frequent focus of illegal migration, trafficking, and smuggling.

In the presence of growing concerns about complexity and the scale of maritime security threats, that range from illegal fishing and smuggling to environmental damage and human trafficking, the demand for more intelligent and automated surveillance solutions exists more than ever. Traditional methods heavily rely on manual monitoring and physical patrolling. These are insufficient against the volume, stealth, and sophistication of modern maritime threats. In this context, the AI application, especially deep learning techniques, has emerged as a transformative approach for enhancing maritime domain awareness (MDA) and operational response capabilities. Deep learning provides capabilities vital for enabling proactive security interventions and efficient resource allocation.

In this review study, we examined how recent advances in deep learning have contributed to object detection and tracking, situational awareness, anomaly detection, and activity recognition in maritime environments. Across these security challenges, deep learning models have demonstrated notable performance due to their ability to process and learn from vast, heterogeneous datasets that include optical and radar imagery, AIS signals, SAR data, and real-time sensor inputs from UxVs and satellites.

Deep learning models such as CNNs (YOLO, SSD, RCNN), sequence models (LSTM, GRU, Transformers), and hybrid architectures (CNN+RNN, CNN+Attention, VAE+Transformer) have shown significant capability in detection accuracy and temporal modelling of maritime threats. Particularly in vessel detection and classification, YOLO-based models have shown high reliability in identifying multiple vessel types with precision under various environmental conditions.

It is also observed in the literature that combining data from various sources enhances comprehensive situational awareness in maritime environments. In this case, multimodal data fusion approaches have successfully integrated EO/IR imagery, radar, AIS, and acoustic data to enhance robustness and accuracy. These deep fusion architectures (e.g., multi-stream CNNs, BiLSTM-CNN-Attention, and sensor-level fusion) offer a unified situational picture that compensates for the limitations of individual data streams. For example, SAR proves effective during adverse weather and nighttime, while optical sensors provide rich visual detail during clear conditions. Radar and AIS data further complement these sources, especially when used in combination with CNNs or sequence models to detect trajectory anomalies and behavioural patterns.

Despite these advancements, the maritime literature has also highlighted persisting challenges. For example, the detection of small or partially occluded objects, especially in cluttered or reflective maritime backgrounds, remains difficult and requires significant research effort. Similarly, distinguishing between legitimate vessel behaviour and malicious activity also poses challenge for anomaly detection algorithms, especially in congested waterways. Moreover, in the practical scenarios where nature plays its role in sea conditions, deep learning models suffer from depreciated performance.

Explainability and real-time deployment also demand more research. While saliency maps, rule-based logic overlays, and interpretive dashboards provide some transparency, fully interpretable deep learning systems for maritime security are yet to be researched. Lightweight models (e.g., Tiny YOLO) and onboard processing strategies are increasingly being employed to achieve high frame-rate performance, especially in UxV and edge-computing scenarios.