An AI-Assisted Multispectral Thermal Monitoring System for Maritime Safety: A Case Study at the Port of Nazaré

1. Introduction

Coastal fishing ports and maritime infrastructures operate as vital economic nodes, yet they present severe, daily operational challenges to maritime safety. The entry and exit of vessels through narrow harbour bar entrances represent the highest-risk phase of coastal navigation, particularly during adverse weather conditions such as dense fog, heavy rain, or complete darkness. A paradigmatic example of this environment is the historic Port of Nazaré, Portugal—managed by the public entity Docapesca—where both commercial and traditional fishing fleets must routinely negotiate one of the most difficult harbour bar entrances on the Iberian coastline. Beyond navigational hazards, the unauthorised entry of individuals into restricted port zones poses a continuous threat to both operational security and the physical safety of the public. Mitigating these dual risks requires continuous, reliable, and automated wide-area surveillance.

Historically, port surveillance has relied on human operators monitoring conventional visible-spectrum (RGB) camera feeds [1]. However, optical sensors are fundamentally constrained by their reliance on reflected visible light; their detection efficacy degrades precipitously in low-light environments and drops to near zero during severe maritime weather events [2,3]. To overcome this limitation, maritime research has increasingly turned to Long-Wave Infrared (LWIR) multispectral thermal imaging [4]. Because thermal cameras capture the infrared radiation emitted by any object with a temperature above absolute zero, they visualise the distinct thermal signatures of vessel engines, human bodies, and marine wildlife entirely independent of ambient lighting, maintaining high performance even in total darkness [5,6].

While raw thermal video solves the atmospheric visibility problem, the manual, 24-hour monitoring of continuous thermal streams inevitably induces operator fatigue, leading to high rates of missed incidents [7]. Consequently, the integration of Auto-mated Target Recognition (ATR) via Artificial Intelligence has become a primary focus of marine engineering [8,9]. Within the current literature, two diverging methodological hypotheses have emerged: purely heuristic, rule-based computer vision algorithms (such as thermal thresholding and background subtraction) [10], and modern deep learning models utilising Convolutional Neural Networks (CNNs), predominantly single-stage object detectors [11,12,13]. Pure deep learning models often struggle with the highly chaotic, dynamic nature of the maritime background (such as thermal re-flections on breaking waves), whereas purely heuristic models lack the semantic capability to reliably separate overlapping classes [14].

Addressing this tension, this work introduces an operational hybrid AI architecture developed within the scope of the European BRIGHTER project (funded by the EU Chips JU). Developed through a close collaboration between Docapesca and the Institute for Systems and Computer Engineering (INOV), a pioneering multispectral thermal monitoring system was deployed at the Port of Nazaré. The system leverages hybrid AI—combining conventional image processing, heuristic rules, and YOLOv7-based deep CNNs optimised for thermal imagery—to automatically detect, classify, and track vessels, people, and birds continuously 24 hours a day.

The main aim of this Communication is to detail the physical deployment, the resilient hybrid network architecture (integrating parallel fiber-optic links and long-range Wi-Fi), and the centralised GPU processing pipeline that drives this “thermal eye that never sleeps”. Over two years testing campaign spanning diverse seasonal conditions, the system generated a domain-specific dataset comprising over 48,000 annotated thermal images. The principal conclusions demonstrate that this hybrid approach significantly reduces false negatives in critical navigational phases while keeping false positives at a manageable level for operators. By automatically generating low-latency JSON metadata for tracking and geofenced alarm activation, this project moves beyond a pilot study to establish a functionally mature, scalable blueprint for the digital transformation of maritime port safety.

When integrated with the advanced Artificial Intelligence algorithms developed by INOV, the core technical contributions of this operational deployment are fourfold:

1. Optimised Thermal Inference: Real-time target processing executed via deep Convolutional Neural Networks (CNNs) utilising YOLOv7 architectures fine-tuned specifically for maritime thermal imagery [10];

2. Heterogeneous Classification: Simultaneous detection and semantic classification of overlapping maritime object classes, specifically distinguishing between commercial fishing vessels, recreational boats, individuals, and marine wildlife;

3. Persistent Kinematic Tracking: Continuous, multi-target trajectory association across consecutive thermal frames using the Hungarian algorithm paired with state estimation filters [5,6];

4. Automated Geofencing Alarms: Low-latency generation of structured JSON metadata containing object coordinates, confidence scores, and directional vectors to trigger intelligent early warnings upon restricted area breaches.

2. System Architecture

The BRIGHTER system was designed to provide complete, gap-free coverage of the entrance bar of the Port of Nazaré. The architecture consists of two remote camera sites (Site One and Site Two) connected to a central Communication Centre.

• Site One (S1): Located at the harbour edge on a road junction, equipped with two high-resolution Xenics thermal cameras (1280 × 1024 pixels) with a 25° horizontal field of view.
• Site Two (S2): Positioned higher up on the marina pier, equipped with a compact SENSIA thermal camera (384 × 288 pixels) with a 30° horizontal field of view.

From these two locations, three overlapping fields of view (F1, F2, and F3) ensure comprehensive surveillance of the entire port area, including the entrance channel, beach, and marina.

Located next to Site Two, the Communication Centre (CC) serves as the core of the system. It houses a high-performance tower server equipped with an NVIDIA Ge-Force RTX 4090 graphics card, responsible for intensive AI processing, data fusion, and stream distribution. The CC also includes fiber-to-Ethernet media converters, a vertical outdoor/indoor Wi-Fi access point/bridge, and a direct Ethernet connection to the Internet.

Figure 1. An Aerial view illustrates the coverage zones of the system.

Figure 1. An Aerial view illustrates the coverage zones of the system.

Hybrid Connectivity

The communication between sites was designed to maximise reliability, band-width, and redundancy, it utilises specialised contributions from industry partners Xenics and Sensia:

• Site One: Connected to the CC via two parallel fiber-optic links, ensuring high bandwidth.
• Site Two: Connected via a long-range Wi-Fi link using vertical out-door/indoor access point/bridges.
• At Site Two, a compact fanless industrial mini-PC processes the SENSIA camera signal locally via USB 3.0.

This hybrid architecture (fiber optics + long-range Wi-Fi) balances cost, distance, and reliability, ensuring stable transmission even in a demanding maritime environ-ment. The complete system delivers continuous, robust, and high-bandwidth monitor-ing by combining edge computing at remote sites with powerful central AI processing at the Communication Centre.

Deployed System

Located at the harbour edge, Site One (S1) comprises a secure, robust outdoor monitoring installation. A galvanised steel mast was erected on a reinforced concrete base and protected by a metallic fence. At the top of the mast, two Xenics thermal cameras were installed in complementary orientations, providing wide and overlap-ping coverage of the port area.

The installation included a rigorous process of camera alignment and calibration to maximise surveillance coverage and ensure accurate thermal detection. Additional procedures involved connectivity verification, environmental sealing inspection, camera synchronisation, and functional testing under operational conditions.

The deployment required extensive preparatory civil, electrical, and telecommunications works: site survey and terrain assessment, excavation, construction of a wind-resistant concrete foundation, grounding systems, power distribution, surge protection, and fiber-optic cabling. An outdoor weatherproof cabinet houses the electrical system, protection devices, and fiber-optic equipment.

Figure 2. Site One—Two Xenics cameras with communication and power support.

Figure 2. Site One—Two Xenics cameras with communication and power support.

At the marina pier, Site Two (S2) utilised an existing metal pole, minimising civil works. A compact SENSIA thermal camera was installed at the top, with a vertical Wi-Fi access point/bridge mounted directly above it to provide long-range wireless connectivity to the Communication Centre.

Lower on the pole, a weatherproof enclosure houses the networking equipment, power systems, and a fanless industrial mini-PC. All cabling was properly routed and secured to withstand the aggressive marine environment.

Figure 3. Site Two—SENSIA camera and control system on the marina pier.

Figure 3. Site Two—SENSIA camera and control system on the marina pier.

The heart of the system is the Communication Centre. Inside, a high-performance server equipped with an NVIDIA GeForce RTX 4090 GPU executes all advanced AI processing. This server runs the applications that fuse data from the three cameras, perform real-time detection, and distribute video streams.

The workstation is completed with a monitor displaying the processed video feeds, along with a keyboard and mouse for local configuration. The system receives video streams from Site One (via fiber optics) and Site Two (via Wi-Fi), processes them with AI algorithms, and makes the information available in real time to Docapesca operators and INOV researchers via the Internet.

In addition to generating annotated images (with bounding boxes), the AI services produce metadata in JSON format containing positions and detected classes. This enables automatic tracking and the triggering of alarms when restricted zones are breached. A remote user terminal allows Docapesca operators to view annotated streams and receive timely alerts.

Figure 4. Interior of the Communication Centre.

Figure 4. Interior of the Communication Centre.

Exploitation

To operationalise the developed system, INOV developed a customised Proof-of-Concept (PoC) Control Centre interface tailored specifically for Docapesca’s port management workflows. This centralised platform enables the automated detection of spatial breach events, such as vessels crossing critical harbour bar entrance thresholds or unauthorised recreational craft entering restricted maritime zones. Furthermore, the system functions as a forensic logging engine; operators can execute historical event playback and query trajectory data via granular filters customised by temporal window (date/time) or spatial geofence.

Figure 5. Main Control Center window.

Figure 5. Main Control Center window.

Figure 6. Alarm list menu.

Figure 6. Alarm list menu.

Figure 7. Play back events menu.

Figure 7. Play back events menu.

After several months of continuous operation under real and varied conditions — from bright sunny days to dense fog, heavy rain, and complete darkness — the BRIGHTER system demonstrated high robustness, stability, and operational reliability.

3. Automatic Detection

This section describes our efforts to create an automatic detection system mainly to recognise fishing and recreational vessels, but also to identify people and birds. People identification may later be used to ensure that people do not enter dangerous or restricted zones, and birds are relevant to avoid false positives. A large dataset was created, comprising more than 48,000 annotated thermal images collected over two years, across different seasons. The images were collected over 65 days in August and September 2024 and February and March 2025. These datasets enabled the training and continuous refinement of YOLOv7-based AI algorithms specifically optimised for thermal imagery. Table 1 shows how the images are distributed per camera: Camera 1 accounts for 50.3% of the total (24289 images), Camera 2 for 37.9% (18291 images), and Camera 3 for 11.8% (5715 images). Table 2 presents the distribution of images per class. The Recreational Boat class is the most represented, with 13615 objects in total (7946 in training), followed by Fishing Boat with 8917 objects (5933 in training) — together, the two boat classes total 22532 objects. The Person class has 5600 objects, and the Bird class is the least represented, with 940 objects in total.

Table 3 presents the confusion matrices for the best-performing model, evaluated on the test set. The normalisation is column-wise, with each column summing to 100%. The results shown are for the decision threshold that provides the largest F1 score across all classes. The best model was trained with a frozen backbone, an SGD optimiser, an initial learning rate of 0.001, and a batch size of 16 for 100 epochs with Optimal Transport Assignment (OTA).

The Fishing Boat class achieves near-perfect recall on the test set at 98.6% (1459 out of 1480), with background false negatives negligible at 0.5% and background false positives at 11.8% (54 out of 457). This is the strongest performing class. The Recreational Boat class also performs very well. Recall holds at 92.2% (2617 out of 2838) and background false negatives at 5.7%. The background false positive rate is high at 40.0% (183 out of 457). A key question for the present work is whether the model can reliably distinguish between fishBoat and recBoat. The results are encouraging: cross-confusion is low in both directions. Only 2.1% of true recBoat instances are predicted as fishBoat, and only 0.9% of true fishBoat instances are predicted as recBoat, confirming that the model has learned a meaningful distinction between the two vessel types. For Person detection, the recall is 58.4% (616 out of 1,054), and the background false-negative rate is 41.4%, while the background false-positive rate is rather high at 38.5% (176 out of 457). Overall, person detection remains the weakest class. The Bird class shows a recall of 72.0% (126 out of 175) on the test set, and a background false-negative rate of 27.4%, while the back-ground false-positive rate is 9.6% (44 out of 457). Figure 8 and Figure 9 illustrate real examples of automatic detections performed by the system.

The results are very encouraging: high precision in vessel detection (fishing and recreational), good differentiation between classes (people, birds, and vessels), and a reduced rate of false positives, even in complex scenarios. The system showed excellent performance in real-time JSON metadata generation, which is essential for automatic tracking and intelligent alarm activation in restricted areas. These results not only validate the technical maturity of the system but also show that the rich datasets can be used to improve future versions of the automatic detection.

4. Contribution to the Future

The BRIGHTER project represents a significant step forward in the modernisation of port surveillance in Portugal. By integrating state-of-the-art multispectral thermal cameras with hybrid Artificial Intelligence algorithms (computer vision + heuristic rules), the system delivers 24/7 monitoring capabilities that were previously extremely difficult to achieve in demanding maritime environments.

It is important to say that while the BRIGHTER hardware demonstrated total physical and operational resilience during instances of heavy rain and dense fog throughout the deployment time, the current dataset predominantly evaluates model performance across the general maritime timeline. In atmospheric physics, it is well established that while Long-Wave Infrared (LWIR) sensors maintain vastly superior transmission through scattering media compared to visible-spectrum cameras [4,5], extreme precipitation inevitably induces ‘thermal washout’—a meteorological phenomenon that homogenises the thermal contrast between a vessel’s hull or motor and the ambient sea surface [14]. Because the present collection campaign did not systematically isolate these severe weather subsets for independent algorithmic benchmarking, determining the exact degradation curve of the YOLOv7 [10] confidence threshold under extreme atmospheric attenuation represents a recognised limitation of this communication. Quantifying model precision within these specific, zero-visibility meteorological envelopes constitutes the immediate next phase of this research.

For Docapesca, BRIGHTER constitutes a strategic operational tool: capable of enhancing bar safety, improving maritime traffic control, detecting intrusions in restricted areas, and providing objective records for incident analysis.

For INOV, the installation in Nazaré enabled real-world validation of laboratory developed technologies, the creation of unique maritime thermal image datasets, and the refinement of detection and tracking algorithms adapted to the Portuguese context.

This project paves the way for replicating the solution in other national and international ports, advancing maritime safety, operational efficiency, and the digital transformation of port infrastructure.

5. Conclusions

The successful implementation of the BRIGHTER system at Nazaré Harbour is the result of a high-level technical synthesis between Docapesca, INOV, Sensia, and Xenics. By integrating multispectral LWIR hardware with advanced CNN optimisation and a resilient hybrid network, the project has established a scalable blueprint for the digital transformation of maritime safety.

By combining robust thermal sensors, hybrid communication architecture, and advanced AI algorithms, the project has created a mature, reliable platform adapted to the real conditions of the Portuguese coastline. More than just a pilot experience, the system is now a functional prototype that strengthens the safety of lives, assets, and infrastructure in one of the country’s most challenging maritime areas.

BRIGHTER is not merely a successful European project — it is a concrete step to-ward smarter, safer, and more resilient ports, prepared to face the present and future challenges of national maritime activity.