Secure Hardware Assurance Using Visual AI and Deep Learning on AOI Imaging of Electronic Assemblies

1. Introduction

The reliability and security of modern electronic systems are increasingly challenged by component-level anomalies, including counterfeit parts, unauthorized modifications, and assembly inconsistencies[1,2,3]. These issues are particularly critical in domains such as aerospace, medical, energy infrastructure, and defense electronics, where undetected deviations can have severe functional or national security implications[4]. Although automated optical inspection (AOI) systems are standard in electronics manufacturing, their capabilities are largely limited to rule-based checks for component presence or orientation, leaving more subtle, security-relevant issues undetected [5,6,7,8,9].

Recent advances in deep learning and pattern recognition offer new opportunities to enhance visual inspection using AOI-generated images[10,11,12]. These high-resolution images, which are already captured during manufacturing, can serve as a valuable source of structured data for detecting anomalies based on shape, marking consistency, placement geometry, and surface features. However, traditional AOI systems do not leverage this potential due to their reliance on static rules and limited image interpretation.

The need for secure hardware assurance has become increasingly urgent in light of recent revelations concerning supply chain vulnerabilities. A May 2025 investigative report by Reuters revealed that rogue communication devices, undocumented wireless modules, were discovered embedded in Chinese-manufactured solar inverters deployed across U.S. infrastructure [13]. These components, not listed in the product documentation, enabled covert remote access and bypassed existing cybersecurity protections. This incident underscores the critical gap between software-level defenses and the physical verification of hardware, especially when undocumented modifications remain invisible to conventional traceability and inspection tools. A related and widely publicized example is the “Big Hack,” in which printed circuit board assemblies (PCBAs) were allegedly implanted with rogue components to facilitate cyber-espionage against infrastructure and government-controlled systems [3]. Together, these incidents illustrate the need for systematic, scalable, and intelligent methods to verify the physical integrity of hardware—beyond what rule-based inspection or supply chain documentation alone can provide.

This paper presents a deep learning–based visual inspection framework that applies object detection and segmentation models to AOI image data. The system learns expected visual and spatial patterns from reference board assemblies and identifies deviations indicative of tampering, component substitution, rework, or contamination. We evaluate the performance of this framework across production-grade and degraded board conditions and demonstrate its ability to detect security-relevant anomalies with high precision and low false-positive rates. By enabling scalable, image-based verification of electronic assemblies, this approach contributes a critical layer to secure hardware assurance frameworks.

1.1. Limitations of Existing Inspection Practices

Despite the growing importance of hardware assurance, current industry practices rely heavily on indirect or partial verification methods. Traceability protocols such as IPC-1782 or NIST's cybersecurity frameworks focus on documenting supply chain events but do not validate the actual components assembled on the board[14,15,16,17]. In many cases, visual or electrical inspection is performed on a sample basis only, [12,18]leaving the majority of components unchecked and vulnerable to undetected substitution, mislabelling, or tampering.

Physical inspection methods such as X-ray imaging and electrical testing are both expensive and impractical for real-time inspection in high-volume environments [19]. Moreover, they often fail to detect surface-level visual anomalies that may indicate subtle forms of rework or component fraud. Trusted supplier agreements reduce but do not eliminate risk, particularly when rework or substitution occurs after initial sourcing, during manufacturing, or in logistics.

Given these constraints, there is a pressing need for a full-coverage, image-based verification strategy that can operate automatically, scalably, and with minimal production disruption. The approach proposed in this work transforms underutilized AOI imaging data into a foundation for deep learning–based inspection, enabling the identification of visual anomalies at the component level with precision previously unattainable in standard production environments.

2. Method

The proposed inspection framework is designed to analyze high-resolution images generated by Automated Optical Inspection (AOI) systems, enabling component-level verification and anomaly detection through deep learning. It integrates two core capabilities: object detection for identifying and localizing all components on a board, and semantic segmentation[20] for detailed analysis of component geometry and solder interfaces. Together, these models enable visual fingerprinting and structural validation, allowing the system to detect deviations from reference board designs caused by tampering, substitution, or untracked rework[10,12,18,21].

2.1. Limitations of Existing Inspection Practices

The system operates on image datasets acquired from inline AOI systems or high-resolution post-production scanners. The primary dataset consists of clean, high-contrast images from boards captured in production environments under controlled lighting and camera conditions. These images serve as the training baseline for learning spatial layouts and component shapes. To ensure robustness, additional datasets were created from scrap boards collected at recycling centers. These boards were imaged both in cleaned (lightly dusted and washed) and uncleaned (oxidized, damaged) states, capturing realistic visual variability such as corrosion, surface wear, and foreign material presence.

This multi-domain dataset supports model generalization across pristine, degraded, and field-recovered board conditions, critical for real-world deployment where production variability and contamination are common.

2.2. Object Detection [22]

We use the YOLOv12 architecture for component-level object detection [23]. The model is trained to localize all visible components on a board and classify them by package type. Initial training is conducted on images from pristine AOI datasets, using CAD-aligned bounding boxes for supervision. To improve generalization, a bootstrapping procedure is used: the model is applied to images from the degraded datasets, predictions are manually corrected, and the network is retrained iteratively.

This approach allows the detection model to maintain high accuracy even in low-contrast, noisy, or partially damaged conditions. Detection outputs include bounding box coordinates, confidence scores, and component class labels, which are used downstream for structural consistency checks and anomaly filtering. An example of the model's output on a production-grade board is shown in Figure 1.

2.3. Component Segmentation

In parallel to detection, we apply a semantic segmentation model trained to isolate each component’s body and solder pad regions. This segmentation enables pixel-level analysis of shape, orientation, and solder coverage, features critical for identifying unauthorized modifications or quality issues such as rework or improper mounting.

Segmentation training uses high-resolution AOI images annotated with masks for each component and pad. As with detection, a bootstrapping workflow is employed to extend training across degraded boards. The model achieves high precision and recall on both clean and noisy datasets, supporting robust feature extraction for downstream comparison. Figure 2 illustrates the output of the segmentation model, showing distinct masks for the component body and pad regions.

Segmented component shapes are compared to a reference models, enabling bottom-up visual fingerprinting and detection of anomalies such as footprint mismatch, lead deformation, missing pads, or suspicious markings.

2.4. Anomaly Detection Pipeline

The final stage of the framework integrates detection and segmentation results to identify structural or visual anomalies. Detected components are first validated against expected layouts using geometric and spatial analysis. A learned model of layout tolerance, captured from 20 to 100 reference boards, accounts for typical manufacturing variation while highlighting deviations such as missing, added, or swapped components.

Each detected anomaly is then classified by a second-stage AI module that evaluates severity and type (e.g., missing, extra, piggybacked, rotated, or contaminated). Thresholds can be configured by the user to tune sensitivity for specific use cases or board classes.

This layered architecture enables the system to distinguish between benign visual noise and security-relevant anomalies, delivering precise and scalable verification suitable for both production QA and post-deployment forensics.

3. Experimental Results and Case Studies

To validate the proposed AOI image–based inspection framework, we conducted a series of experiments on diverse datasets representing both production-grade and degraded board conditions. The evaluation focuses on detection accuracy, segmentation quality, anomaly classification, and overall robustness across different visual environments. Additionally, we include case studies that demonstrate the framework’s ability to detect genuine tampering, component substitution, and undocumented rework events in real-world PCB assemblies.

3.1. Evaluation of Detection and Segmentation Models

The object detection model, based on YOLOv12, was trained using three datasets:

• Clean AOI images from production environments aligned with CAD data,
• Cleaned scrap boards scanned under controlled conditions, and
• Uncleaned scrap boards representing worst-case scenarios including oxidation, broken components, and noise.

Training followed a staged bootstrapping strategy, progressing from pristine to degraded datasets. The model’s convergence behavior over 360 epochs is illustrated in Figure 3, which presents combined and individual loss curves, as well as validation dynamics and inter-loss correlations.

Performance on unseen test boards is summarized in Table I. The model achieved an F1 score of 0.96 on pristine boards, 0.92 on cleaned scrap boards, and 0.82 on degraded scrap boards, with an overall average of 0.90. Precision and recall remained consistently high across all conditions, confirming the model’s robustness.

The segmentation model was evaluated on 2,371 component instances from 241 images. Using separate masks for body and pad regions, the model achieved a mean average precision (mAP50) of 96.9%, with class-specific precision exceeding 0.91 for bodies and 0.97 for pads, as reported in Table II.

The segmentation model’s performance across varying confidence thresholds is shown in Figure 4. It maintains high reliability and consistent behavior, making it suitable for fine-grained structural verification.

These detection and segmentation results form the core of the anomaly detection pipeline. Their ability to generalize across board quality and environmental conditions ensures effective performance in real-world manufacturing and field-inspection scenarios.

3.2. Real-World Detection Case Studies

The framework was deployed on real-world production datasets to evaluate its effectiveness in detecting unauthorized modifications and undocumented deviations from design specifications. These case studies highlight scenarios where conventional traceability and rule-based inspection would fail to detect critical anomalies.

In one case, the system flagged a microcontroller due to inconsistent markings compared to the approved bill of materials. Visual comparison revealed that the component originated from a different manufacturer, despite matching electrical specifications. The discrepancy, confirmed through image-based fingerprinting, illustrated the system’s capability to detect unapproved substitutions invisible to traceability logs.

In another case, a passive component logged as Panasonic was identified visually as a Vishay part. Differences in package geometry and surface finish were detected through both object classification and segmentation. This mismatch, caused by a supplier labeling error, would not have been caught by traditional inspection methods.

A third case involved a component logged as a Bourns device but which could not be matched to any known reference in the golden dataset. The visual fingerprint failed classification, and the system flagged the part as an unknown. Manual investigation confirmed it to be an unauthorized or counterfeit component.

The system also demonstrated its ability to detect layout-level anomalies. As shown in Figure 5, top-side inspection identified boards with missing, added, and swapped components relative to the learned reference layout.

Visual inspection also revealed cases of hardware tampering techniques such as piggybacking, where an unauthorized component is soldered atop an approved one. This covert alteration, shown in Figure 6, bypassed standard AOI systems but was identified by the anomaly classifier.

Another notable case involved the post-production addition of a wire between two points on a board—an undocumented modification likely intended to override circuit behavior or enable a backdoor function. Figure 7 illustrates this anomaly, captured and flagged by the system.

A representative example is shown in Figure 8, where a crystal oscillator was substituted with a visually similar part from a different manufacturer. Though electrically equivalent, the system detected discrepancies in marking, pad layout, and surface finish, triggering an anomaly alert.

3.3. Scalability and Throughput

The framework was benchmarked for processing performance on standard AOI image resolutions. The detection pipeline processed images at <1000 ms per board, with component-level inferences running in under 10 ms per instance. This allows efficient post-production analysis of full board datasets without introducing bottlenecks to production workflows.

The method achieves:

• >99.3% anomaly detection accuracy
• <0.5% false positive rate
• <1% false negative rate

These metrics demonstrate the suitability of the framework for high-volume board analysis, enabling full coverage verification without reliance on manual inspection or destructive testing.

4. Discussion

The results demonstrate that deep learning models trained on AOI image data can achieve high accuracy in component detection, segmentation, and anomaly classification across diverse board conditions. Unlike traditional inspection techniques, which are often constrained by rule-based logic or sample-based strategies, the proposed framework provides data-driven, full-coverage inspection at the component level, using visual cues alone.

A key strength of the system lies in its ability to generalize across pristine, mildly degraded, and severely contaminated boards. This resilience is essential for real-world deployment, where imaging conditions, component aging, or handling artifacts may affect visual quality. The bootstrapping-based training pipeline, incorporating clean and field-grade boards, ensures that the model is robust to such variations, maintaining low false positive rates even under challenging conditions.

Importantly, the framework does not rely on electrical signals, traceability logs, or physical probing to detect anomalies. Instead, it leverages high-resolution imaging data already produced by AOI systems, making it non-invasive and readily integrable into existing manufacturing workflows. This enables scalable deployment without the need for additional hardware or inspection stations.

The anomaly detection pipeline also highlights a key benefit of combining object detection and segmentation. While detection provides spatial and classification data, segmentation enables pixel-level geometry analysis. Together, these allow for nuanced differentiation between normal variation and security-relevant modifications such as component substitution, piggybacking, or footprint mismatch.

While the performance metrics are strong, several limitations remain. First, the model depends on access to high-quality AOI images, which may vary by manufacturer and inspection system. Second, while the system can flag unknown components or mismatches, root cause analysis, such as determining whether a discrepancy is due to fraud, error, or rework, still requires human investigation. Finally, expanding the reference database of verified components and improving automated labeling of anomalies remain ongoing challenges for scaling the system across new product lines.

Despite these considerations, the approach marks a shift toward intelligent, vision-based quality and security assurance. By transforming AOI data into a rich source of structured information, manufacturers can move beyond static rules and manual inspection toward adaptive, AI-driven verification pipelines.

This methodology can also complement existing traceability and cybersecurity frameworks, providing visual evidence of component integrity and enhancing supply chain transparency. As electronic systems become increasingly interconnected and mission-critical, the ability to verify every board’s composition through image-based AI may become an essential tool in ensuring both quality and trust.

5. Conclusion

This work presents a secure hardware assurance solution based on deep learning applied to AOI images of electronic assemblies. By integrating object detection, segmentation, and spatial anomaly classification, the system performs high-precision component-level verification without requiring electrical tests or destructive analysis. Trained across a diverse set of production and degraded boards, the framework generalizes well to real-world conditions and operates efficiently within high-throughput environments.

Unlike traditional inspection systems limited to rule-based validation or sample testing, this approach provides full-coverage verification using only existing imaging infrastructure. With over 99% anomaly detection accuracy and millisecond-scale inference times, it enables practical deployment at scale, enhancing quality control and elevating hardware trust.

Recent events, including the discovery of rogue communication modules embedded in solar inverters, have exposed the limitations of conventional traceability and underscored the urgency of verifying what is physically present on a board. The proposed visual AI framework addresses this gap by transforming AOI data into a tool for structural and provenance-level analysis.

By bridging manufacturing analytics with cybersecurity principles, this method lays the foundation for scalable, non-invasive, and intelligent assurance of hardware integrity—essential for safeguarding the global electronics supply chain against tampering, counterfeiting, and cyber-physical compromise.