Fusion of Simulated and Experimental Thermal Data for Robust Electronic Packaging Defect Detection

1. Introduction

As semiconductor packaging technology develops towards higher integration and stronger functional density, devices have put forward more stringent requirements for thermal reliability. Defects in the packaging structure, such as missing solder balls, faulty soldering of pad, bridging, etc., usually cause the heat flow conduction path to shift, resulting in abnormal local temperature rise, which may cause chip failure or even burning. How to efficiently and accurately detect internal defects in the package has become one of the core challenges in chip manufacturing and packaging quality control.

In terms of defect detection, traditional detection methods rely on manual visual inspection or single-mode imaging technology, which is difficult to meet the requirements of accurate and rapid identification of micron-level defects. In recent years, infrared imaging technology has become an important means of packaging defect detection due to its non-invasive thermal distribution capture capability. Due to its non-contact, real-time and non-destructive characteristics, infrared defect detection technology has been applied in aerospace[1,2,3,4], automotive industry[5], construction[6,7], photovoltaic industry[8,9] and chip packaging[10,11]. At the same time, researchers have tried to use infrared thermal imaging to detect thermal anomalies inside chips, especially active heating infrared imaging[12,13], which can observe thermal distribution distortion in a non-destructive manner to identify internal defects. Technologies such as infrared lock-in thermal imaging[14] and double-thermal imaging[15] have been continuously developed to improve temperature contrast and image accuracy. However, this technology still has limitations in terms of resolution, penetration depth and noise immunity.

In addition to infrared detection, finite element analysis reveals the correlation mechanism between defects and temperature field distortion through multi-physics field modeling. It is widely used to model thermal-mechanical multi-physics field problems and has been successfully applied to chip thermal warpage prediction[16], material thermal conductivity difference analysis and solder ball defect modeling[17]. The finite element simulation method can systematically reveal the influence of defects on thermal field distribution and construct simulated samples to make up for the lack of experimental data. However, its challenge is that the modeling accuracy is limited by material parameters and computing costs, and its adaptability to batch scenarios still needs to be improved.

With the support of infrared detection and finite element simulation, deep learning algorithms provide reliable technical support for high-precision real-time detection. In defect detection, deep learning algorithms such as the YOLO series [18] have been widely used in industrial inspection tasks in recent years. They have the advantages of end-to-end training, strong small target recognition ability, and fast reasoning speed, and are suitable for high-speed and high-precision defect positioning in chip packaging. Many studies have applied algorithms such as YOLO v4 [19,20], v5 [21], v7 [22], v8 [23], and YOLO X [24] to chip surface or internal defect detection, and combined with improvement strategies to achieve improved detection accuracy. However, its performance is still limited under the conditions of small samples, cross-modal images, and data distribution offset. Relying solely on infrared or measured image training has the problem of insufficient robustness.

To address the limitations of existing methods, recent research trends have focused on integrating physics-based simulation with deep learning to establish multimodal defect detection frameworks. Finite element analysis (FEA) provides accurate modeling of heat conduction behavior under defined material properties and boundary conditions, offering a reliable physical foundation for understanding temperature field distributions. Meanwhile, end-to-end deep learning models, such as the YOLO series, demonstrate superior image recognition capabilities, enabling rapid localization of defect regions in complex images. By combining these two complementary approaches, it is possible to achieve collaborative modeling that bridges physical mechanisms and data-driven recognition, thereby enhancing both the interpretability and robustness of defect detection systems.

This work proposed a multimodal detection framework for semiconductor packaging defects that integrates finite element thermal simulation, infrared thermal imaging, and the YOLO11 deep learning detection network. The main contributions of this work are summarized as follows:

(1) A comprehensive finite element model of a typical BGA package is constructed, incorporating representative defects such as missing solder balls and bridging. Both steady-state and transient thermal responses are analyzed to characterize defect-induced temperature anomalies.

(2) An experimental infrared thermal imaging system is developed to capture real thermal images of defective packages. These measurements are compared with simulation results to validate the physical consistency of the thermal model.

(3) A hybrid dataset, combining simulated and experimental thermal images, is constructed to train and evaluate the YOLO11 network. This integration significantly enhances the model’s detection accuracy and generalization capability across simulated and real-world scenarios.

2. Proposed Simulation and Experimental Methods

Here, we present a detailed description of the finite element thermal simulation process for chip packaging defects, the YOLO11-based target detection architecture, and the infrared thermal image acquisition platform. The overall technical workflow is illustrated in Figure 1. First, temperature distribution images of packaged chips under various operating conditions are generated through finite element simulations and infrared thermal imaging experiments, resulting in two types of datasets: simulated images and experimental images. These datasets are then merged and expanded through data augmentation techniques, including random rotation, occlusion, noise injection, and flipping, to enhance model robustness and generalization. The augmented dataset is subsequently fed into the YOLO11 detection model, which comprises three primary components: a backbone network for feature extraction, a neck network for multi-scale feature fusion, and a detection head for defect localization and classification. The model outputs include detected defect regions, defect types, and corresponding confidence scores. By combining physics-based simulation with data-driven learning, this method leverages both physical interpretability and high-precision detection capability, providing a comprehensive solution for automated chip packaging defect identification.

2.1. Finite Element Modeling of Packaging Structure

A three-dimensional finite element model of a Ball Grid Array (BGA) package was developed, consisting of the package substrate, solder pads, solder balls, and printed circuit board (PCB), as illustrated in Figure 2a.

The package substrate is the core carrier of the chip package, which is mainly used to realize the electrical interconnection between the die and the external circuit, mechanical support and heat dissipation functions. In this simulation, we use a 5mm×5mm×0.8mm rectangular block to construct the outer size of the package substrate. Compared with the size of the package substrate, the size of the die is small and can be ignored in terms of geometric scale.

Solder pads, represented as cylindrical copper structures with a diameter of 0.5 mm and thickness of 0.1 mm, were arranged in a 4×4 grid on the bottom surface of the substrate with a 1 mm pitch. These pads serve as electrical and mechanical interfaces for solder ball connections. The solder balls were modeled as bulging spheres with a diameter of 0.56 mm and thickness of 0.4 mm, simulating their deformed shape after the reflow process. The PCB, modeled as a 10 mm × 10 mm × 1.6 mm FR-4 block, supports the package and facilitates system-level interconnection.

An adaptive meshing strategy was employed to refine the mesh in regions with high thermal gradients (Figure 2b), ensuring accurate capture of localized temperature variations. Material properties, including density, thermal conductivity, and specific heat capacity, were defined based on Comsol’s material library and are summarized in Table 1. Considering that the packaging substrate is the main source of heat, the boundary conditions are set as follows:

1) Heat source conditions: The temperature at the top of the packaging substrate is a constant temperature, such as 100°C, 200°C, etc., and solid heat transfer is performed downward.

2) Heat dissipation conditions: The initial temperature of all external boundaries is room temperature 20°C, and the convection heat transfer coefficient between the external 20°C air is 5W/(m²·K). And the additional contact thermal resistance between the package substrate, pads, solder balls, and printed circuit boards is not considered. Finally,the thermal radiation phenomenon of the packaged chip is ignored.

2.2. Dataset Construction and Deep Learning Framework

In order to improve the generalization ability and robustness of the model in defect detection tasks, this study constructs a dataset that integrates finite element simulation images and infrared thermal imaging images, and uses the YOLO11 deep neural network as the core detection framework. Data construction includes simulation image annotation, format conversion, image augmentation and other links to ensure that defect features are fully expressed.

Data augmentation includes geometric transformation (rotation, flipping) and thermal noise injection. This experiment adopts a probabilistic combination augmentation strategy, through random rotation (30% trigger probability, maximum rotation angle ±45°), random flipping (50% trigger probability), random cropping (40% trigger probability, retaining at least 60% of the frame) and other geometric transformations, combined with Gaussian noise injection (20% trigger probability) and random occlusion (30% trigger probability, occlusion area 2-20%, random color block filling) to build an augmented dataset. A typical example is shown in Figure 3.

The original simulation samples are 68 images of missing solder balls, missing solder ball and pad, bridge and mixed defects, which are expanded to 768 images after augmentation and divided into training, validation and test sets (8:1:1). The model labels include four categories: missing solder balls, missing solder ball and pad and bridge, corresponding to category indexes 0 to 2 respectively.

The YOLO11 network structure is divided into three parts: the backbone network, the neck network and the detection head[25]. The backbone adopts the improved structure of CSPDarknet-Deep, and introduces the C2PSA attention mechanism and SPPF module to improve the feature extraction capability. And the neck adopts a bidirectional weighted feature fusion module to enhance the multi-scale information integration. Finally, the detection head supports three-scale output, and has a good effect on small target defect positioning.

2.3. Experimental Setup for Infrared Thermal Imaging

An experimental platform was constructed to validate the simulation and detection methods (Figure 4). Defective BGA samples were prepared on FR-4 PCBs using a 4×4 solder ball array. Defects such as missing solder balls and bridging were manually introduced. A FLUKE TiS55 infrared thermal imager, coupled with a VECTECH V-1515 heater, was used to capture thermal images. Imaging parameters, including emissivity, background temperature, and transmittance, were calibrated using SmartView software before data collection.

2.4. Model Training and Evaluation Metrics

The YOLO11 model was trained using the PyTorch framework on an NVIDIA RTX 3090 GPU for 200 epochs, with an initial learning rate of 0.01 and a batch size of 128. Performance was evaluated using the following metrics: 1) Precision, Recall, and mAP (mean Average Precision) at IoU thresholds of 0.5 (mAP50) and 0.5–0.95 (mAP50-95); 2) FPS (Frames Per Second) to assess real-time detection capability; 3) Params (number of model parameters) and FLOPs (floating-point operations per second) to evaluate model complexity. These metrics provide a comprehensive assessment of the model’s accuracy, speed, and computational efficiency, supporting its application in high-speed industrial defect detection.

3. Results and Discussion

3.1. Thermal Simulation Analysis of Packaging Defects

The impact of different packaging defects on the thermal behavior of BGA structures was systematically analyzed through steady-state and transient simulations. Four scenarios were considered: normal packaging, missing solder balls, missing solder ball and pad, and bridging. The heat dissipation pathways under these conditions are schematically illustrated in Figure 5. Under normal conditions, the heat generated by the package substrate can be evenly transferred through the conduction path of substrate-solder pad-solder ball-PCB. However, in the case of a missing solder ball, the heat cannot be conducted through the above path, but only conducts to the pad or conducts through the normal solder balls around the missing ball, and finally conducts heat laterally on the PCB. When both the solder ball and its corresponding pad are missing, the heat conduction mode is similar to that of missing solder ball. The heat on the package substrate is conducted around the normal solder balls around, and it is also conducted laterally on the PCB. In the case of bridging, where two or more solder balls are unintentionally connected, an additional lateral heat conduction path is introduced. This alters the local temperature distribution by enabling heat to flow not only through the normal vertical path but also across the bridged solder balls, potentially impacting the system’s overall thermal performance.

Under the heating condition of 200 ℃ on the top of the FR-4 package substrate, the temperature distribution on the top of the PCB is shown in Figure 6. Under normal conditions, as shown in Figure 6a, the temperature field exhibits a symmetrical diffusion pattern, with heat transferred from the highest-temperature region at the center of the chip to the PCB through the solder ball array. The temperatures of the four central solder balls are significantly higher than those of the twelve peripheral solder balls, indicating that the peripheral solder balls dissipate heat more efficiently due to enhanced thermal convection. In contrast, the slower heat dissipation of the central solder balls leads to heat accumulation. Meanwhile, the four corners of the square PCB represent the lowest temperature zones, forming cold regions as both adjacent sides facilitate heat dissipation via convection.In the absence of solder balls, as illustrated in Figure 6b, the original symmetrical temperature distribution is disrupted. A cold spot with a temperature below 170 °C emerges in the upper-right corner of the solder ball array, and the temperature gradient of the adjacent solder balls increases. This suggests that heat is redirected around the missing region toward neighboring solder balls, which is consistent with the thermal resistance network analysis. At this point, the four central solder balls still maintain relatively high temperatures, and the cold zones at the PCB corners remain prominent.When the pad is further removed (Figure 6c), the resulting temperature distribution remains highly similar to the case involving only missing solder balls. This implies that a pad not in direct contact with the PCB has a limited effect on the overall temperature field.Under solder ball bridging conditions (Figure 6d), the temperature field largely reverts to a nearly symmetrical diffusion pattern, with no distinct cold spots observed. However, in the lower-left bridging region, the temperatures of the two connected solder balls tend to equalize due to lateral heat conduction between them. Additionally, the temperature in the bridged area is higher than that observed in the original air gap, thereby confirming that bridging may induce abnormal hot spots.

In order to evaluate whether pad loss affects the temperature distribution, the temperature field of the package substrate is further analyzed. By integrating the heat dissipation path analysis with the substrate simulation results, several conditions are compared. Under normal conditions (Figure 7a), the temperature field exhibits a symmetrical diffusion pattern. When solder balls are missing (Figure 7b), a distinct hot spot with a temperature of 197 °C appears at the defect location in the upper-right corner, and a clear circular boundary corresponding to the pad is observed.

Under the condition where both solder balls and solder pads are missing (Figure 7c), a similar hot spot emerges at the upper-right defect, with a slightly higher temperature of 197.22 °C. However, in this case, no clear circular boundary is observed, suggesting that the absence of the pad reduces thermal coupling in that region. In the solder ball bridging condition (Figure 7d), the temperature at the lower-left bridging site is notably lower than that of the surrounding solder balls, with a recorded temperature of 185.36 °C. This phenomenon corresponds to an abnormal cold spot, indicating that bridging may lead to localized thermal anomalies distinct from those caused by missing solder elements.

Further transient analysis reveals that different types of defects have markedly different effects on the heating rate and temperature saturation time. As illustrated in Figure 8, a comparison between the normal package and the solder ball missing condition shows that the heating rate at the center point of the solder ball during the first 2 seconds reaches 41.9 °C/s under the missing condition, which is 47% higher than the 28.5 °C/s observed under normal conditions. Moreover, temperature saturation is achieved in just 1.4 seconds, significantly faster than the 37.5 seconds required under normal conditions, indicating a pronounced local heat load concentration effect.

In contrast, under the solder ball bridging condition, the heating rate at the center of the bridged region during the first 2 seconds is 32.3 °C/s, representing a 13% increase compared to the normal case. The corresponding temperature saturation time is 17.0 seconds—faster than the normal condition but still slower than that observed in the solder ball missing condition. This suggests that although the lateral heat conduction path introduced by bridging accelerates the initial heat diffusion, the added thermal mass of the solder reduces the efficiency of longitudinal heat transfer.

Notably, the heating rate and temperature saturation time in the solder ball missing condition and the combined solder ball and pad missing condition are almost completely overlapping. This is because both defects disrupt the vertical heat conduction path, forcing heat to diffuse laterally through the substrate, which has relatively low thermal conductivity. As a result, the heating rate is significantly reduced and the thermal responses tend to converge.

In terms of heating rate, the solder ball missing condition exhibits the fastest rate at 41.9 °C/s, followed by the solder ball bridging condition at 32.3 °C/s, while the normal packaging condition is the slowest at 28.5 °C/s. When evaluated based on temperature saturation time, the solder ball missing condition again shows the fastest response, reaching saturation in just 1.4 seconds, followed by the bridging condition at 17.0 seconds, with the normal condition being the slowest at 37.5 seconds.

These differences among the three cases are fundamentally attributed to variations in heat transfer paths. Specifically, solder ball loss results in a rapid temperature rise due to circumferential heat flow around the defect. Solder ball bridging, on the other hand, partially compensates for heat transfer efficiency through an alternative lateral conduction channel. Meanwhile, the effect of solder pad absence is essentially masked by the low thermal conductivity of the substrate, as the missing pad does not contribute to effective thermal coupling.

3.2. Performance Evaluation of Defect Detection

In order to evaluate the effect of training in defect detection, the YOLO11 model was trained on two sets of data: using only simulated images and hybrid data of simulated and infrared images, and performing defect detection in simulated images and real infrared images respectively. Before evaluating the results, the following evaluation indicators need to be briefly explained. Precision and recall are important indicator parameters, which are calculated from TP, FP and FN as shown in Equations (1) and (2):

$$\mathrm{Precison}={\displaystyle \frac{TP}{TP+FP}}$$

(1)

$$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}$$

(2)

where TP refers to the model correctly detecting an object, FP refers to the model incorrectly detecting an object (including incorrect category or no object), and FN refers to the model not detecting an actual object. Precision indicates how many of the detected objects are correct. The closer its value is to 1, the more accurate the detection. And recall indicates how many of all the real objects the model has found. The closer its value is to 1, the more it has detected.

Under ideal conditions, both precision and recall need to be high. Therefore, with Precision as the vertical axis and Recall as the horizontal axis, multiple (Precision, Recall) points are plotted through different confidence thresholds to form a PR curve. The area enclosed by the curve and the coordinate axis is AP (Average Precision). mAP is the average of APs of multiple categories. Its calculation is shown in Equation (3) and Equation (4):

$$AP={\displaystyle \underset{0}{\overset{1}{\int }}p(r)dr}$$

(3)

$$mAP={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^{N}A{P}_i}$$

(4)

The PR curve in Equation (3) is$p=p(r)$: N is the number of classification categories in Equation (4), and the subscript$i$is the$i^{th}$category.

In order to refine the accuracy reference indicators in different fields, two indicators, mAP50 and mAP50-95, are introduced here. mAP50 is $IoU\ge 0.5$to calculate the mAP of all categories when . At this time, the evaluation accuracy is relatively loose, so mAP50 is suitable for the accuracy evaluation of lightweight models or rapid deployment. mAP50-95 calculates the mAP for every 0.05 interval from IoU from 0.5 to 0.95, and then takes the average. Therefore, mAP50-95 can better reflect the actual detection quality of the model than mAP50, and is suitable for more rigorous accuracy evaluation. The numbers contained in the two indicators refer to the intersection over union (IoU), which is defined by Equation (5):

$$IoU={\displaystyle \frac{\left|A\cap B\right|}{\left|A\cup B\right|}}$$

(5)

where the set$A$refers to the set of predicted boxes, the set$B$is the set of real boxes, the numerator refers to the overlapping area of the predicted box and the real box, and the denominator refers to the total coverage area of the two boxes. The ratio ranges from [0,1].

In addition, in the object detection network, the frame rate (Frames Per Second, FPS) is often used to measure the inference speed of the model. In the actual training process, the calculation of FPS is shown in Equation (6):

$$FPS={\displaystyle \frac{1}{t_{pre}+t_i+t_{post}}}$$

(6)

where $t_{pre}$refers to the image pre-processing time, $t_i$refers to the image inference time, and $t_{post}$refers to the image post-processing time. The larger the FPS value, the faster the model detection speed.

As for model complexity, commonly used indicators are model parameter quantity (Params) and computation quantity (Floating-point Operations Per Second, FLOPs), which correspond to spatial complexity and time complexity respectively. Params refers to the total number of parameters that need to be trained in the network model, while FLOPs refers to the number of floating-point operations per second. The smaller the values of the two, the more streamlined the model.

Figure 9 presents representative examples of detection results for different defect types in this test, while Table 2 summarizes the key performance metrics of various models on the simulated dataset. From the perspective of detection accuracy, YOLOv8 demonstrates the best overall performance, achieving an mAP50–95 of 94.5%, slightly surpassing other models. This indicates greater robustness across varying IoU thresholds. All models exhibit excellent performance in terms of mAP50, with values exceeding 99.3%. Among them, YOLOv8, YOLOv9t, and YOLO11n each reach 99.5%, confirming the reliability of the YOLO series in high-confidence detection tasks. YOLOv10n shows a slightly lower mAP50, which may be attributed to reduced feature representation capability due to its lightweight architecture.

In terms of inference speed, both YOLOv8 and YOLOv10n achieve 667 FPS, significantly outperforming YOLO11n and YOLOv9t. Notably, YOLOv10n maintains the same inference speed as YOLOv8 while reducing the number of parameters by 10%, showcasing the advantages of a lightweight design. YOLOv9t has the lowest FPS among the models. Although it delivers high accuracy, the trade-off in speed makes it more suitable for high-precision but non-real-time applications.

Regarding model complexity, YOLOv9t has the smallest number of parameters, yet its FLOPs are comparable to those of YOLOv10n, suggesting a compact structure but suboptimal computational efficiency. In contrast, YOLO11n exhibits the lowest FLOPs—22% lower than those of YOLOv8—and a parameter count similar to YOLOv10n, reflecting an efficient architectural design for computational performance. Although YOLOv8 does not achieve the lowest parameter count or FLOPs, it provides the best trade-off between accuracy and speed.

For YOLO11n, the mAP50 is on par with YOLOv8 and YOLOv9t, and surpasses YOLOv10n. Its FLOPs are the lowest among the compared models, ranging from 18% to 24% lower, which makes it well-suited for low-power environments. Additionally, its frame rate meets real-time industrial detection requirements. The lightweight design also facilitates deployment on edge devices. Overall, YOLO11n presents a viable solution for intelligent industrial defect detection by offering high accuracy, computational efficiency, and strong real-time performance.

In the hybrid dataset, the training effect of the YOLO11 model is shown in Figure 10. In the figure, box_loss represents the bounding box loss, that is, the loss between the predicted box and the marked box. As the training progresses, box_loss gradually decreases, with an obvious convergence trend, indicating that the training has been able to locate the defect. cls_loss represents the classification loss, which is used to measure the difference between the predicted category and the true category. There are fewer classification categories here, cls_loss converges quickly, and the category prediction ability training effect is better. dfl_loss represents the distribution focal loss, which is used to assist box_loss and improve recognition accuracy. It shows a convergence trend similar to the first two curves, indicating that the accuracy of model training has steadily improved.

The final prediction results are shown in Table 3. After training with the hybrid dataset, the target detection effect is greatly improved. Of the same 12 images, 11 images can be successfully identified, with a recognition rate of 91.7%. Compared with the 41.7% recognition rate of the model trained only with the simulation dataset, the recognition rate has increased by 120%. At the same time, the images that failed to be successfully identified were checked, and the results showed that a solder ball missing defect had been successfully marked, but the confidence was only 0.27, so it was ignored. In summary, in terms of real target detection, real datasets need to be added to further optimize detection capabilities and realize industrial applications.

4. Conclusion

This study presents an integrated defect detection framework for semiconductor packaging, combining finite element thermal simulation, infrared thermal imaging, and YOLO11-based deep learning. By modeling typical defects—including missing solder balls, missing pads, and solder ball bridging—the study systematically analyzed their influence on steady-state and transient thermal distributions. Simulation results demonstrated strong consistency with experimental infrared measurements, with a maximum temperature discrepancy of less than 5%, validating the physical accuracy of the simulation model.

A hybrid dataset composed of simulated and experimentally captured thermal images was constructed to train the YOLO11 network. The proposed model achieved a high detection accuracy of 99.5% mAP50 on the simulation dataset and maintained a real-time inference speed of 556 FPS. More importantly, integrating real thermal images into the training process improved the model’s defect recognition rate on experimental data from 41.7% to 91.7%, effectively addressing domain adaptation challenges. These results confirm that the proposed multimodal framework offers a practical, high-precision, and real-time solution for industrial semiconductor packaging defect detection.

Although the hybrid dataset significantly improves the model performance, it still has the following limitations: First, the number of experimental image samples is limited, and the defect types are unevenly distributed, which may affect the model’s ability to distinguish between multiple types of targets. Second, the resolution of infrared imaging is limited, and the boundaries of some defects are blurred, affecting the accuracy of target positioning. In future work, efforts will focus on enlarging the experimental dataset, integrating advanced imaging technologies, refining the model for edge deployment, and extending the method to support a wider range of packaging structures and defect types. These advancements aim to promote the development of intelligent, automated, and scalable defect detection solutions for semiconductor manufacturing.