Submitted:
05 November 2024
Posted:
06 November 2024
You are already at the latest version
Abstract
Keywords:
1. Introduction
1.1. Convolutional Neural Networks (CNNs)
1.2. Physics-Informed Neural Networks (PINNs)
2. Materials and Methods
2.1. The training Dataset
2.2. The Neural Network
- Input: The input tensor has dimensions .
- Layer 1: A 3D convolutional layer with 16 filters, kernel size , strides of , and ’same’ padding. This is followed by a residual block with 16 filters.
- Layer 2: A 3D convolutional layer with 32 filters, kernel size , strides of , and ’same’ padding. This downscales the input. This is followed by a residual block with 32 filters.
- Layer 3: A 3D convolutional layer with 64 filters, kernel size , strides of , and ’same’ padding. This is followed by a residual block with 64 filters.
- Layer 4: A 3D convolutional layer with 128 filters, kernel size , strides of , and ’same’ padding. This is followed by a residual block with 128 filters.
- Bottleneck: A 3D convolutional layer with 256 filters, kernel size , and ’same’ padding, followed by a residual block with 256 filters.
- Layer 5: A 3D transposed convolutional layer with 128 filters, kernel size , strides of , and ’same’ padding. This is followed by a residual block with 128 filters.
- Layer 6: A 3D transposed convolutional layer with 64 filters, kernel size , strides of , and ’same’ padding. This is followed by a residual block with 64 filters.
- Layer 7: A 3D transposed convolutional layer with 32 filters, kernel size , strides of , and ’same’ padding. This is followed by a residual block with 32 filters.
- Layer 8: A 3D transposed convolutional layer with 16 filters, kernel size , strides of , and ’same’ padding. This is followed by a residual block with 16 filters.
- Output Layer: A 3D convolutional layer with 1 filter, kernel size , ’same’ padding, and ReLU activation, providing the predicted temperature field with dimensions .
2.3. Evaluation Metrics
3. Results and Discussion
3.1. Prediction with Ideal Conditions
3.2. Beyond Ideal Conditions
4. Improving Prediction Using Noise and Physics-Informed Training
5. Conclusions and Future Outlook
Author Contributions
Funding
Conflicts of Interest
References
- Vavilov, V.; Nesteruk, D.; Shiryaev, V. Thermal (infrared) tomography: terminology, principal procedures, and application to nondestructive testing of composite materials. Russ. J. Nondestr. Test. 2010, 46, 151–161. [Google Scholar] [CrossRef]
- Chen, C.Y.; Yeh, C.H.; Chang, B.; Pan, J.M. 3D reconstruction from IR thermal images and reprojective evaluations. Math. Probl. Eng. 2015, 1–8. [Google Scholar] [CrossRef]
- Schollemann, F.e.a. 3D Thermal Tomography with Experimental Measurement Data. Applied Thermal Engineering 2022, 200, 345–355. [Google Scholar]
- Ledwon, D.; Sage, A.; Juszczyk, J.; Rudzki, M.; Badura, P. Tomographic reconstruction from planar thermal imaging using convolutional neural network. Scientific Reports 2022, 12, 2347. [Google Scholar] [CrossRef]
- Toivanen, J.M.; Tarvainen, T.; Huttunen, J.M.J. 3D thermal tomography with experimental measurement data. Int. J. Heat Mass Transfer 2014, 78, 1126–1134. [Google Scholar] [CrossRef]
- Sage, A.; Ledwoń, D.; Juszczyk, J.; Badura, P. 3D thermal volume reconstruction from 2D infrared images—a preliminary study. Innovations in Biomedical Engineering. Springer, Cham, 2021, Vol. 1223, Advances in Intelligent Systems and Computing, pp. 371–379.
- Koutsantonis, L.; Rapsomanikis, A.N.; Stiliaris, E.; Papanicolas, C.N. Examining an image reconstruction method in infrared emission tomography. Infrared Physics & Technology 2019, 98, 266–277. [Google Scholar]
- He, Y.; Pan, M.; Luo, F. Infrared machine vision and infrared thermography with deep learning: A review. Infrared Physics & Technology 2021, 116, 103754. [Google Scholar]
- Wang, H.e.a. Deep Learning for 3D Thermal Tomography Using Infrared Imaging. Applied Thermal Engineering 2019, 150, 1145–1155. [Google Scholar]
- Mukherjee, R.e.a. Convergence and Robustness in 3D Thermal Tomography with Neural Networks. Journal of Imaging 2023, 9, 240–253. [Google Scholar]
- Zhang, X.e.a. Application of Neural Networks in Thermal Tomography for Subsurface Defect Detection. IEEE Transactions on Neural Networks and Learning Systems 2020, 31, 3762–3774. [Google Scholar]
- Bhatnagar, S.; Comerford, A.; Banaeizadeh, A. Physics-Informed Neural Networks for Modeling of 3D Flow Thermal Problems with Sparse Domain Data. Journal of Machine Learning for Modeling and Computing 2024, 5, 39–67. [Google Scholar] [CrossRef]
- Zobeiry, N.; Holler, M. Physics-informed neural networks for thermal imaging applications. Journal of Computational Physics 2021, 438, 110–133. [Google Scholar]
- Jones, M.; Tezuka, A.; Yamada, Y. Thermal tomographic detection of inhomogeneities. J. Heat Transfer 1995, 117, 969. [Google Scholar] [CrossRef]
- Bakirov, V.; Kline, R.; Winfree, W. Multiparameter thermal tomography. AIP Conference Proceedings 2004, 700, 461. [Google Scholar]
- Kolehmainen, V.; Kaipio, J.; Orlande, H. Reconstruction of thermal conductivity and heat capacity using a tomographic approach. Int. J. Heat Mass Transfer 2007, 50, 5150–5160. [Google Scholar] [CrossRef]
- Kaipio, J.; Somersalo, E. Statistical and Computational Inverse Problems; Springer Science+Business Media, Inc., 2005. [Google Scholar]
- Kervalishvili, G.; Schlichting, J.; Maierhofer, C.; Kreutzbruck, M. Heat Transfer Modelling of Local Thermal Excitation for Surface Crack Detection and Characterization. Proceedings of the 10th International Conference on Quantitative InfraRed Thermography (QIRT), 2010.
- Wysocka-Fotek, O.; Oliferuk, W.; Maj, M. Reconstruction of size and depth of simulated defects in austenitic steel plate using pulsed infrared thermography. Infrared Physics & Technology 2012, 55, 363–367. [Google Scholar]
- Marinetti, S.; Vavilov, V. IR thermographic detection and characterization of hidden corrosion in metals: General analysis. Corrosion Science 2010, 52, 865–872. [Google Scholar] [CrossRef]
- Ibarra-Castanedo, C.; Maldague, X. Pulsed phase thermography reviewed. Quant. Infrared Therm. J. 2004, 1, 47–70. [Google Scholar] [CrossRef]
- Sun, J.G. Pulsed thermal imaging measurement of thermal properties for thermal barrier coatings based on a multilayer heat transfer model. J. Heat Transfer 2014, 136, 081601. [Google Scholar] [CrossRef]
- Sun, J.G. Quantitative three-dimensional imaging of heterogeneous materials by thermal tomography. J. Heat Transfer 2016, 138, 112004. [Google Scholar] [CrossRef]
- Curran, D.; Porter, J.M. A tomography-based effective thermal conductivity model for ceramic fiber insulation. International Journal of Heat and Mass Transfer 2020, 160. [Google Scholar] [CrossRef]
- Wiegmann, A.; Wagner, H.; Kowarschik, R. Human face measurement by projecting bandlimited random patterns. Optics Express 2006, 14, 7692–7698. [Google Scholar] [CrossRef] [PubMed]
- Titman, D. Applications of thermography in non-destructive testing of structures. NDT & E International 2001, 34, 149–154. [Google Scholar]
- Vadivambal, R.; Jayas, D.S. Applications of Thermal Imaging in Agriculture and Food Industry—A Review. Food and Bioprocess Technology 2011, 4, 186–199. [Google Scholar] [CrossRef]
- Xu, Y.; Wei, X.; Wang, G. Temperature-Change-Based Thermal Tomography. International Journal of Biomedical Imaging 2009, 2009, 464235. [Google Scholar] [CrossRef]
- Ring, E.F.J.; Ammer, K. Infrared thermal imaging in medicine. Physiological Measurement 2012, 33, R33. [Google Scholar] [CrossRef]
- Kaczmarek, M.; Nowakowski, A. Active Dynamic Thermography in Medical Diagnostics; Springer, 2017. [Google Scholar]
- Hakim, A.; Awale, R.N. Thermal Imaging - An Emerging Modality for Breast Cancer Detection: A Comprehensive Review. Journal of Medical Systems 2020, 44. [Google Scholar] [CrossRef]
- Papanicolas, C.; Frixou, A.; Koutsantonis, L. Thermal Tomography for Medical Applications. 2023 IEEE Nuclear Science Symposium, Medical Imaging Conference and International Symposium on Room-Temperature Semiconductor Detectors (NSS MIC RTSD), 2023, pp. 1–1.
- Yao, G.; Lei, T.; Zhong, J. A review of Convolutional-Neural-Network-based action recognition. Pattern Recognition Letters 2019, 118. [Google Scholar] [CrossRef]
- Tod, G.; Ompusunggu, A.P.; Struyf, G.; Pipeleers, G.; Grave, K.D.; Hostens, E. Physics-Informed Neural Networks (PINNs) for Improving a Thermal Model in Stereolithography Applications. Procedia CIRP 2021, 104, 1559–1564, 54th CIRP CMS 2021 - Towards Digitalized Manufacturing 4.0. [Google Scholar] [CrossRef]
- Habring, S.; Holler, M. Neural-network-based regularization methods for inverse problems in imaging. GAMM-Mitteilungen 2024, 47, e202200020. [Google Scholar] [CrossRef]
- Gilton, D.; Ongie, G.; Willett, R. Deep equilibrium architectures for inverse problems in imaging. IEEE Transactions on Computational Imaging 2021, 7, 1123–1133. [Google Scholar] [CrossRef]
- Mukherjee, S.; Jabari, S.; Maier, A. Convergence guarantees for learned reconstruction methods with noisy data. IEEE Transactions on Pattern Analysis and Machine Intelligence 2023. [Google Scholar]
- Zobeiry, N.; Barati, M.R.; Soltani, M. Physics-informed machine learning for heat transfer problems: A comprehensive review. International Journal of Heat and Mass Transfer 2021, 172, 121100. [Google Scholar]







| Metric | No. of Hotspots | |||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
| SSIM | 0.95 ± 0.07 | 0.98 ± 0.02 | 0.98 ± 0.01 | 0.99 ± 0.01 | 0.98 ± 0.01 | 0.98 ± 0.01 |
| MSE | 0.0061 ± 0.0064 | 0.011 ± 0.0095 | 0.019 ± 0.017 | 0.028 ± 0.018 | 0.046 ± 0.027 | 0.060 ± 0.033 |
| PSNR | 35 ± 4 | 36 ± 3 | 35 ± 3 | 36 ± 2 | 34 ± 2 | 33 ± 2 |
| MAE | 0.034 ± 0.0047 | 0.046 ± 0.0090 | 0.057 ± 0.011 | 0.073 ± 0.013 | 0.093 ± 0.017 | 0.11 ± 0.023 |
| NMSE | 0.0340 ± 0.0414 | 0.0216 ± 0.0122 | 0.0202 ± 0.0130 | 0.0180 ± 0.0092 | 0.0234 ± 0.0110 | 0.0256 ± 0.0123 |
| Dice | 0.83 ± 0.23 | 0.86 ± 0.14 | 0.83 ± 0.25 | 0.85 ± 0.19 | 0.84 ± 0.19 | 0.83 ± 0.19 |
| IoU | 0.75 ± 0.23 | 0.78 ± 0.17 | 0.77 ± 0.26 | 0.78 ± 0.21 | 0.76 ± 0.21 | 0.74 ± 0.22 |
| ROI Metrics | ||||||
| SSIM (ROI) | 0.91 ± 0.08 | 0.95 ± 0.03 | 0.96 ± 0.02 | 0.97 ± 0.02 | 0.96 ± 0.02 | 0.96 ± 0.02 |
| MSE (ROI) | 0.13 ± 0.29 | 0.10 ± 0.14 | 0.10 ± 0.13 | 0.11 ± 0.10 | 0.16 ± 0.12 | 0.17 ± 0.13 |
| PSNR (ROI) | 23 ± 6 | 27 ± 4 | 29 ± 4 | 30 ± 3 | 29 ± 3 | 30 ± 3 |
| MAE (ROI) | 0.15 ± 0.11 | 0.15 ± 0.09 | 0.15 ± 0.06 | 0.16 ± 0.05 | 0.20 ± 0.06 | 0.21 ± 0.06 |
| NMSE (ROI) | 0.1675 ± 0.2087 | 0.0965 ± 0.0707 | 0.0768 ± 0.0626 | 0.0538 ± 0.0408 | 0.0657 ± 0.0384 | 0.0590 ± 0.0311 |
| Dice (ROI) | 0.83 ± 0.23 | 0.87 ± 0.14 | 0.84 ± 0.25 | 0.86 ± 0.19 | 0.85 ± 0.19 | 0.83 ± 0.19 |
| IoU (ROI) | 0.76 ± 0.24 | 0.79 ± 0.17 | 0.77 ± 0.26 | 0.79 ± 0.21 | 0.77 ± 0.22 | 0.75 ± 0.22 |
| Metric | No. of Hotspots (Ideal → Noisy) | ||
|---|---|---|---|
| 1 | 2 | 3 | |
| SSIM | 0.95 ± 0.056 → 0.45 ± 0.19 | 0.98 ± 0.017 → 0.49 ± 0.17 | 0.99 ± 0.0065 → 0.61 ± 0.14 |
| NMSE | 0.038 ± 0.038 → 0.83 ± 0.48 | 0.030 ± 0.018 → 0.79 ± 0.41 | 0.027 ± 0.020 → 0.69 ± 0.34 |
| ROI Metrics | |||
| SSIM (ROI) | 0.93 ± 0.058 → 0.37 ± 0.31 | 0.95 ± 0.028 → 0.45 ± 0.22 | 0.96 ± 0.017 → 0.56 ± 0.17 |
| NMSE (ROI) | 0.15 ± 0.16 → 2.8 ± 1.9 | 0.10 ± 0.076 → 2.1 ± 1.4 | 0.079 ± 0.058 → 1.6 ± 0.97 |
| Metric | No. of Hotspots | |||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
| Without Noise or PINN | ||||||
| SSIM | 0.43 ± 0.17 | 0.38 ± 0.15 | 0.38 ± 0.15 | 0.49 ± 0.14 | 0.55 ± 0.12 | 0.60 ± 0.11 |
| NMSE | 0.95 ± 0.43 | 1.09 ± 0.44 | 1.22 ± 0.49 | 0.97 ± 0.43 | 0.95 ± 0.40 | 0.84 ± 0.34 |
| SSIM (ROI) | 0.28 ± 0.26 | 0.31 ± 0.20 | 0.35 ± 0.19 | 0.49 ± 0.18 | 0.52 ± 0.14 | 0.57 ± 0.12 |
| NMSE (ROI) | 3.23 ± 1.73 | 2.87 ± 1.56 | 2.81 ± 1.45 | 1.96 ± 1.16 | 1.87 ± 1.01 | 1.55 ± 0.76 |
| Noise Added During Training | ||||||
| SSIM | 0.65 ± 0.19 | 0.82 ± 0.13 | 0.92 ± 0.07 | 0.95 ± 0.04 | 0.95 ± 0.02 | 0.96 ± 0.01 |
| NMSE | 0.30 ± 0.30 | 0.16 ± 0.16 | 0.08 ± 0.06 | 0.06 ± 0.03 | 0.06 ± 0.03 | 0.06 ± 0.03 |
| SSIM (ROI) | 0.72 ± 0.21 | 0.82 ± 0.13 | 0.89 ± 0.05 | 0.91 ± 0.04 | 0.91 ± 0.04 | 0.91 ± 0.03 |
| NMSE (ROI) | 0.82 ± 0.93 | 0.42 ± 0.53 | 0.19 ± 0.14 | 0.14 ± 0.09 | 0.14 ± 0.07 | 0.13 ± 0.06 |
| With Noise During Training and PINN | ||||||
| SSIM | 0.91 ± 0.09 | 0.96 ± 0.02 | 0.97 ± 0.01 | 0.97 ± 0.01 | 0.97 ± 0.01 | 0.97 ± 0.01 |
| NMSE | 0.08 ± 0.09 | 0.05 ± 0.03 | 0.05 ± 0.03 | 0.04 ± 0.02 | 0.05 ± 0.02 | 0.06 ± 0.02 |
| SSIM (ROI) | 0.87 ± 0.11 | 0.91 ± 0.05 | 0.93 ± 0.03 | 0.93 ± 0.03 | 0.92 ± 0.02 | 0.92 ± 0.03 |
| NMSE (ROI) | 0.29 ± 0.35 | 0.17 ± 0.11 | 0.13 ± 0.09 | 0.11 ± 0.07 | 0.13 ± 0.06 | 0.12 ± 0.05 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).