Thermal Single-Image Super-Resolution: A Review and Demosaicking-Inspired Deep Fusion Framework

1. Introduction

Thermal imaging has become an important sensing technology in building inspection, structural health monitoring, industrial monitoring, surveillance, autonomous navigation, and security applications. Unlike conventional RGB cameras, thermal cameras capture infrared radiation emitted by objects. Therefore, thermal images can reveal temperature-related information that is not directly visible in the visible spectrum, such as heat leakage, moisture accumulation, insulation failure, and abnormal thermal patterns.

Thermal imaging is particularly important for inspection tasks because many defects are not visible in the RGB domain. Regions that appear visually homogeneous in RGB images may contain hidden thermal patterns caused by moisture, heat leakage, insulation failure, or structural inconsistency [1,2].

As illustrated in Figure 1 and summarized in Table 1, thermal imaging provides complementary information to RGB imaging by revealing temperature distributions and hidden defects that are often invisible in the visible spectrum. These characteristics make thermal imaging particularly valuable for building inspection, where moisture intrusion, heat leakage, insulation failure, and structural inconsistencies must be accurately identified.

Despite these advantages, thermal imaging is still constrained by the relatively low spatial resolution of infrared sensors. High-resolution infrared detectors are expensive and technically difficult to manufacture, making low-resolution thermal cameras the predominant choice in many practical applications. Consequently, improving the spatial resolution of thermal images has become an important research problem, motivating the development of thermal image super-resolution methods.

Image super-resolution (SR) aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) observation. In visible-light imaging, SR has progressed from classical interpolation methods to convolutional neural networks (CNNs), generative adversarial networks (GANs), transformers, state-space models, hypercomplex neural networks, and diffusion-based methods [4,5]. However, thermal image SR remains more challenging because infrared images have different statistical properties from RGB images, including weaker textures, lower contrast, sensor-dependent noise, and fewer large-scale public datasets [6,7].

This paper reviews thermal and infrared image super-resolution from the perspectives of datasets, evaluation metrics, reconstruction methods, and open challenges. In addition to reviewing existing methods, this paper introduces a demosaicking-inspired upsampling framework for thermal SR and presents FusionNet as an adaptive fusion case study. The proposed direction is motivated by the similarity between demosaicking and image upsampling: in both cases, only a subset of samples is known, while missing values must be reconstructed from neighboring observations.

The main contributions of this paper are summarized as follows:

• Comprehensive Review: A comprehensive review of thermal image super-resolution (SR) is presented, covering thermal imaging characteristics, publicly available datasets, evaluation metrics, and representative classical and deep-learning-based SR methods.
• Demosaicking-Inspired Deep Fusion Framework: A novel thermal-image SR framework is proposed that reformulates bilinear interpolation, Menon interpolation, and the Malvar–He–Cutler demosaicking method using a zero-insertion strategy. FusionNet fuses the resulting reconstruction candidates with the RLKA-Net output, learning spatially adaptive, pixel-wise fusion weights.
• Performance Evaluation and Hypercomplex Fusion: Extensive experiments demonstrate that FusionNet consistently outperforms the individual reconstruction methods. In addition, quaternion-based variants, QFusionNet and QFusionNet+, reduce parameters by about 75% while maintaining comparable reconstruction quality, with QFusionNet+ achieving slight performance improvements over the real-valued model.

The remainder of this paper is organized as follows. Section 2 introduces the background and problem formulation. Section 3 reviews datasets and assessment metrics. Section 4 presents a taxonomy of thermal SR methods. Section 5 describes the proposed demosaicking-inspired framework and FusionNet. Section 6 presents the experimental evaluation. Section 7 discusses open challenges and future directions. Section 8 concludes the paper.

2. Background

2.1. Thermal Imaging

Thermal cameras measure infrared radiation emitted by objects and convert it into image intensities. Since emitted radiation is related to temperature, thermal images provide information that is complementary to visible-light images. This makes thermal imaging valuable in applications where lighting conditions are poor or where the target information is temperature-related rather than appearance-related.

In building inspection, thermal imaging can help identify heat loss, water leakage, insulation defects, cracks, and other hidden structural problems. In surveillance and autonomous systems, thermal cameras can operate at night or in low-light environments. In industrial monitoring, thermal images can reveal overheating components and abnormal operating conditions.

However, thermal imaging has several limitations. Thermal sensors often have lower spatial resolution than RGB sensors. Thermal images may also contain sensor noise, non-uniformity, low contrast, and blurred edges. These limitations motivate the development of thermal image enhancement and super-resolution methods.

2.2. Image Super-Resolution

Image super-resolution (SR) aims to reconstruct a high-resolution image from one or more low-resolution observations. Figure 2 illustrates the objective of image super-resolution. Starting from a low-resolution thermal image, the image is progressively reconstructed at higher spatial resolutions, enabling finer structural details to become visible.

Depending on the number of input images, SR methods can be broadly categorized into single-image super-resolution (SISR) and multi-frame super-resolution (MFSR). SISR reconstructs a high-resolution image from a single low-resolution input, whereas MFSR exploits information from multiple frames or images. Since the reconstruction setting considered in this work uses a single low-resolution thermal image as input, this work focuses on single-image super-resolution.

Figure 3 summarizes the taxonomy of image super-resolution methods reviewed in this paper. This work focuses on the single-image super-resolution branch.

As shown in Figure 3, the reviewed single-image SR methods include classical interpolation, CNN-based methods, GAN-based methods, transformer-based methods, state-space models, diffusion/autoregressive methods, and fusion-based approaches. The proposed framework later builds on this branch by adapting demosaicking algorithms to thermal upsampling and combining them with deep learning through adaptive fusion.

A common degradation model for single-image super-resolution is

$$I_{LR}=D(I_{HR};{\theta }_D),$$

(1)

where $I_{HR}$ is the original high-resolution image, $I_{LR}$ is the degraded low-resolution image, $D(·)$ is the degradation operator, and ${\theta }_D$ represents degradation parameters such as the blur kernel, scale factor, noise level, and downsampling operator.

The goal of single-image super-resolution is to learn or define a reconstruction function $R(·)$ such that

$${\widehat{I}}_{HR}=R(I_{LR};{\theta }_R),$$

(2)

where ${\widehat{I}}_{HR}$ is the reconstructed high-resolution image and ${\theta }_R$ denotes the parameters of the reconstruction method.

In many super-resolution benchmarks, $I_{LR}$ is generated by bicubic downsampling. Although this setting allows controlled evaluation, it does not fully represent real thermal degradation. Real infrared images may suffer from optical blur, motion blur, sensor noise, temperature drift, compression artifacts, and non-uniform degradation.

2.3. Challenges in Thermal Image Super-Resolution

Thermal SR is more challenging than conventional RGB SR for several reasons:

• Low contrast: Thermal images often contain small intensity differences, making edge and texture recovery difficult.
• Weak textures: Infrared images usually contain fewer high-frequency textures than RGB images.
• Sensor noise: Thermal cameras may introduce non-uniform noise and sensor-specific artifacts.
• Dataset scarcity: Large paired LR-HR thermal datasets are limited compared with RGB SR datasets.
• Real-world degradation: Real thermal degradation is often different from synthetic bicubic downsampling.
• Physical meaning: Thermal intensities may correspond to temperature-related information; therefore, hallucinated details can be misleading in inspection applications.

3. Datasets and Assessment Metrics

Datasets and evaluation metrics play a fundamental role in the development and comparison of image super-resolution algorithms. While RGB image super-resolution benefits from several standardized benchmarks, including Set5, Set14, BSD100, Urban100, Manga109, and DIV2K [7], thermal image super-resolution has comparatively fewer publicly available datasets specifically designed for this task. Consequently, many existing thermal datasets were originally developed for object detection, multispectral fusion, surveillance, or autonomous driving and are later adapted for super-resolution by synthetically generating low-resolution images through bicubic downsampling.

3.1. Thermal and Infrared Image Datasets

Table 2 summarizes representative datasets that are frequently used in thermal and infrared image processing research and have been employed, either directly or indirectly, for thermal image super-resolution.

Compared with RGB image super-resolution, thermal image super-resolution still lacks large standardized benchmarks. Consequently, most existing studies construct low-resolution inputs by bicubic downsampling of high-resolution thermal images. The PBVS Thermal Image Super-Resolution Challenge series represents one of the few benchmarks specifically developed for thermal SR. Earlier editions focused on thermal-only reconstruction, whereas recent editions additionally investigate RGB-guided thermal super-resolution, reflecting the growing interest in cross-spectral reconstruction methods.

The dataset used in this work, Thermal Imaging in Building Inspection, was originally developed for thermal building inspection rather than super-resolution. Therefore, low-resolution inputs are generated using bicubic downsampling with a scaling factor of $\times 2$, and the original thermal images are used as high-resolution ground-truth references.

3.2. Assessment Metrics

Thermal image super-resolution methods are commonly evaluated using full-reference image quality metrics when ground-truth high-resolution images are available. The most frequently reported metrics are Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). In addition, perceptual and efficiency metrics are often reported to provide a more comprehensive evaluation.

3.2.1. Peak Signal-to-Noise Ratio [7]

PSNR measures the pixel-wise fidelity between the reconstructed image and its ground-truth counterpart. It is defined as

$$\mathrm{PSNR}=10{log}_{10}\left({\displaystyle \frac{L^2}{\mathrm{MSE}}}\right),$$

(3)

where L denotes the maximum possible pixel value. The mean squared error (MSE) is given by

$$\mathrm{MSE}={\displaystyle \frac{1}{HW}}{\displaystyle \sum _{i=1}^H}{\displaystyle \sum _{j=1}^W}{\left(I_{HR}(i,j)-{\widehat{I}}_{HR}(i,j)\right)}^2.$$

(4)

Higher PSNR values indicate smaller reconstruction errors.

3.2.2. Structural Similarity Index Measure [7]

SSIM evaluates perceptual similarity by jointly considering luminance, contrast, and structural information:

$$\mathrm{SSIM}(x,y)={\displaystyle \frac{(2{\mu }_x{\mu }_y+C_1)(2{\sigma }_{xy}+C_2)}{({\mu }_x^2+{\mu }_y^2+C_1)({\sigma }_x^2+{\sigma }_y^2+C_2)}}.$$

(5)

Here, ${\mu }_x$ and ${\mu }_y$ denote the local means, ${\sigma }_x^2$ and ${\sigma }_y^2$ represent the local variances, ${\sigma }_{xy}$ is the covariance, and $C_1$ and $C_2$ are small constants introduced for numerical stability. SSIM values range from 0 to 1, where values closer to 1 indicate greater structural similarity.

3.2.3. Additional Evaluation Metrics

Although PSNR and SSIM remain the dominant evaluation metrics, they do not always correlate well with perceptual image quality. Therefore, recent studies increasingly report additional objective and perceptual metrics. Table 3 summarizes the most commonly used evaluation metrics, their categories, purposes, and preferred optimization directions.

For application-oriented thermal imaging, image quality metrics alone may not fully reflect practical performance. Therefore, many recent studies additionally evaluate downstream tasks such as object detection, semantic segmentation, or defect localization to determine whether the reconstructed thermal images improve real-world inspection performance.

4. Taxonomy of Thermal Image Super-Resolution Methods

Thermal image super-resolution (SR) methods can be broadly divided into classical, learning-based, and fusion-based approaches. Classical methods estimate unknown samples through fixed mathematical operations and do not require training data. Learning-based methods instead learn a nonlinear mapping from low-resolution (LR) images to high-resolution (HR) images and can be further classified according to their main architectural principle, including convolutional neural networks (CNNs), transformers, state-space models, and autoregressive models. Fusion-based methods combine the outputs or features of multiple reconstruction approaches to exploit their complementary characteristics.

Although several influential SR architectures were originally developed for visible-spectrum images, they are relevant to thermal SR because they are frequently adopted as benchmark models or adapted to single-channel infrared data. Therefore, this taxonomy distinguishes between general-image SR architectures used as comparison baselines and methods developed specifically for infrared or thermal imagery.

4.1. Classical Methods

Classical SR methods calculate the intensity of an unknown output pixel from the available LR samples using a predefined interpolation rule. They are deterministic, computationally inexpensive, and independent of training data. However, because they do not learn image-specific priors, their ability to reconstruct missing high-frequency information is limited.

4.1.1. Interpolation-Based Methods [17]

Nearest-neighbor interpolation assigns each output location the value of the closest input pixel. It is extremely fast but commonly produces block-like regions, jagged boundaries, and abrupt transitions.

Bicubic interpolation applies cubic convolution over a larger neighborhood, typically involving $4\times 4$ input samples. It generally preserves gradual intensity variations more effectively than bilinear interpolation. However, it still estimates new pixels only from nearby intensity values and cannot recover genuinely missing structures.

Lanczos interpolation approximates ideal sinc interpolation using a finite-support windowed sinc kernel. It can produce sharper boundaries than bilinear or bicubic interpolation, but oscillations around strong intensity transitions may lead to ringing artifacts.

These limitations are particularly important in thermal inspection images, where defects such as cracks may appear as thin, low-contrast structures. Excessive smoothing can remove or weaken such structures, while ringing can introduce artificial thermal boundaries.

4.1.2. Demosaicking-Derived Methods

Demosaicking algorithms were originally designed to estimate missing color components from sparsely sampled color filter array measurements. Although demosaicking and image super-resolution are distinct reconstruction problems, both require the estimation of unavailable samples from spatially incomplete observations. This similarity motivates the adaptation of demosaicking principles to thermal image upsampling.

Bilinear interpolation constitutes the simplest reconstruction strategy in this group. In conventional demosaicking, missing samples are estimated through local weighted averaging of neighboring observations. In this work, the same principle is reformulated for single-channel thermal images within a zero-insertion upsampling framework, where only the inserted spatial locations are reconstructed while the original thermal samples are preserved.

The Malvar–He–Cutler method improves basic bilinear demosaicking by applying larger-support linear filters together with Laplacian correction terms [18,19]. These correction terms use information from neighboring samples to reduce interpolation errors around edges and local intensity variations.

The Menon method performs directional interpolation and applies an a posteriori decision to determine the more reliable interpolation direction [20]. By distinguishing between horizontal and vertical structures, it reduces averaging across strong edges and can preserve direction-dependent image details more effectively than nondirectional interpolation.

The original Bilinear, Malvar–He–Cutler, and Menon formulations are associated with missing-sample reconstruction in demosaicking and cannot be transferred unchanged to the single-channel thermal SR setting considered here. Accordingly, all three are reformulated within the proposed zero-insertion upsampling procedure. Their mathematical definitions and processing steps are presented in Section 5.

Although interpolation- and demosaicking-derived methods are computationally efficient and require no training data, their reconstruction capability remains limited by fixed mathematical assumptions and local pixel relationships. Figure 4 summarizes their principal limitations.

4.2. Learning-Based Methods

Learning-based methods estimate an HR image using parameters learned from training examples. Compared with fixed interpolation, these methods can model complex nonlinear relationships between LR and HR images and can reconstruct stronger edges and finer structures. Their performance, however, depends on the quantity and representativeness of the training data and on the degradation model used to generate LR–HR pairs.

4.2.1. CNN-Based Methods

CNN-based SR methods extract hierarchical spatial features through learned convolutional filters [21,22,23,24,25]. Their local receptive fields, parameter sharing, and translation-equivariant processing make them suitable for image reconstruction.

General-image CNN baselines.

The Deep Recursive Residual Network (DRRN) combines recursive learning with local and global residual connections [26]. Recursive parameter sharing allows DRRN to construct a deep network while limiting the number of independent parameters. Residual learning facilitates the optimization of the deep recursive architecture. Although DRRN was designed for general-image SISR, it can be applied to single-channel thermal images and is therefore used as a comparison baseline in this study.

The Enhanced Deep Super-Resolution Network (EDSR) modifies conventional residual architectures by removing modules that are unnecessary for SR, particularly batch-normalization layers, and by increasing the capacity of the remaining residual blocks [27]. Removing batch normalization reduces memory consumption and avoids normalization of the image intensity range. EDSR achieved strong reconstruction performance on natural-image SR benchmarks and has subsequently become a widely used baseline.

The Information Multi-Distillation Network (IMDN) was proposed as a lightweight SR architecture [28]. Its Information Multi-Distillation Blocks progressively extract selected feature channels and aggregate hierarchical representations. A contrast-aware channel attention mechanism emphasizes informative channels during feature fusion. IMDN provides a favorable trade-off between reconstruction accuracy, parameter count, and computational cost, making it relevant to resource-constrained thermal imaging applications.

These networks were originally developed using visible-spectrum datasets and do not explicitly model infrared sensor characteristics. Their inclusion in thermal SR experiments should therefore be interpreted as an evaluation of their architectural transferability rather than as evidence that they are infrared-specific models.

Infrared-specific CNN and adversarial methods.

PSRGAN was proposed to address infrared SR under limited infrared training data through progressive reconstruction and transfer learning [29]. Its architecture contains a main path for infrared feature reconstruction and a branch path used to transfer information learned from visible images. The main path employs depth-wise residual blocks, whereas the branch path uses shallow lightweight distillation residual blocks. A multistage transfer-learning strategy is used to reduce the discrepancy between visible and infrared feature spaces.

Because PSRGAN incorporates adversarial learning, it can produce perceptually sharper reconstructions than methods optimized only with pixel-wise losses. However, adversarially generated details are not guaranteed to represent physically observed thermal structures. This limitation is important in inspection and measurement applications, where visually plausible patterns may be misinterpreted as actual defects.

RLKA-Net, the Recurrent Large Kernel Attention Network, was developed specifically for efficient single infrared image SR [30]. It employs recurrent learning units to reuse network parameters and progressively refine feature representations. Large-kernel attention enlarges the effective receptive field and enables the model to capture spatial information at different granularities. An attention gate is used to regulate feature propagation and reduce unnecessary parameters.

The recurrence in RLKA-Net represents repeated internal feature refinement and does not imply that the model processes a thermal video sequence. RLKA-Net is a single-image method. Its relatively lightweight design makes it suitable for the proposed fusion framework. Nevertheless, like other supervised networks, its performance depends on the correspondence between the training degradation and the degradation present in real thermal images.

CNN-based super-resolution methods substantially improve reconstruction quality by learning hierarchical image features. Nevertheless, their performance often depends on the availability of representative training data, and their ability to generalize across sensors and imaging conditions may remain limited. Figure 5 summarizes the common limitations of this method family.

4.2.2. Transformer-Based Methods

Transformer-based SR methods aim to model broader spatial dependencies than conventional small-kernel CNNs [31,32,33]. Their attention or attention-inspired operations allow distant image regions to contribute to the reconstruction of a local area.

General-image transformer baseline.

SwinIR is an image-restoration architecture based on the Swin Transformer [34]. It consists of shallow feature extraction, deep feature extraction, and image reconstruction stages. Its deep feature extractor contains Residual Swin Transformer Blocks, each of which combines several Swin Transformer layers with a residual connection. Shifted-window self-attention enables information exchange between neighboring windows while avoiding the full computational cost of global self-attention.

SwinIR was evaluated on classical, lightweight, and real-world image SR, as well as denoising and JPEG artifact reduction. Although it was not designed specifically for thermal imagery, its ability to model spatial dependencies makes it an important transformer-based comparison method.

Infrared-specific transformer model.

LKFormer is a large-kernel transformer-style network developed for infrared image SR [35]. Instead of conventional self-attention, it uses a Large Kernel Residual Depth-wise Convolutional Attention module. Large-kernel depth-wise convolutions model nonlocal spatial relationships while maintaining linear computational complexity with respect to the number of pixels.

LKFormer also introduces a Gated-Pixel Feed-Forward Network to control feature flow while retaining spatially localized information. This design is intended to address common infrared-image characteristics, including relatively homogeneous regions, sparse textures, and a limited gradient range.

Although LKFormer avoids the quadratic complexity of global self-attention, its large-kernel processing remains more complex than basic interpolation and small lightweight CNNs. Its practical performance also depends on whether the training degradation accurately represents the optical blur, sensor noise, and preprocessing of the target thermal camera.

4.2.3. State-Space Methods

State-space architectures have recently been introduced into image restoration as an alternative to convolutional processing and self-attention. Selective state-space models can represent long-range dependencies with computational complexity that grows approximately linearly with sequence length.

IRSRMamba applies Mamba-based state-space modeling to infrared image SR [36]. The method combines state-space blocks with wavelet-transform feature modulation. Wavelet decomposition separates low-frequency structural information from high-frequency detail components, allowing the network to adaptively process information at different spatial frequencies.

The state-space blocks model long-range contextual relationships, whereas the wavelet components provide multi-scale local information. IRSRMamba also introduces a semantic-consistency objective intended to reduce fragmented contextual representations produced during block-wise processing.

This combination is well suited to infrared images, which often contain broad homogeneous regions together with weak but diagnostically important boundaries. However, the method remains dependent on supervised training and on the assumed LR-generation process. Furthermore, state-space image models are newer and currently have less extensive cross-dataset and cross-sensor validation than established CNN architectures.

4.2.4. Autoregressive Methods

Autoregressive SR methods reconstruct image information progressively rather than predicting the complete HR output through a single feed-forward pass. They can model dependencies between reconstruction stages or image representations but generally require sequential computation.

Real-IISR was introduced for real-world infrared image SR and is accompanied by the FLIR-IISR benchmark [37]. In contrast to studies that construct LR images using only bicubic downsampling, FLIR-IISR contains paired LR and HR infrared observations acquired using focus variation and motion-induced blur.

Real-IISR performs scale-by-scale reconstruction using thermal-structural guided visual autoregression. Its Thermal-Structural Guidance module encodes thermal-distribution and structural information to account for the possible misalignment between thermal radiation patterns and visible object edges. A Condition-Adaptive Codebook adjusts discrete image representations according to degradation-aware thermal priors.

The method additionally introduces a Thermal Order Consistency Loss. Rather than enforcing exact temperature values, this loss preserves the relative ordering between image intensities, thereby accounting for spatial misalignment and thermal drift in real image pairs.

Real-IISR directly addresses several limitations of synthetic infrared SR benchmarks. However, its autoregressive reconstruction is sequential and is therefore likely to require greater inference time than a compact single-pass CNN. Because both the framework and FLIR-IISR dataset are recent, broader independent evaluation across thermal cameras, spectral ranges, environments, and acquisition conditions is still required.

Recent learning-based super-resolution methods use Transformer architectures, state-space models, and autoregressive reconstruction to improve long-range feature modeling and structural recovery. However, these methods may introduce greater computational complexity, slower inference, and continued dependence on representative training data. Their common limitations are summarized in Figure 6.

4.2.5. Hypercomplex-Based Methods

While conventional convolutional neural networks process multi-channel images (such as RGB or multi-spectral visible-thermal data) by treating the spectral bands as separate, independent feature maps, hypercomplex neural networks represent multi-channel observations as unified multidimensional entities. In particular, Quaternion Neural Networks (QNNs) extend standard real-valued operations to the four-dimensional quaternion space $\mathbb{H}$ [38,39,40,41,42,43].

A quaternion $q\in \mathbb{H}$ consists of a real part and three imaginary parts:

$$q=q_r+q_i\mathbf{i}+q_j\mathbf{j}+q_k\mathbf{k},$$

(6)

where $\mathbf{i}$, $\mathbf{j}$, and $\mathbf{k}$ are imaginary units satisfying the fundamental multiplication rules:

$${\mathbf{i}}^2={\mathbf{j}}^2={\mathbf{k}}^2=\mathbf{i}\mathbf{j}\mathbf{k}=-1.$$

(7)

For a four-channel image—such as cross-spectral RGB-Thermal (RGB-T) data—the thermal channel $I_T$ and the visible color channels $I_R$, $I_G$, and $I_B$ can be naturally mapped to the real and imaginary components of a quaternion matrix:

$$q=I_T+I_R\mathbf{i}+I_G\mathbf{j}+I_B\mathbf{k}.$$

(8)

Similarly, standard three-channel RGB color images can be represented by setting the real part to the grayscale version of the image $I_{\mathrm{gray}}$, and the imaginary parts to the red, green, and blue components:

$$q=I_{\mathrm{gray}}+I_R\mathbf{i}+I_G\mathbf{j}+I_B\mathbf{k}.$$

(9)

This formulation preserves inter-channel spectral correlations and prevents independent channel degradation during feature learning.

The core operation in these networks is the quaternion convolution (QConv) [44,45,46]. Given a quaternion input $x=x_r+x_i\mathbf{i}+x_j\mathbf{j}+x_k\mathbf{k}$ and a quaternion filter weight $W=w_r+w_i\mathbf{i}+w_j\mathbf{j}+w_k\mathbf{k}$, the convolution is defined using the Hamilton product:

$$y=W\otimes x.$$

(10)

By expanding the Hamilton product, the forward convolution can be represented in matrix form as:

$$\left[\begin{array}{c}{y}_r\\ y_i\\ y_j\\ y_k\end{array}\right]=\left[\begin{array}{cccc}{w}_r& -w_i& -w_j& -w_k\\ w_i& w_r& -w_k& w_j\\ w_j& w_k& w_r& -w_i\\ w_k& -w_j& w_i& w_r\end{array}\right]*\left[\begin{array}{c}{x}_r\\ x_i\\ x_j\\ x_k\end{array}\right].$$

(11)

where * denotes the spatial convolution operation. The weight-sharing structure in the Hamilton matrix forces cross-spectral interactions among the channels while reducing the number of independent parameters by $75\%$ (a $4\times$ reduction) compared to a real-valued convolution layer with the same input-output channel configuration.

In the context of super-resolution, the QSRNet framework [46] demonstrates the advantages of hypercomplex representations by adapting real-valued architectures to the quaternion domain. By replacing standard convolutions with quaternion convolutions, QSRNet processes structural features, gradient-based soft edges, and multi-scale details as unified multidimensional entities. By exploiting hypercomplex algebra, the model achieves competitive restoration fidelity with an approximate $75\%$ reduction in network parameters, making it highly suitable for computationally constrained edge-computing devices in thermal and multispectral inspection systems.

4.3. Fusion-Based Methods

Classical and learning-based SR methods have different reconstruction characteristics, and no individual approach performs optimally in all image regions. Bilinear interpolation generally produces smooth and stable results but may weaken narrow boundaries and fine details. Larger-support Malvar–He–Cutler filtering can introduce additional correction information, whereas Menon interpolation can better preserve direction-dependent structures. Learning-based methods such as RLKA-Net can reconstruct richer features through learned representations, although their performance remains dependent on the characteristics of the training data.

Fusion-based methods aim to exploit these complementary properties by combining the outputs or feature representations of multiple reconstruction approaches. A simple fixed average assigns the same importance to every input method and therefore cannot adapt to spatial variations within an image. Adaptive fusion instead allows the contribution of each reconstruction method to vary according to the local image content, such as homogeneous regions, object boundaries, and fine structural details.

In this work, a fusion-based model named FusionNet is proposed to combine the outputs of Bilinear interpolation, the adapted Malvar–He–Cutler method, the adapted Menon method, and RLKA-Net. The model learns spatially varying contributions for the four reconstructed images and produces a single fused high-resolution output. This approach is intended to preserve the stable reconstruction of classical methods while benefiting from the richer details recovered by the learning-based model.

The architecture, mathematical formulation, training procedure, and implementation details of FusionNet are presented in Section 5.

4.4. Comparison of Representative Methods

Table 4 summarizes the principal characteristics of the methods discussed in this section. DRRN, EDSR, IMDN, and SwinIR are identified as general-image methods because they were not originally designed for infrared imaging. They are nevertheless included because they provide representative CNN and transformer baselines in the experimental comparison.

4.5. Discussion

The development of thermal image SR demonstrates a gradual transition from fixed local interpolation to data-driven reconstruction with increasingly broad spatial modeling. Classical interpolation remains attractive because it is transparent, inexpensive, and does not require a training dataset. Demosaicking-derived methods introduce larger-support and directional estimation but must be modified before they can be applied to single-channel thermal upsampling.

CNN-based methods learn stronger nonlinear reconstruction mappings. DRRN, EDSR, and IMDN represent recursive, high-capacity residual, and lightweight information-distillation designs, respectively. Although these architectures were developed for visible-spectrum data, their inclusion makes it possible to evaluate whether general SR principles transfer effectively to thermal imagery. PSRGAN and RLKA-Net introduce infrared-specific transfer learning, adversarial learning, recurrent refinement, and large-kernel attention.

Transformer and state-space methods further enlarge the modeled spatial context. SwinIR uses shifted-window attention, while LKFormer replaces conventional self-attention with large-kernel depth-wise processing. IRSRMamba combines long-range state-space modeling with wavelet-domain modulation. Real-IISR extends the field toward real-world acquisition by introducing paired data with optical and motion degradations and by preserving relative thermal intensity ordering.

Despite this progress, supervised SR models remain sensitive to dataset bias and degradation mismatch. A network trained using bicubic downsampling may perform well on similarly generated test data while generalizing poorly to optical blur, motion, nonuniform sensor noise, fixed-pattern noise, or camera-specific image processing. Generative models require additional caution because perceptually sharp details are not necessarily physically accurate.

The proposed FusionNet follows a complementary direction. Instead of assuming that one reconstruction mechanism is optimal for every image region, it combines stable interpolation, correction-based filtering, directional estimation, and learned infrared reconstruction. The spatially varying fusion weights allow different methods to dominate in homogeneous regions, along structural boundaries, and around fine thermal details.

5. Proposed Demosaicking-Inspired Thermal SR Framework

This section presents the proposed thermal image super-resolution framework. The method reformulates image upsampling as a missing-sample reconstruction problem and adapts three interpolation strategies—Bilinear, Malvar–He–Cutler-style, and Menon directional interpolation—to thermal image super-resolution. Their outputs are subsequently combined with the reconstruction produced by RLKA-Net using the proposed FusionNet.

5.1. Demosaicking-Inspired Thermal Upsampling

The proposed approach is motivated by the similarity between demosaicking and image upsampling. In both cases, unavailable pixel values must be estimated from observed neighboring samples. For a low-resolution thermal image

$$g\in {\mathbb{R}}^{H\times W},$$

(12)

a zero-inserted image

$$u\in {\mathbb{R}}^{2H\times 2W}$$

(13)

is constructed as

$$u(2i,2j)=g(i,j),0\le i<H,0\le j<W.$$

(14)

All remaining positions are initialized to zero and form the set of unknown samples ${\mathrm{\Omega }}_0$. During reconstruction, the original thermal samples are preserved and only the inserted locations are estimated.

5.1.1. Bilinear Interpolation Adaptation

The first reconstruction branch adapts Bilinear interpolation to the zero-inserted thermal image. The interpolation kernel is

$$H_{\mathrm{bil}}={\displaystyle \frac{1}{4}}\left[\begin{array}{ccc}1& 2& 1\\ 2& 4& 2\\ 1& 2& 1\end{array}\right].$$

(15)

Algorithm 1 summarizes the complete procedure.

Algorithm 1: Demosaicking-Inspired Bilinear Thermal Upsampling

Input: Low-resolution thermal image $g\in {\mathbb{R}}^{H\times W}$ Output: Upsampled thermal image $I_{\mathrm{bil}}\in {\mathbb{R}}^{2H\times 2W}$ Construct the zero-inserted image u using Equation (14) Define the set of inserted samples ${\mathrm{\Omega }}_0$ Apply the Bilinear kernel: $$\tilde{u}=u*H_{\mathrm{bil}}$$

This procedure fills the missing positions using local weighted averages while preserving the original thermal samples.

5.1.2. Malvar–He–Cutler-Style Adaptation

The Malvar–He–Cutler method was originally developed for color demosaicking. Since thermal images are single-channel, its cross-channel correction mechanism is not applicable. Instead, the proposed adaptation retains the larger-support filtering principle and applies it to the zero-inserted thermal image.

The first interpolation filter is

$$K_{\mathrm{diag}}={\displaystyle \frac{1}{8}}\left[\begin{array}{ccccc}0& 0& -{\displaystyle \frac{3}{2}}& 0& 0\\ 0& 2& 0& 2& 0\\ -{\displaystyle \frac{3}{2}}& 0& 6& 0& -{\displaystyle \frac{3}{2}}\\ 0& 2& 0& 2& 0\\ 0& 0& -{\displaystyle \frac{3}{2}}& 0& 0\end{array}\right]$$

(16)

and the refinement filter is

$$K_{\mathrm{main}}={\displaystyle \frac{1}{8}}\left[\begin{array}{ccccc}0& 0& -1& 0& 0\\ 0& 0& 2& 0& 0\\ -1& 2& 4& 2& -1\\ 0& 0& 2& 0& 0\\ 0& 0& -1& 0& 0\end{array}\right]$$

(17)

The proposed adaptation is summarized in Algorithm 2.

Algorithm 2: Malvar–He–Cutler-Style Thermal Upsampling

Input: Low-resolution thermal image $g\in {\mathbb{R}}^{H\times W}$ Output: Upsampled thermal image $I_{\mathrm{mal}}\in {\mathbb{R}}^{2H\times 2W}$ Construct zero-inserted image u using Equation (14) Define inserted sample locations ${\mathrm{\Omega }}_0$ Compute the first-stage filtered image: $$T_{\mathrm{diag}}=u*K_{\mathrm{diag}}$$

    The wider spatial support allows the method to incorporate more neighboring information than simple Bilinear interpolation.

5.1.3. Menon Directional Interpolation Adaptation

The Menon adaptation exploits local edge direction by generating horizontal and vertical interpolation candidates and selecting the direction with lower local variation.

The one-dimensional interpolation filters are

$$h_0=\left[0,{\displaystyle \frac{1}{2}},0,{\displaystyle \frac{1}{2}},0\right],h_1=\left[-{\displaystyle \frac{1}{4}},0,{\displaystyle \frac{1}{2}},0,-{\displaystyle \frac{1}{4}}\right]$$

(18)

The horizontal and vertical candidates are

$$H={(u*h_0)}_H+{(u*h_1)}_H$$

(19)

$$V={(u*h_0)}_V+{(u*h_1)}_V$$

(20)

and the corresponding local variations are

$$D_H=\left|H-H(·,·+2)\right|$$

(21)

$$D_V=\left|V-V(·+2,·)\right|$$

(22)

The decision mask is

$$k=\left[\begin{array}{ccccc}0& 0& 1& 0& 1\\ 0& 0& 0& 1& 0\\ 0& 0& 3& 0& 3\\ 0& 0& 0& 1& 0\\ 0& 0& 1& 0& 1\end{array}\right]$$

(23)

with directional costs

$$d_H=D_H*k,d_V=D_V*k^T$$

(24)

Algorithm 3 summarizes the adaptation.

Algorithm 3: Menon Directional Thermal Upsampling

Input: Low-resolution thermal image $g\in {\mathbb{R}}^{H\times W}$ Output: Upsampled thermal image $I_{\mathrm{men}}\in {\mathbb{R}}^{2H\times 2W}$ Construct zero-inserted image u using Equation (14) Define inserted sample locations ${\mathrm{\Omega }}_0$ Compute horizontal candidate H Compute vertical candidate V Compute directional variations $D_H$ and $D_V$ Aggregate the variations: $$d_H=D_H*k,d_V=D_V*k^T$$

Unlike non-directional interpolation, this procedure allows reconstruction to adapt to local horizontal and vertical structures.

5.2. RLKA-Net Reconstruction

RLKA-Net is used as the learning-based reconstruction branch of the proposed framework. It was designed for infrared image super-resolution and combines recurrent feature refinement with large-kernel attention. In this work, its super-resolved output, denoted by $I_{\mathrm{RLKA}}$, is used directly as the fourth input to FusionNet.

The architecture of RLKA-Net is shown in Figure 7. Since RLKA-Net is an existing method, only its role within the proposed framework is described here; further architectural details are available in the original work [30].

5.3. Proposed FusionNet

The three demosaicking-inspired reconstructions and the RLKA-Net output exhibit complementary reconstruction characteristics. FusionNet is therefore proposed to combine them adaptively using spatially varying learned weights. Figure 8 illustrates the proposed architecture.

Let

$$I_1=I_{\mathrm{bil}},I_2=I_{\mathrm{mal}},I_3=I_{\mathrm{men}},I_4=I_{\mathrm{RLKA}}$$

(25)

denote the four reconstruction candidates. They are concatenated along the channel dimension as

$$X=Concat(I_1,I_2,I_3,I_4)\in {\mathbb{R}}^{4\times H\times W}.$$

(26)

FusionNet consists of three $3\times 3$ convolutional layers with ReLU activations followed by a four-channel weight-prediction layer, as summarized in Table 5.

The predicted weight maps are normalized at each spatial position using Softmax:

$$w_i(x,y)={\displaystyle \frac{exp\left(z_i(x,y)\right)}{{\displaystyle {\displaystyle \sum _{j=1}^4}exp\left(z_j(x,y)\right)}}},i=1,\dots ,4,$$

(27)

where $z_i(x,y)$ denotes the unnormalized score for the ith reconstruction. Consequently,

$${\displaystyle \sum _{i=1}^4}{w}_i(x,y)=1.$$

(28)

The final super-resolved image is obtained by pixel-wise weighted fusion:

$$I_{\mathrm{fused}}(x,y)={\displaystyle \sum _{i=1}^4}{w}_i(x,y)I_i(x,y).$$

(29)

The complete procedure is summarized in Algorithm 4.

The learned spatial weights allow the contribution of each reconstruction method to vary across the image, enabling FusionNet to adapt the combination of classical and learning-based reconstructions to local image content.

5.4. Training Objective

FusionNet is trained using the Charbonnier loss,

$${\mathcal{L}}_{\mathrm{char}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N}\sqrt{{\left({\widehat{x}}_n-y_n\right)}^2+ϵ},$$

(30)

where ${\widehat{x}}_n$ and $y_n$ denote the reconstructed and ground-truth pixel values, respectively, and $ϵ={10}^{-6}$ is used for numerical stability.

6. Experimental Evaluation

6.1. Dataset

Experiments are conducted on the Thermal Imaging in Building Inspection dataset [3]. The dataset contains 306 thermal images and is divided into 277 training images, 14 validation images, and 15 test images. Since paired LR-HR images are not directly provided, LR inputs are generated by bicubic downsampling with scale factor $\times 2$. The original thermal images are used as HR ground truth.

6.2. Training Details

RLKA-Net is fine-tuned on the thermal building-inspection dataset. FusionNet is trained using the outputs of Bilinear interpolation, Malvar-style interpolation, Menon-style interpolation, and RLKA-Net. The fusion network is optimized using the Adam optimizer with an initial learning rate of ${10}^{-4}$. Training is performed using image patches of size $128\times 128$ and batch size 8.

Algorithm 4: Proposed FusionNet Adaptive Reconstruction

Input: $I_{\mathrm{bil}}$, $I_{\mathrm{mal}}$, $I_{\mathrm{men}}$, and $I_{\mathrm{RLKA}}$ Output: Final super-resolved image $I_{\mathrm{fused}}$ Concatenate the four reconstructions: $$X\leftarrow Concat(I_{\mathrm{bil}},I_{\mathrm{mal}},I_{\mathrm{men}},I_{\mathrm{RLKA}})$$ Extract features using three $3\times 3$ convolutional layers with ReLU activations Predict four spatial weight maps Normalize the predicted weights using Softmax according to Equation (27) Compute the pixel-wise weighted reconstruction using Equation (29) return$I_{\mathrm{fused}}$

6.3. Hypercomplex Analysis

To evaluate the potential of hypercomplex representations for computationally constrained edge devices, we draw inspiration from quaternion-based frameworks such as QSRNet [46] and replace the real-valued convolutions in FusionNet with quaternion convolutions. Because quaternion convolutions intrinsically reduce the parameter count by approximately 75% compared to real-valued counterparts (while maintaining the same number of output channels), we evaluate two quaternion-based variants: QFusionNet, which matches the channel dimensions of the baseline, and QFusionNet⁺, which increases the channel capacity to try to match the original parameter count.

Table 6 presents the quantitative comparison. The baseline FusionNet contains 20.8K parameters. QFusionNet achieves competitive performance (37.6149 dB PSNR and 0.9742 SSIM) despite utilizing only 5.3K parameters. Furthermore, the scaled QFusionNet⁺ (19.8K parameters) outperforms the real-valued baseline, achieving 37.7958 dB PSNR and 0.9749 SSIM. These results demonstrate that the structural priors introduced by quaternion networks can lead to improved thermal image super-resolution performance given a similar parameter budget.

6.4. Compared Methods

The taxonomy presented in Section 4 includes a broad range of thermal image super-resolution methods. Ideally, all reviewed methods would be evaluated under identical experimental conditions. In practice, this is not always feasible because some methods cannot be fairly reproduced or compared within the scope of this work.

The experimental evaluation therefore includes representative methods from each major category:

• Classical interpolation: Nearest Neighbor, Bicubic, and Lanczos;
• Demosaicking-inspired methods: the proposed Bilinear, Malvar–He–Cutler-style, and Menon adaptations;
• CNN-based methods: DRRN, IMDN, EDSR, PSRGAN, and RLKA-Net;
• Transformer-based methods: SwinIR;
• State-space models: IRSRMamba;
• Proposed method: FusionNet, QFusionNet, and QFusionNet⁺.

LKFormer is omitted because the authors have not officially released pretrained weights, making a reproducible evaluation difficult. Real-IISR is not included in the quantitative comparison because it targets real-world infrared super-resolution using a degradation model and paired real LR–HR benchmark that differ substantially from the synthetic bicubic $\times 2$ protocol adopted in this study. Its autoregressive reconstruction and degradation-aware preprocessing therefore do not constitute a directly matched experimental setting.

6.5. Quantitative Results

Table 7 reports the average quantitative performance on the thermal building-inspection test set using PSNR, SSIM, RMSE, LPIPS, and NIQE.

Among the conventional interpolation baselines, Lanczos achieves the highest PSNR, while Nearest Neighbor provides the highest SSIM and the lowest RMSE and LPIPS. Bicubic obtains the lowest NIQE within this group. Among the demosaicking-derived adaptations, Menon-style reconstruction achieves the highest PSNR, lowest RMSE, and lowest NIQE, whereas the Bilinear adaptation provides the highest SSIM and lowest LPIPS.

The learning-based methods generally provide stronger full-reference reconstruction quality than the classical and demosaicking-derived approaches. Among the individual learned models, RLKA-Net achieves the highest PSNR and SSIM and the lowest RMSE and LPIPS. However, PSRGAN obtains the lowest NIQE, showing that rankings based on no-reference perceptual quality do not necessarily follow the same trend as full-reference fidelity measures.

The proposed FusionNet further improves upon RLKA-Net, achieving a PSNR of 37.6891 dB and an SSIM of 0.9746. The quaternion-based variants demonstrate two complementary advantages. QFusionNet reduces the parameter count from 20.8K to 5.3K while maintaining comparable reconstruction quality, achieving 37.6149 dB PSNR and 0.9742 SSIM. In contrast, QFusionNet⁺ uses a parameter budget comparable to FusionNet (19.8K) and achieves the highest PSNR and SSIM among the evaluated methods, reaching 37.7958 dB and 0.9749, respectively. These results indicate that quaternion-based representations can substantially improve parameter efficiency while preserving reconstruction fidelity and, when additional channel capacity is used, can slightly improve reconstruction performance over the real-valued FusionNet. PSRGAN nevertheless retains the best NIQE, highlighting the difference between distortion-based, learned perceptual, and no-reference quality measures.

6.6. Qualitative Discussion

Figure 9 and Figure 10 present a qualitative comparison of the evaluated classical interpolation, demosaicking-inspired, learning-based, and fusion-based super-resolution methods on representative thermal images. The full images on the left indicate the selected regions of interest, while the enlarged crops on the right facilitate a detailed comparison of local structures, edges, textures, and thermal details.

Among the classical interpolation methods, Nearest Neighbor preserves sharp transitions but introduces noticeable aliasing, whereas Bilinear, Bicubic, and Lanczos produce smoother reconstructions with progressively improved edge continuity. The proposed demosaicking-inspired methods further enhance local structure preservation, with the Menon adaptation generally producing sharper directional features than the Bilinear and Malvar-style variants.

The learning-based methods consistently outperform the interpolation-based approaches by recovering clearer structures and more accurate thermal boundaries. Among the individual learning-based SR models shown in Figure 9 and Figure 10, RLKA-Net provides the strongest reconstruction. FusionNet further combines the complementary information provided by the interpolation-based branches and RLKA-Net, producing sharp edges, well-preserved thermal structures, and fewer visible reconstruction artifacts.

7. Open Challenges and Future Directions

7.1. Limited Thermal SR Datasets

A major limitation in thermal SR is the lack of large paired LR–HR datasets. Many existing thermal datasets were originally developed for detection, fusion, surveillance, or autonomous driving rather than super-resolution. Although synthetic bicubic downsampling enables controlled evaluation, it does not fully represent the degradation produced by real thermal imaging systems.

7.2. Real-World Degradation Modeling

Many thermal SR studies rely on simplified degradation models, particularly bicubic downsampling. In practice, thermal images may be affected by optical blur, motion blur, sensor noise, non-uniformity, temperature drift, compression, and camera-specific processing. Future research should therefore focus on realistic degradation models and paired real-world thermal LR–HR datasets.

7.3. Cross-Spectral Guided Super-Resolution

Recent PBVS challenge settings demonstrate increasing interest in RGB-guided thermal super-resolution. Since RGB images generally contain sharper boundaries and richer spatial textures, they can provide useful structural guidance. However, cross-spectral reconstruction requires accurate registration and careful fusion because visible-spectrum structures do not always correspond directly to thermal patterns.

Furthermore, standard cross-spectral fusion approaches treat visible and thermal channels as separate, independent feature maps, which can lead to feature degradation and loss of spatial-spectral consistency. A promising future direction to address this is the adoption of hypercomplex representations, such as Quaternion Super-Resolution (QSR) [47]. By encoding the thermal band along with the three visible channels (red, green, blue) as a single quaternion entity (e.g., $q=I_T+I_R\mathbf{i}+I_G\mathbf{j}+I_B\mathbf{k}$), hypercomplex networks can leverage the Hamilton product to model cross-spectral correlations directly within the algebraic structure, preserving the boundaries between color and thermal signals without treating them as independent feature maps.

7.4. Efficient Models for Practical Deployment

Thermal cameras are frequently deployed in embedded, mobile, or real-time systems. Therefore, reconstruction accuracy should be balanced with computational cost, memory consumption, and inference speed. Lightweight CNNs, efficient transformers, state-space models, pruning, quantization, and knowledge distillation represent promising directions for practical thermal SR.

7.5. Physical and Thermal Consistency

Thermal image intensities encode temperature-related radiometric information. Consequently, super-resolution methods should avoid generating visually plausible but physically misleading structures. Future methods should better preserve thermal ordering, temperature-related relationships, structural boundaries, and defect-relevant information.

7.6. Uncertainty-Aware Fusion

FusionNet predicts pixel-wise softmax weights for combining multiple reconstruction candidates. Future fusion methods could additionally incorporate uncertainty estimation, confidence-aware weighting, attention mechanisms, or iterative refinement. Such strategies may improve robustness in regions where the individual reconstruction methods provide conflicting estimates.

7.7. Evaluation Beyond PSNR and SSIM

PSNR and SSIM remain widely used, but they do not fully characterize perceptual quality or practical thermal-image fidelity. Future evaluation should combine full-reference, perceptual, no-reference, efficiency, and task-level measures, while also considering whether improvements in reconstructed images benefit downstream applications such as defect detection, segmentation, or inspection.

8. Conclusions

This paper reviewed thermal and infrared image super-resolution from the perspectives of datasets, evaluation metrics, reconstruction methods, and open research challenges. The reviewed approaches include conventional interpolation, demosaicking-derived reconstruction, CNNs, GANs, transformers, state-space models, diffusion and autoregressive methods, and fusion-based approaches. Thermal SR remains challenging because of low spatial resolution, weak textures, sensor-dependent degradation, limited paired datasets, and the need to preserve physically meaningful thermal information.

In addition to the review, this paper introduced a demosaicking-inspired thermal super-resolution framework. Bilinear, Malvar–He–Cutler-style, and Menon directional reconstruction were reformulated for single-channel thermal upsampling using zero insertion. Their outputs were combined with RLKA-Net using the proposed FusionNet, which learns spatially adaptive pixel-wise fusion weights.

Experiments on a thermal building-inspection dataset showed that FusionNet achieved 37.6891 dB PSNR and 0.9746 SSIM, outperforming the individual reconstruction methods used in the fusion framework. The hypercomplex analysis further demonstrated two complementary advantages of quaternion-based fusion. QFusionNet reduced the parameter count from 20.8K to 5.3K while maintaining comparable reconstruction quality, whereas QFusionNet⁺ achieved the best overall PSNR, SSIM, RMSE, and LPIPS among the evaluated methods, reaching 37.7958 dB PSNR and 0.9749 SSIM. PSRGAN achieved the lowest NIQE, illustrating that full-reference and no-reference quality measures may rank reconstruction methods differently. These results highlight the potential of quaternion-based fusion for both parameter-efficient and high-fidelity thermal super-resolution.