Employing Wavelets Based Upsampling to Improve U-Net Super-Resolution

1. Introduction

Artificial Intelligence (AI) is rapidly transforming the world and significantly improving the processing capabilities of computer vision systems. Over the last few years, these technologies have matured enough to deploy in applications spanning a wide range of areas; including vehicle detection [1], real-time object detection/tracking in Unmanned Aerial Vehicles (UAVs) [2], pathological image segmentation, and knowledge distillation [3]. Many of these applications rely on Super-resolution (SR) whereby a high-resolution (HR) image is reconstructed from low-resolution (LR) input(s) [4] to improve image clarity, identify ground targets, and extract fine-grained visual details that are lacking in original LR inputs [5,6]. SR can be broadly categorized into three major types: single-image SR (SISR), video SR, and multi-image SR. Usually, higher-quality reconstruction is required in scenarios where only limited LR data are available. For this reason, SISR has become one of the most studied research topics in computer vision and image processing [7].

In recent years, U-Net [8] has emerged one of the popular SR tool. It was originally conceived for biomedical image segmentation and has an encoder-decoder architecture that employs convolutions. The underlying idea was to use skip connections in order to compensate for the fine spatial information lost during the downsampling phase. Even with limited annotated data, the model achieved enviable performance using data augmentation for biomedical segmentation.

Building upon this foundation, the standard U-Net decoder relies on transpose convolution, pixel-shuffle, and bilinear interpolation for upsampling. However, these operations function entirely in the pixel domain and lack explicit frequency awareness, making it difficult to recover high-frequency details such as edges and textures and other fine details. This may result in information loss during down-sampling and reduce overall reconstruction quality.

To address the above limitation, we propose a frequency-aware wavelet-based method to improve U-Net in which the standard encoder down-sampling and decoder up-sampling operations are replaced with Discrete Wavelet Transform (DWT) and Inverse Discrete Wavelet Transform (IDWT), respectively. Rather than learning to up-sample pixel values directly, the network is trained to predict the missing high-frequency wavelet sub-bands, namely LH, HL, and HH, from the available LR input, assuming it to be the low-frequency LL sub-band. The final reconstruction is performed using a mathematically exact IDWT, a lossless operation rather than a learned approximation, thereby preserving spatial and structural fidelity. This design achieves improved Peak Signal-to-Noise Ratio (PSNR) and The Structural Similarity Index (SSIM) for $\times 2$ SR across all tested baselines, including pixel-shuffle, transposed convolution, bilinear and bi-cubic interpolation.

The main contributions of this work are summarized as follows:

• We propose a wavelet-based U-Net framework that integrates DWT and IDWT as frequency-aware replacements for the standard encoder-decoder operations, enabling lossless high-frequency detail preservation for SISR reconstruction.
• We introduce a flexible learned feature split concept that constrains the network to predict missing high-frequency wavelet sub-bands (LH, HL, HH) from the spatial domain feature map; combined with mathematically exact IDWT reconstruction, it reduces the learning burden while achieving superior PSNR and SSIM over all baselines under identical training conditions.
• Extensive experiments on multiple standard SR benchmarks, including Set5, Set14 and BSD100,ICDAR2003. The results also demonstrate that our method competes with state-of-the-art SR techniques while effectively retaining high-frequency details, edges, textures, and structural features.

The rest of the paper is arranged as follows. Section 2 gives a brief overview of the concepts necessary to understand the article. The literature survey is presented in Section 3, while Section 4 presents our methodology in detail. The ensued results and their analyses are presented in Section 5 and Section 6 concludes the paper.

2. Background

The fundamental operation in SISR is the estimation of missing pixel values in an HR grid corresponding to the input LR image. Traditionally, the spatial resolution was realized through a predefined and a weakly constrained mathematical mapping that risks to miss the extremely high frequency information mapped to its HR counterpart. Over the years, SR techniques have evolved from simple mathematical interpolations to very sophisticated DL approaches, thanks to the advent of Generative AI.

2.1. Upsampling in SISR

Upsampling an image during SISR can be broadly classified as interpolation oriented and DL based.

2.1.1. Interpolation Based Upsampling

Here are some of the well-known interpolation methods that are also useful for other geometric transformation tasks such as rotation [9][10]:

• Nearest Neighbor Interpolation copies the nearest pixel values. This simple interpolation approach assigns each unknown HR pixel the value of its nearest LR pixel. For a $\times 2$ upscale, each LR pixel is replicated in a $2\times 2$ block of identical HR pixels. While this method is fast and produces a closed-form solution, the resulting SR images are of low visual quality due to blocky artifacts.
• Bilinear Interpolation computes a weighted average of the four nearest neighbors. This method extends the nearest-neighbor approach by performing a linear interpolation along both horizontal and vertical axes. Each unknown HR pixel value is estimated as a weighted average of it’s 4-neighbors in LR, where the weight is proportional to the inverse distance from each neighbor. This method produces a smoother output than the nearest-neighbor but still lacks the ability to recover sharp edges.
• Bicubic Interpolation uses sixteen neighboring pixels and cubic weighting. The bicubic interpolation considers 16-neighbors instead of four. A smooth cubic polynomial is fitted over this neighborhood to estimate each HR pixel intensity. As a standard SR baseline, bicubic interpolation generally outperforms bilinear interpolation thanks to its larger support and the use of interpolation kernels with negative coefficients, which help preserve image details and produce sharper edges [11]. However, it still relies on a fixed polynomial approximation of unknown pixel values rather than learning image structures and textures from data.

On another note, interpolation-based upsampling [12] operates on fixed handcrafted kernel values with no learning capacity. These methods tend to produce smooth and blurred outputs, lacking the ability to recover fine texture and edge details. While computationally inexpensive, they may not achieve high reconstruction accuracy for SR tasks where the recovery of unknown pixel values is the primary objective.

2.1.2. DL Based Upsampling Methods

• Transpose convolution is a learnable upsampling technique that increases the spatial resolution of feature maps through trainable filters:

  $$Y(i,j)=\sum _m\sum _{n}X(m,n)K(i-ms,j-ns),$$

  (1)

  where:
  • X is the input feature map,
  • Y is the upsampled output,
  • K is the learnable kernel,
  • s is the stride.
  Compared to the fixed mathematical methods, it is computationally more expensive because it operates in a multi-dimensional space, performing convolution-based operations to compute new pixel values from existing feature representations. However, this learning capacity allows it to recover spatial details that fixed-rule methods may not. It has a structural problem, however; when the kernel size and stride overlap unevenly, the response may be typically non-uniform if different output pixels are received that are contributing to different numbers of input locations. This produces the checkerboard artifacts; the overlapping learned kernel and overlap patterns become uniformly distributed over the image.
• Pixel-Shuffle (also known as sub-pixel convolution) is a highly efficient technique for spatial upsampling in Convolutional Neural Networks (CNNs), often used in SR to increase the spatial resolution of feature maps. It works by rearranging the depth dimension (channels) of an LR feature map into spatial positions, effectively transforming a feature map into an HR output.

  $$X\in {\mathbb{R}}^{H\times W\times C{s}^2},$$

  (2)
  PixelShuffle reshapes it into

  $$Y\in {\mathbb{R}}^{sH\times sW\times C}.$$

  (3)
  Unlike methods that rely on explicit interpolation or trainable upsampling, Pixel-Shuffle operates by, 1) increasing the channel dimension using standard convolution to $C{s}^2$ (where s is the upscale factor), and 2) rearranging these channels into spatial pixels.

2.2. Discrete Wavelet Transform (DWT)

When a 2D DWT is applied to an image, it produces four frequency sub-bands at each level [13]; LL (low-frequency), LH (horizontal edges), HL (vertical edges), and HH (diagonal details), as illustrated in Figure 1 for a grayscale image. The 2D IDWT reconstructs the original image from all four [14] sub-bands. Any missing sub-band prevents full recovery [15]. Mathematically, an image $I(x,y)$ of size (say $256\times 256$ is:

$$I(x,y)\in {\mathbb{R}}^{256\times 256},$$

where

$$x,y\in \{0,1,2,\dots ,255\}.$$

After one-level 2D-DWT:

$$DWT\left(I\right)=\{LL,LH,HL,HH\},$$

where each sub-band has size:

$$LL,LH,HL,HH\in {\mathbb{R}}^{128\times 128}$$

Let:

• L = low-pass filter
• H = high-pass filter

We get 2D-DWT after applying filters on rows and columns horizontally and vertically.

• LL sub-band
  Low-pass in rows and low-pass in columns:

  $$LL(m,n)=\sum _x\sum _{y}I(x,y)L(2m-x)L(2n-y)$$

  (4)
  This sub-band contains a low-frequency approximation of the image while the main global structure and smooth content.
• LH sub-band
  Low-pass in rows and high-pass in columns:

  $$LH(m,n)=\sum _x\sum _{y}I(x,y)L(2m-x)H(2n-y)$$

  (5)
  This band contains information about horizontal edges that is captured because it preserves low-frequency variations in one direction and high-frequency variations in the other.
• HL sub-band
  High-pass in rows and low-pass in columns:

  $$HL(m,n)=\sum _x\sum _{y}I(x,y)H(2m-x)L(2n-y)$$

  (6)
  This sub-band contains only vertical edge information.
• HH sub-band
  High-pass in rows and high-pass in columns:

  $$HH(m,n)=\sum _x\sum _{y}I(x,y)H(2m-x)H(2n-y)$$

  (7)
  This sub-band contains high-frequency diagonal details, such as sharp textures, corners, and fine structures.

Hence, for a $256\times 256$ image:

$$256\times 256\stackrel{\mathrm{DWT}}{\to }4\mathrm{sub}-\mathrm{bands}\mathrm{of}\mathrm{size}128\times 128$$

or equivalently,

$$\begin{array}{c}\hfill I_{256\times 256}\to \{L{L}_{128\times 128},L{H}_{128\times 128},\\ \hfill H{L}_{128\times 128},H{H}_{128\times 128}\}\end{array}$$

One can also write it as:

$$DWT\left(I\right)=\left[\begin{array}{cc}LL& HL\\ LH& HH\end{array}\right]$$

(8)

with each block being $128\times 128$.

From a $256\times 256$ input, after a one-level 2D-DWT decomposion, the image is partitioned into four sub-bands $LL$, $LH$, $HL$, and $HH$ of $128\times 128$ with the $LL$ component representing the low frequency approximation while the other bands representing the horizontal, vertical and diagonal high frequency details.

2.3. The U-Net

The standard U-Net [8] is a fully convolutional network which owes its name to its unique symmetric U-shaped architecture that predicts a class label for every individual pixel in an input image. As illustrated in Figure 2, it has the following main components:

• Encoder path,
• Bottleneck,
• Decoder path,
• Skip connections,
• Final output layer.

The lower layers of the U-net form an encoder path consisting of a series of down-sampling operations, through conv2D and max-pooling layers, that progressively compress the spatial information into a compact latent features representation via a bottleneck. The upper layers form the decoder path and consists of four blocks, each beginning with a transposed convolution that doubles the spatial dimensions of the feature map, followed by skip connections that concatenate encoder features of the corresponding resolution to preserve spatial detail while up-sampling the features.

The U-Net decoder relies on transpose convolution, pixel-shuffle, and bilinear interpolation operations for up-sampling. However, these operations function entirely in the pixel domain and lack explicit frequency awareness, making it difficult to recover high-frequency details such as edges and textures while up-sampling. During the encoding process, this information is lost as the operations could not identically down-sample the pixel distribution. This loss of information during the down-sampling process can lead to a reduced overall reconstruction quality.

To address this limitation, it would be expedient to improve the U-Net model by effectively replacing its pixel-domain up-sampling operation with a frequency-aware domain in the form of DWT/IDWT. Rather than training the network to directly learn or up-sample pixel values, instead train it to predict the missing high-frequency wavelet sub-band [LH, HL, HH] coefficients from the available low-frequency LR image. The final reconstruction can be readily performed using a mathematically exact IDWT function, which is a lossless operation rather than any learned approximation. It is expected that this design may yield a potentially improved SR over all the tested baselines, including pixel-shuffle, transposed convolution, as well as the bilinear and bicubic interpolations; based on metrics, like PSNR and SSIM.

3. Related Work

Many DL methods have been proposed in the literature [16] to learn a plausible mapping between LR and HR images. Dong et al. [17] pioneered by proposing their SR convolutional neural network (SRCNN), an end-to-end mapping from LR to HR images that outperformed the bicubic interpolation. The work opened a floodgate of efforts [18,19,20,21], including what the authors called the Efficient Sub-Pixel Convolutional Network (ESPCN) [22], that was claimed to have improved the reconstruction speed and quality by performing upsampling through sub-pixel convolutional layers; by moving the upsampling to the final layer via a sub-pixel convolution. Another work [23] combines a lightweight CNN backbone with efficient multi-head attention transformer and pixel-shuffle upsampling. The approach in [24] is to employ sliding window based self-attention, a hierarchical vision transformer, to model multiscale SR. Yet another effort [25] propose what the authors call an adaptive token dictionary with category-based cross-attention to extend receptive fields beyond local windows; a pixel-domain operation with the token-dictionary mechanism risking increased computational complexity. A CNN-based residual attention network [26] tries to retain global context for SR through context-reasoning attention, constrained by its CNN backbone in modeling long-range dependencies.

The literature is replete with efforts based on the use of DWT in SR. The work in [15] proposed combining wavelet transformation with a deep CNN to predict missing high-frequency coefficients in LR images for improved SR reconstruction. Another effort [27] employs a GAN-based SR model that predicts wavelet subband residuals in the frequency domain for better edge and texture recovery, although there is a risk of hallucinating artificial details. To improve the resolution of medical images, especially brain Magnetic Resonance Imaging (MRI), images a SwinV2 [28] transformer based method [29] decomposes the LR input via DWT, extracts deep features using the residual blocks and reconstructs the corresponding HR image through IDWT. The authors in [30] combine spatial-domain learning with wavelet-domain refinement, by employing an Artificial Neural Network (ANN) with bilinear interpolation, for shallow feature extraction, and DWT/IDWT for enhancing the coarse structures. After upsampling the LR input by PixelShuffle, the method in[31] applies DWT to decompose it into frequency sub-bands, which are passed through residual blocks built on the Heat Conduction Operator (HCO) to model long-range spatial dependencies in the deep feature extraction stage. Finally, the refined features are reconstructed back into a high-resolution image using IDWT. The work in [32] uses 2D-DWT to explicitly separate the input into four frequency channels (LL, LH, HL, HH) before the network, reducing training difficulty, then applies lightweight Residual Attention Blocks with channel and spatial attention plus multi-kernel convolutions to adaptively aggregate multi-scale features efficiently.

A DWT/IDWT-based feature reconstruction [33] work utilises a wavelet-enhanced U-Net model for medical image SR and claims to restore fine anatomical details and improve multi-scale feature extraction, edge preservation, and texture recovery. In order to enhance Computed Tomography (CT) images, the work in [34] fuses the LR image with its DWT-extracted high-frequency (HH) component before feeding it to their CNN. Remote sensing images are enhanced in [35] by using DWT to decouple deep features into low-frequency (LF) and high-frequency (HF) components, Their architecture uses self-attention for LF, CNN for HF and cascades multi-level wavelet units to expand the receptive field and capture multi-scale context. A Bayesian Nonparametric (BNP) method [36] estimates DWT filter-bank coefficients for SISR using a Hierarchical Dirichlet Process with blocked Gibbs Sampling in order to exploit inter-scale and intra-scale statistical dependencies of wavelet sub-bands, while simultaneously using a Compound Poisson Process to model and remove Poisson noise within each wavelet scale during reconstruction. Yet another paper[37] proposes a GAN with a DWT-based generator that super-resolves LR label maps into refined HR weak reference maps, enabling weak-supervised learning without requiring detailed ground-truth annotations. For facial SR, the authors in [38] use a U-shaped CNN with depth-separable convolutions and a double branch distillation fusion wherein DWT/IDWT are applied to extract and retain high-frequency facial detail information.

The first efforts in Generative Artificial Intelligence (GenAI) include [39], an edge-enhanced Generative Adversarial Network (GAN) for remote sensing SR, but large upscaling factors create information gaps that complicate training and limit reconstruction under constrained conditions. With the advent of attention/transformer based methods, the SR research has taken a new turn and in this context, a group of related methods from one team [1,40,41] relies on window-based self-attention and overlapping cross-attention within residual groups to capture larger receptive fields for HR reconstruction. Cao et al. [42] enhanced U-Net for super-resolution by integrating wavelet transforms, learning multi-scale wavelet features to simplify low-to-high resolution mapping. Their use of inverse wavelet transforms during upsampling improves sparsity and feature richness, demonstrating the effectiveness of wavelet prediction—aligning with our DWT- and IDWT-based approach for preserving high-frequency details. Another effort [43] builds a lightweight U-Net with DWT based encoder/decoder to achieve real-time SR on resource-constrained drone hardware. For the sake of efficiency, without affecting the structural edges. In [44], the authors introduced a modified U-Net for single-image super-resolution, eliminating batch normalization and reducing the depth of convolutional layers in each block to decrease computational overhead. To mitigate the blurred edges often produced by by networks optimized with mean squared error (MSE), they incorporated a mixed gradient loss that combines MSE with a weighted mean gradient error term. Although this approach enhanced edge sharpness, the architecture remains confined to the spatial domain, lacking an explicit mechanism to separate or prioritize high-frequency content. This limitation restricts its capacity to recover fine structural details when compared to frequency-aware methods. The GAN-based method in [45], replaces the standard discriminator with a U-Net-based discriminator and spectral normalization to fuse multi-scale real/fake image features. A transformer based effort [46] employs a combination of sparse attention (for global features) and local dense attention (for local features), integrated into a U-Net with multi-resolution paths (for multi-scale features). Another method [47] introduces shuffle pooling to improve down-sampling and feature extraction by combining it with the U-Net’s encoder–decoder structure. Another transformer based method in [48] applies adaptive token sparsification and early-exit halting to reduce compute while operating entirely in the pixel domain without frequency-domain awareness. A diffusion based work [49] combines a diffusion model with implicit neural representation for continuous-scale SR via coordinate-based upsampling, at the cost of higher compute and slower inference. A comparison based summary of the methods is illustrated in Table 1.

4. Methodology

The standard U-Net architecture, designed for super-resolution, employs an encoder that progressively reduces spatial resolution through max pooling, average pooling, and stridden convolution, while compressing the spatial dimensions. These operations are inherently lossy, with max pooling retaining only the most prominent activations, whereas stridden convolution discarding some spatial information. As a consequence, high-frequency spatial information – such as edges, fine textures, and diagonal structures – is affected, as the feature maps propagate towards the bottleneck. We build upon the lightweight modified U-Net from Lu and Chen [44] as our backbone, preserving its reduced-parameter encoder-decoder structure. Our extension introduces DWT decomposition in the encoder and a learned channel-split mechanism in decoder that adaptively weights each frequency sub-band before IDWT reconstruction. Unlike the original, which relies on a mixed gradient loss for edge sharpness, our architecture explicitly models frequency content, enabling the network to prioritize high-frequency sub-bands where structural detail loss is most significant.

The proposed network, as illustrated in Figure 3, operates in the wavelet domain rather than in the pixel domain. Given an LR input of size $128\times 128$, the model does not directly predict a $256\times 256$ pixel output. Instead, it learns the missing high-frequency wavelet sub-bands [LH, HL, and HH] while preserving the available low-frequency LL sub-band. The IDWT is then applied to all four sub-bands to reconstruct a full $256\times 256$ HR image. This formulation enables the network to learn frequency-domain information more effectively, since the reconstruction target is structured and mathematically interpretable rather than a direct pixel prediction problem. By substituting the standard encoder-decoder operations in U-Net with DWT and IDWT, respectively, we are vying to empirically demonstrate that this approach outperforms conventional upsampling methods under identical training conditions. Furthermore, constraining the network to treat the input as an LR through the proposed architectural optimization reduces the learning burden while maintaining superior reconstruction quality across all tested configuration variants.

Step 1: Network Input and Initial Feature Extraction

The proposed network performs $\times 2$ super-resolution in the wavelet domain. For each training sample, two versions of the low-frequency wavelet component are used: the normalized LR component $L{R}_{\mathrm{norm}}$ and the raw LR component $L{R}_{\mathrm{raw}}$. Both have the same spatial and channel dimensions:

$$L{R}_{\mathrm{norm}},L{R}_{\mathrm{raw}}\in {\mathbb{R}}^{B\times 3\times 128\times 128}.$$

(9)

The normalized component $L{R}_{\mathrm{norm}}$ is used as the network input because it provides a stable scale for learning, while $L{R}_{\mathrm{raw}}$ is preserved for the final inverse wavelet reconstruction. The input first passes through an initial double-convolution block, which maps the 3-channel LR input to 64 shallow feature channels:

$$x_0=F_{\mathrm{stem}}\left(L{R}_{\mathrm{norm}}\right),x_0\in {\mathbb{R}}^{B\times 64\times 128\times 128}.$$

(10)

This stem block extracts low-level spatial and structural features from the normalized LR input. The output $x_0$ is also retained as a skip connection for the decoder stage.

Step 2: Wavelet-Based Encoder (DWT ↓)

The encoder progressively reduces the spatial resolution while increasing the number of feature channels. In conventional U-Net, this reduction is commonly performed using max-pooling or strided convolution. In the proposed architecture, the downsampling operation is replaced by a DWT-based block (Figure 4. For an input feature map $x\in {\mathbb{R}}^{B\times C\times H\times W}$, the DWT decomposes the feature map into four sub-bands:

$$\begin{array}{cc}\hfill \mathrm{DWT}\left(x\right)& \to \{L{L}_f,L{H}_f,H{L}_f,H{H}_f\},\hfill \\ \hfill L{L}_f,L{H}_f,H{L}_f,H{H}_f& \in {\mathbb{R}}^{B\times C\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}}.\hfill \end{array}$$

(11)

The four sub-bands are concatenated along the channel dimension, increasing the channel count from C to $4C$:

$$[L{L}_f\parallel L{H}_f\parallel H{L}_f\parallel H{H}_f]\in {\mathbb{R}}^{B\times 4C\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}}.$$

(12)

The concatenated representation is then passed through a double-convolution block. This block applies two consecutive $3\times 3$ convolution layers, each followed by LeakyReLU activation. It refines the concatenated wavelet features, changes the channel depth from the input channel size to the required output channel size, and produces the downsampled feature map:

$$\begin{array}{cc}\hfill \mathrm{DoubleConv}\left(x\right)& =\varphi \left(W_2*\varphi \left(W_1*x+b_1\right)+b_2\right),\hfill \\ \hfill x_{\mathrm{down}}& =\mathrm{DoubleConv}\left([L{L}_f\parallel L{H}_f\parallel H{L}_f\parallel H{H}_f]\right),\hfill \end{array}$$

(13)

where $\varphi (·)$ denotes the LeakyReLU activation, * denotes a $3\times 3$ convolution operation, and ∥ denotes channel-wise concatenation. This design explicitly represents both low-frequency and high-frequency components before spatial reduction. Therefore, edge-, texture-, and structure-related responses are carried through the $L{H}_f$, $H{L}_f$, and $H{H}_f$ sub-bands instead of being directly suppressed by pooling. In the implemented encoder, two DWT-based downsampling stages are used. The first stage receives the output of stem $x_0$ and reduces the spatial size from $128\times 128$ to $64\times 64$, while increasing the channel depth from 64 to 128:

$$\begin{array}{cc}\hfill x_1& =F_{\mathrm{down}1}\left(x_0\right),\hfill \\ \hfill x_0& \in {\mathbb{R}}^{B\times 64\times 128\times 128},\hfill \\ \hfill x_1& \in {\mathbb{R}}^{B\times 128\times 64\times 64}.\hfill \end{array}$$

(14)

The second stage further reduces the feature resolution from $64\times 64$ to $32\times 32$ and increases the channel depth from 128 to 256:

$$\begin{array}{cc}\hfill x_2& =F_{\mathrm{down}2}\left(x_1\right),\hfill \\ \hfill x_1& \in {\mathbb{R}}^{B\times 128\times 64\times 64},\hfill \\ \hfill x_2& \in {\mathbb{R}}^{B\times 256\times 32\times 32}.\hfill \end{array}$$

(15)

Thus, the encoder transforms the initial feature representation as:

$$\begin{array}{cc}\hfill & B\times 64\times 128\times 128\hfill \\ \hfill & \to B\times 128\times 64\times 64\hfill \\ \hfill & \to B\times 256\times 32\times 32.\hfill \end{array}$$

(16)

Step 3: Extended Bottleneck with Channel Attention (Residual Channel Attention Block (RCAB) Stack)

After the encoder, the feature map reaches the lowest spatial resolution of the network, $32\times 32$. This central part of the encoder–decoder architecture is used as the bottleneck because it operates on the most spatially compact representation before the decoder begins reconstruction. At this stage, the feature map has already been compressed spatially; therefore, the bottleneck first applies a double-convolution block to increase the feature depth from 256 channels to 512 channels. This increases the representational capacity of the model while keeping the computational cost manageable due to the reduced spatial size. The bottleneck transformation is expressed as:

$$\begin{array}{cc}\hfill x_b& =F_{\mathrm{bottleneck}}\left(x_2\right)=\mathrm{DoubleConv}\left(x_2\right),\hfill \\ \hfill x_2& \in {\mathbb{R}}^{B\times 256\times 32\times 32},\hfill \\ \hfill x_b& \in {\mathbb{R}}^{B\times 512\times 32\times 32}.\hfill \end{array}$$

(17)

The purpose of this channel expansion is to learn a deeper and richer feature representation before decoding. Since the model later predicts the missing high-frequency wavelet sub-bands, the expanded bottleneck representation (Figure 5) helps encode structural, edge-related, and frequency-relevant information at the most compact spatial scale.

The bottleneck feature map is then refined using a stack of RCABs. In the implemented configuration, the stack contains 30 RCABs. It is important to note that the channel expansion from 256 to 512 occurs only in the bottleneck double-convolution block. The RCAB stack does not further expand the number of channels; instead, it refines and reweights the existing 512-channel feature representation.

Each RCAB contains a residual convolutional branch followed by a channel attention module. The residual branch can be written as:

$$r=F_{\mathrm{res}}\left(x\right),$$

(18)

where $F_{\mathrm{res}}(·)$ denotes two convolutional layers used for residual feature extraction. The channel attention (CA) module is then applied to the residual feature r, and the RCAB output is obtained through residual addition:

$$x_{\mathrm{RCAB}}=x+\mathrm{CA}\left(r\right).$$

(19)

For a feature tensor $r\in {\mathbb{R}}^{B\times C\times H\times W}$, the CA module first applies global average pooling (GAP):

$$z=\mathrm{GAP}\left(r\right),z\in {\mathbb{R}}^{B\times C\times 1\times 1}.$$

(20)

This operation compresses only the spatial dimensions $H\times W$ into $1\times 1$, while preserving the number of channels C. The pooled descriptor is then passed through a lightweight channel-wise Multilayer Perceptron (MLP) followed by a sigmoid activation to obtain one attention weight for each channel:

$$A\left(r\right)=\sigma \left(\mathrm{MLP}\left(z\right)\right),A\left(r\right)\in {\mathbb{R}}^{B\times C\times 1\times 1}.$$

(21)

The attention weights are multiplied with the residual feature map through channel-wise multiplication:

$$\mathrm{CA}\left(r\right)=r\odot A\left(r\right),\mathrm{CA}\left(r\right)\in {\mathbb{R}}^{B\times C\times H\times W}.$$

(22)

where $\sigma (·)$ denotes the sigmoid activation and ⊙ denotes channel-wise multiplication with broadcasting over the spatial dimensions. Thus, channel attention does not increase or decrease the number of channels. It only learns channel-wise importance weights and rescales the existing feature channels.

In the bottleneck of the proposed architecture, this means that the RCAB stack receives and outputs feature maps with the same channel depth:

$$x_b\in {\mathbb{R}}^{B\times 512\times 32\times 32}⟶x_{\mathrm{CA}}\in {\mathbb{R}}^{B\times 512\times 32\times 32}.$$

(23)

Therefore, the bottleneck first increases the channel depth through the double-convolution block and then uses the RCAB stack to refine the 512-channel representation. This allows the model to emphasize informative feature channels before decoding, which is useful for wavelet-domain super-resolution because different channels may contribute differently to the later prediction of $LH$, $HL$, and $HH$ high-frequency sub-bands.

Step 4: Wavelet-Based Decoder (IDWT ↑)

The decoder restores the spatial resolution in a reverse multi-scale manner. Instead of using transposed convolution or pixel-shuffle, the proposed decoder uses IDWT-based upsampling blocks, as illustrated in Figure 6. At each decoder stage, the input feature channels are divided into four equal groups and treated as pseudo wavelet sub-bands. For an input tensor with C channels, let $c=C/4$. The four channel groups are given by:

$$x_{LL}=x_{0:c},x_{LH}=x_{c:2c},x_{HL}=x_{2c:3c},x_{HH}=x_{3c:4c}.$$

(24)

These four groups are passed through IDWT to double the spatial resolution:

$$x_{\mathrm{up}}=\mathrm{IDWT}\left(x_{LL},x_{LH},x_{HL},x_{HH}\right).$$

(25)

The upsampled feature map is then concatenated with the corresponding encoder skip connection and refined using a double-convolution block:

$$x_{\mathrm{dec}}=\mathrm{DoubleConv}\left([x_{\mathrm{up}}\parallel x_{\mathrm{skip}}]\right).$$

(26)

The first decoder stage receives the bottleneck output with 512 channels at $32\times 32$. After IDWT upsampling, the feature map is restored to $64\times 64$, concatenated with the encoder skip feature $x_1$, and fused to produce 256 output channels:

$$x_{\mathrm{up}2}=F_{\mathrm{up}2}(x_b,x_1),x_{\mathrm{up}2}\in {\mathbb{R}}^{B\times 256\times 64\times 64}.$$

(27)

The second decoder stage restores the resolution from $64\times 64$ to $128\times 128$. It uses the skip connection $x_0$ from the stem level and produces 128 output channels:

$$x_{\mathrm{up}1}=F_{\mathrm{up}1}(x_{\mathrm{up}2},x_0),x_{\mathrm{up}1}\in {\mathbb{R}}^{B\times 128\times 128\times 128}.$$

(28)

Thus, the decoder transforms the bottleneck feature representation as:

$$B\times 512\times 32\times 32\to B\times 256\times 64\times 64\to B\times 128\times 128\times 128.$$

(29)

This decoder design provides a wavelet-consistent upsampling mechanism. The skip connections help recover spatial information from earlier encoder stages, while the IDWT-based blocks restore the feature resolution in a structured sub-band form.

Step 5: High-Frequency Sub-Band Prediction Head

After the two IDWT-based decoder stages, the feature map returns to the LR spatial resolution of $128\times 128$. A refinement block is then applied before predicting the final wavelet high-frequency sub-bands. As shown in Figure 7, this refinement block contains two additional RCABs, followed by a $3\times 3$ convolution and LeakyReLU activation. It reduces the decoder feature depth from 128 channels to 64 channels:

$$x_{\mathrm{refine}}=F_{\mathrm{refine}}\left(x_{\mathrm{up}1}\right),x_{\mathrm{refine}}\in {\mathbb{R}}^{B\times 64\times 128\times 128}.$$

(30)

The refined feature map is then passed to a final $3\times 3$ convolutional prediction head. This head outputs 9 channels, corresponding to three red-green-blue (RGB) channels for each of the three high-frequency wavelet sub-bands:

$$\widehat{H}={\mathrm{Conv}}_{3\times 3}\left(x_{\mathrm{refine}}\right)\times s_{hf},\widehat{H}\in {\mathbb{R}}^{B\times 9\times 128\times 128}.$$

(31)

Here, $s_{hf}$ is a learnable scalar initialized to $0.1$. This scaling factor reduces the magnitude of the initial high-frequency predictions and may improve early training stability.

The 9-channel output is then divided into three groups using channel-wise chunking:

$$\begin{array}{cc}\hfill & =\mathrm{chunk}(\widehat{H},3),\hfill \\ \hfill \widehat{LH},\widehat{HL},\widehat{HH}& \in {\mathbb{R}}^{B\times 3\times 128\times 128}.\hfill \end{array}$$

(32)

Therefore, the network does not directly predict the final high-resolution RGB image. Instead, it predicts the missing wavelet detail components $\widehat{LH}$, $\widehat{HL}$, and $\widehat{HH}$, which are later combined with the preserved raw LR component.

Step 6: Final HR Reconstruction via IDWT

The final super-resolved image is reconstructed using IDWT. In this step, as illustrated in Figure 8, the original raw LR is combined with the predicted high-frequency sub-bands:

$${\widehat{I}}_{HR}=\mathrm{IDWT}\left(L{R}_{\mathrm{raw}},\widehat{LH},\widehat{HL},\widehat{HH}\right).$$

(33)

The input sub-bands to the final IDWT all have the same spatial size and channel count:

$$L{R}_{\mathrm{raw}},\widehat{LH},\widehat{HL},\widehat{HH}\in {\mathbb{R}}^{B\times 3\times 128\times 128}.$$

(34)

After IDWT reconstruction, the output becomes:

$${\widehat{I}}_{HR}\in {\mathbb{R}}^{B\times 3\times 256\times 256}.$$

(35)

This final reconstruction step is important because $L{R}_{\mathrm{raw}}$ is not predicted by the network; it is directly reused in its original coefficient scale. Thus, the low-frequency structure is preserved through the raw LR component, while the network focuses its output prediction on the missing high-frequency detail sub-bands. The complete model can therefore be expressed compactly as:

$$f_{\theta }\left(L{R}_{\mathrm{norm}}\right)=\{\widehat{LH},\widehat{HL},\widehat{HH}\},$$

(36)

$${\widehat{I}}_{HR}=\mathrm{IDWT}\left(L{R}_{\mathrm{raw}},f_{\theta }\left(L{L}_{\mathrm{norm}}\right)\right).$$

(37)

Here, $f_{\theta }$ denotes the proposed wavelet U-Net. The normalized LR component is used for learning the high-frequency sub-band prediction, whereas the raw LR component is used only during the final IDWT reconstruction. Consequently, the model performs $\times 2$ super-resolution by reconstructing a $256\times 256$ RGB image from a $128\times 128$ LR representation and the predicted high-frequency wavelet details. Table 2 sums up our modified U-Net pipeline.

5. Experimental Setup and Results

For the sake of our experiments, we have used images from standard datasets. However, each HR RGB image was first resized to $256\times 256$ to serve as the ground truth (GT) and a single-level 2D-DWT was then applied channel-wise to each such image to get our LR inputs that upon processing by our network, gave us the $256\times 256$ HR output. Synthetic blur or noise degradation was not added during this process.

5.1. Training and Benchmark Datasets

The proposed model was trained and validated using the DIV2K [55] dataset. DIV2K contains 800 high-resolution training images and 100 validation images. Each PNG image was center-cropped into a square shape to handle different aspect ratios, and then resized to $256\times 256$ using bicubic interpolation. The resized images were saved as float32 tensors in the range $[-1,1]$. The standard DIV2K split was used, with 800 images for training and 100 images for validation, without any test set augmentation. In this work, the high-resolution image size is $256\times 256$, while the low-resolution input is represented by the LR image of size $128\times 128$, corresponding to a scaling factor of $\times 2$.

In SISR related works, standard benchmark datasets are very important for a fair comparison with state of the art methods. Commonly used benchmark datasets include Set5, Set14, and BSD100. These datasets are among the most widely used datasets in the super-resolution literature and have been used for evaluating DL-based models such as SRCNN [17] and later methods.

Set5 [56] is a small benchmark dataset containing 5 classic images, including baby, bird, butterfly, head, and woman. Although it is small in size, it contains rich structural information, edges, and complex image details. Therefore, PSNR and SSIM evaluation on Set5 is commonly reported in super-resolution research.

Set14 [57] It contains images with different types of visual diversity, including faces, animals, comic art, text, and fine patterns. Due to this variation, Set14 is also a well-known benchmark dataset widely used in super-resolution literature.

BSD100 [58] is a subset of 100 test images from the Berkeley Segmentation Dataset, introduced by Martin et al. in 2001. It is considered a challenging benchmark because it contains natural photographs with diverse textures, irregular edges, and cluttered backgrounds. For this reason, BSD100 is also widely used for evaluating SISR methods.

5.2. Preprocessing

Since no separate ground-truth HR images were available, the original dataset images were used as the HR reference images and were treated as ground truth for training and evaluation. In this work, each original dataset image with spatial resolution $256\times 256$ was considered as the HR target. The corresponding LR image was generated from the same HR reference image using DWT. Therefore, the LR–HR image pairs were constructed from the dataset itself, where the original $256\times 256$ image served as the ground-truth HR image and the DWT approximation component served as the LR input.

This process produced the LR approximation image, which was used as the LR input to the proposed network. No synthetic blur or noise degradation was added during this process. The scaling factor was $\times 2$, where the spatial resolution was reduced from $256\times 256$ to $128\times 128$ through DWT faimly biorthogonal bior4.4. After training, the network output was used to reconstruct the final SR image at $256\times 256$ resolution through IDWT.

For normalization, the per-channel mean and standard deviation were computed over the entire training split of LR images. During training and inference, the LR input was standardized as

$$l{r}_{\mathrm{norm}}=\left(l{r}_{\mathrm{raw}}-\mu \right)/\sigma ,$$

(38)

where $\mu$ and $\sigma$ denote the training-split mean and standard deviation, respectively. The raw LR images $\left(l{r}_{\mathrm{raw}}\right)$, without normalization, were separately preserved and directly used in the final IDWT reconstruction.

The training target was the original HR RGB image with shape $(3,256,256)$ in the range $[-1,1]$. The high-frequency subband targets, including LH, HL, and HH, were extracted on-the-fly during training from the HR reference image using the same bior4.4 wavelet and periodization DWT setting. After reconstruction, the generated SR image with resolution $256\times 256$ was compared with its corresponding original HR reference image for quantitative evaluation using PSNR and SSIM.

5.3. Training Details

The proposed Wavelet U-Net was trained on the DIV2K dataset using paired LR-HR samples. As we do not have separate ground-truth HR original dataset or LR images we did preprocessing. The original dataset image with spatial resolution $256\times 256$, and the corresponding LR images of size $128\times 128$ were used as input to the network. The LR inputs were normalized using training-set statistics, while the raw LR images were preserved separately for final IDWT reconstruction. The model was trained using the Adam optimizer with a learning rate of $2\times {10}^{-4}$, batch size of 8, and 100 epochs. The loss function was defined on the reconstructed HR image and predicted high-frequency subbands. The performance was evaluated using PSNR and SSIM on the validation set.

Figure 9 illustrates the training convergence analysis of the proposed DWT/IDWT U-Net model on the DIV2K dataset. The L1 loss, Figure 9(a), decreases steadily, confirming stable learning and reduced pixel-level error. Simultaneously, as can be seen in Figure 9(b,c), the PSNR and SSIM consistently increase, demonstrating that predicting high-frequency wavelet sub-bands prior to IDWT reconstruction effectively minimizes distortion and accurately preserves structural details, like edges, textures, and local image patterns.

Figure 10 compares U-Net decoder variants across four benchmark datasets; viz. Set5, Set15, BSD100, and DIV2K (HRO). The proposed DWT/IDWT-based U-Net achieves the highest PSNR (37.85 dB) (a) and SSIM (0.9596) (b) scores. These results demonstrate that the wavelet-based decoder minimizes distortion and preserves structural details more effectively than conventional upsampling designs.

The Loss function:

$$\mathcal{L}={\lambda }_{L1}{\mathcal{L}}_{L1}(SR,HR)+{\lambda }_{HF}·{\displaystyle \frac{1}{3}}\sum _{s\in \{LH,HL,HH\}}{\mathcal{L}}_{L1}(\widehat{s},s_{GT})$$

(39)

where ${\lambda }_{L1}=1.0$ and ${\lambda }_{HF}=0.7$.

Table 3 presents the main training hyperparameters used for model optimization. The network was trained on the DIV2K dataset using normalized low-resolution inputs of size $128\times 128$ and corresponding high-resolution targets of size $256\times 256$. The Adam optimizer with an initial learning rate of $2\times {10}^{-4}$ and a batch size of 8 was employed for 100 epochs. Model performance was evaluated using the PSNR and SSIM metrics to assess reconstruction fidelity and structural similarity.

Table 4 summarizes the training and implementation settings used in this study. The proposed model was implemented in PyTorch with PyTorch Lightning and trained on a single Google Colab L4 GPU using mixed-precision computation for improved efficiency. Training was performed for up to 100 epochs using a MultiStepLR scheduler, with the learning rate reduced at epochs 60, 80, and 95. The network employed 30 RCAB blocks and was optimized using a combination of image reconstruction L1 loss and high-frequency subband supervision. To enhance training stability, gradient clipping and exponential moving average (EMA) of model weights were applied. Model performance was evaluated using PSNR and SSIM, while checkpoint selection was based on the highest validation PSNR-Y scores.

PSNR and SSIM are computed on the luma (Y) channel only, following the standard SR evaluation protocol. The Y channel is derived using the standard formula:

$$Y=16+{\displaystyle \frac{65.481R+128.553G+24.966B}{255}}$$

(40)

A 2-pixel border is shaved from all images before the metric computation to exclude boundary artifacts, consistent with standard SR benchmarks. At validation, EMA generator weights are used instead of the raw training weights, producing more stable metric estimates. Both PSNR and SSIM are reported as epoch-level mean ± standard deviation (std) over the full validation split.

5.4. Comparison with Various Upsampling Approaches in U-Net

Figure 11, Figure 12, and Figure 13 visually compare the proposed method against bicubic interpolation, transposed Convolution, and the baseline U-Net. Magnified regions highlight the differences in fine details. For the lemon image (Figure 11), the proposed model reconstructs sharper edges and textures closest to the ground truth. In the flower image (Figure 11), it recovers detailed petal structures where competing methods cause blurring. Finally, for the human figures (Figure 11), the model accurately preserves object boundaries with fewer artifacts, confirming its superior high-frequency reconstruction capabilities. Overall, the qualitative results demonstrate that the proposed wavelet-based DWT/IDWT U-Net consistently generates visually sharper images with improved detail preservation across diverse scene contents, producing reconstructions that more closely resemble the ground-truth images.

The results presented in Table 5 highlight several important observations. The proposed DWT/IDWT method achieves a PSNR of 30.59 dB, outperforming all baseline approaches. The closest competing method, U-Net combined with bicubic upsampling, attains 29.05 dB, resulting in a performance gain of $+1.54$ dB for the proposed approach.

Furthermore, the HR₀ baseline of the proposed method, which corresponds to IDWT reconstruction using zero high-frequency sub-bands, already achieves a PSNR of 29.20 dB. This value surpasses the HR₀ results of all other methods, including bicubic interpolation, which only reaches 18.08 dB. This observation indicates that the LR-anchored reconstruction provides a significantly stronger initialization even before any learning is applied.

In contrast, transposed convolution yields the lowest performance among the learned methods, with a PSNR of 27.48 dB. This result aligns with prior findings, as transposed convolution is often associated with checkerboard artifacts that degrade reconstruction quality.

For Table 6 (benchmark datasets): The results across the benchmark datasets reveal consistent improvements of the proposed method. The largest PSNR gain is observed on Set5, with an improvement of $+2.45$ dB. This dataset contains relatively simple and clean images, where accurate recovery of high-frequency details has a more pronounced impact on performance.

Urban100 exhibits the highest improvement in SSIM, with a gain of $+0.0337$, suggesting that the wavelet-based approach is particularly effective at preserving structural information and edge details in urban scenes characterized by repetitive and fine patterns.

In contrast, BSD100 shows the smallest PSNR improvement of $+1.26$ dB. This is expected, as natural images with complex textures are inherently more challenging for SR tasks.

5.5. Comparison with State-of-the-Art

To assess the performance of our proposed super-resolution approach, we benchmark its quantitative results against multiple deep learning-based super-resolution techniques, such as SRCNN, FSRCNN, VDSR, DRCN, SRGAN, ESPCN, EDSR, DBPN, UnetSR, and UnetSR+. We also include bicubic interpolation as a baseline for comparison. The evaluation focuses on the $\times 2$ upscaling factor, considering PSNR and SSIM as key metrics. The quantitative results for the Set14, BSD300, and ICDAR2003 datasets are summarized in Table 7, where the best-performing results are marked in bold and the second-best are underlined.

Our method achieves the highest PSNR and SSIM scores on Set14, with values of 32.80 dB and 0.9295, respectively. This demonstrates its effectiveness in reconstructing image details and maintaining structural fidelity for natural image benchmarks. On the BSD300 dataset, our approach also delivers the best performance, with 32.30 dB PSNR and 0.9145 SSIM, surpassing all competing methods. Given the diversity of natural scenes, textures, and structural variations in BSD300, these results highlight the robustness of our wavelet-based reconstruction strategy in handling complex image content.

For text-image super-resolution, our method records the highest PSNR on ICDAR2003 at 37.74 dB, along with a competitive SSIM of 0.96. While UnetSR+ achieves a marginally higher SSIM on this dataset, our approach maintains the best PSNR and strong structural similarity. This is particularly significant for text-image super-resolution, where sharp edges, stroke boundaries, and fine character structures are critical for visual clarity. The strong PSNR performance on ICDAR2003 confirms that our method effectively minimizes reconstruction errors while preserving perceptual quality.

Overall, the results confirm that our wavelet-based U-Net delivers strong reconstruction quality across both natural and text-image super-resolution benchmarks. The consistent improvements on Set14 and BSD300, combined with the highest PSNR on ICDAR2003, demonstrate that integrating DWT and IDWT into the reconstruction process enhances the preservation of high-frequency details, such as edges, textures, and fine structures, during encoder-decoder processing. Thus, our method presents an effective solution for $\times 2$ single-image super-resolution.

6. Conclusions

This paper presented a wavelet-based U-Net that works for $\times 2$ SISR in which conventional encoder-decoder sampling operators were replaced with DWT and IDWT. Under controlled comparisons with standard U-Net variants using bicubic interpolation, transposed convolution, Pixel-Shuffle, and conventional U-Net, the proposed design achieved superior PSNR and SSIM performance. The results indicate that frequency-aware, multi-resolution upsampling improves reconstruction fidelity in a modified encoder–decoder U-Net network. The proposed DWT/IDWT method achieves 37.85 dB PSNR on the Set5 dataset, outperforming all baselines. The closest competitor is U-Net + Bicubic at 35.09 dB, representing a margin of +2.76 dB in favor of the proposed method. The HR₀ base line of the proposed method (IDWT) reconstruction with zero high-frequency sub-bands already scores 29.20 dB, which exceeds all other methods’ HR₀ results (18.08 dB bicubic). This confirms that the single-image reconstruction provides a substantially stronger starting point prior to any learning. We also introduced the flexible learned feature split concept that constrains the network to predict missing high-frequency wavelet sub-bands (LH, HL, HH) from the high frequency domain feature map.

There are some limitations that could be resolved in the future to improve U-Net SR with DWT/IDWT-based encoder-decoder reconstruction efficiency and usefulness. The evaluation focuses exclusively on one architecture family (U-Net), and further tests across diverse backbone architectures, datasets, and scale factors are needed to establish broader generalizability. We used a fixed family of wavelets, like all experiments we did with the use of bior4.4 mode periodisation. Our method only works on the $\times 2$ scale; we have not tested it on multiscale yet. Extending to $\times 4$, $\times 8$, where multi-level wavelet decomposition is still an open direction for future work.