Deep Dynamic Weights for Underwater Image Restoration

1. Introduction

Underwater imaging plays an important role in ocean observation and marine engineering applications. However, underwater images suffer from several artifacts. While capturing underwater images, a considerable portion of the light is absorbed during its propagation in the water, resulting in color distortion [1]. Moreover, backward-forward light scattering severely affects the contrast and details of images, which further deteriorates the performance of underwater industrial applications [2]. Therefore, Underwater Image Enhancement – addressing color restoration, enhancing contrast, and improving details – is an essential task in marine engineering and observation applications.

In the literature, many methods have been proposed for improving underwater image quality. These methods can be broadly categorized into prior-based, imaging-based, and machine-deep learning-based techniques [3]. Prior-based methods utilize the underwater image formation model (IFM) and draw priors from the degraded images. Initially, a transmission map (TM) is derived from priors such as the dark-channel prior (DCP) [4], red-channel prior (RCP) [5], medium-channel prior (MDP) [6], and haze-line prior (NLD) [7]. Subsequently, the image is restored using the IFM, equipped with the TM and atmospheric light. In [8], a red channel prior (RCP)-guided variational framework is introduced to enhance the TM, and the image is restored utilizing the IFM. Generally, these methods heavily depend on hand-crafted priors and excel in dehazing outdoor images. However, their performance is less than satisfactory for underwater images, and they struggle to correctly manage color shifts.

Imaging-based methods, in contrast to the prior ones, do not utilize the IFM. Instead, they rely on foundational image enhancement techniques such as contrast enhancement, histogram equalization, image fusion, and depth estimation. Peng et al. [2] proposed a depth estimation technique for underwater scenes that relies on image blurriness and light absorption. This depth information is fed into the IFM to restore and enhance the underwater visuals. In another study by Ancuti et al. [9], a combined approach of color compensation and white balancing is applied to the original degraded image to restore its clarity. Zhang et al. [10] introduced a strategy guided by the minimum color loss principle and maximum attenuation map to adjust for color shifts. In another recent work by Zhang et al. [11], a Retinex-inspired color correction mechanism is employed to eliminate color cast. The research further incorporates both local and global contrast-enhanced versions of the image to refine the color output. Although these approaches significantly improve the color and contrast of underwater images, they often overlook the specificities of underwater imaging models. This oversight can result in over-enhanced or over-saturated final images.

On the other hand, deep learning methods are mainly divided into ASM-based and non-ASM-based techniques. ASM-based methods use the Atmospheric Scattering Model to clear up hazy images. For instance, DehazeNet [12] by Cai et al. applies a deep architectural approach to estimate transmission maps, generating clear images. Similarly, MSCNN [13] by Ren et al. uses a multi-scale network to learn the mapping between hazy images and their corresponding transmission maps. AOD-Net [14] by Li et al. directly creates clear images with a lightweight CNN, and DCPDN [15] by Zhang et al. leverages an innovative network architecture, focusing on multi-level pyramid pooling to optimize dehazing performance. In contrast, non-ASM-based methods rely on various network designs to transform hazy images directly into clear ones through various structures like the encoder-decoder, GAN-based, attention-based, knowledge transfer, and transformer-based networks. Encoder-decoder structures like the Gated Fusion Network (GFN) by Ren et al. [16] and Gated Context Aggregation Network (GCANet) by Chen et al. [17] utilize multiple inputs and dilated convolutions to effectively reduce halo effects and enhance feature extraction. GAN-based networks such as Cycle-Dehaze by Engin et al. [18] and BPPNet by Singh et al. [19] offer unpaired training processes and are capable of learning multiple complexities, thereby yielding high-quality dehazing results even with minimal training datasets. Attention-based networks like GridDehazeNet by Liu et al. [20] and FFA-Net by Qin et al. [21] implement adaptive and attention-based techniques, providing more flexibility and efficiently dealing with non-homogeneous haze. Knowledge transfer methods like KTDN by Wu et al. [22] leverage teacher-student networks, enhancing performance in non-homogeneous haze conditions by transferring the robust knowledge acquired by the teacher network. Lastly, transformer-based networks like DehazeFormer by Song et al. [23] make significant modifications in traditional structures and employ innovative techniques like SoftReLU and RescaleNorm, presenting better performance in dehazing tasks with efficient computational cost and parameter utilization.

In this study, we propose a method for underwater image restoration that employs linear or non-linear mapping depending on the type of the input image. First, an input image is classified as Type-I or Type-II. Then, the Type-I image is enhanced using the Deep Line Model (DLM), while the Deep Curve Model (DCM) is employed for the Type-II image. The DLM effectively integrates color compensation and contrast adjustment in a unified process, utilizing deep lines for transformation, whereas the DCM is focused on applying higher-order curves for image enhancement. Both models utilize lightweight neural networks that learn per-pixel dynamic weights based on the input image’s characteristics. The efficacy of the proposed method is measured by conducting experiments on benchmark datasets and using quantitative metrics like PSNR and RMSE. The comparative analysis affirms our method’s effectiveness in accurately restoring underwater images, outperforming existing techniques.

2. Motivation

Let $I_c\left(\mathbf{x}\right)$ represent a degraded underwater color image, where $\mathbf{x}=(x,y)$ denotes the coordinates of the image pixels and $c\in \{r,g,b\}$ signifies the red, green, and blue color channels, respectively. The color components of the image can thus be denoted as $\{I_r\left(\mathbf{x}\right),I_g\left(\mathbf{x}\right),I_b\left(\mathbf{x}\right)\}$. In underwater imaging, differential color attenuation across wavelengths frequently leads to compromised visual fidelity, predominantly impacting the red channel while leaving the green comparatively unaltered [24]. Conventional restoration techniques typically adopt a sequential approach: initial color correction to balance channel disparities, followed by linear enhancement methods such as contrast stretching to mitigate the attenuation effects.

In literature, many methods use the mean values from each channel for color compensation [9,24,25,26,27]. This approach is grounded in the Gray World assumption, which suggests that all channels should exhibit equal mean intensities in an undistorted image [25], leading to a straightforward approach for color compensation:

$$\left\{\begin{array}{c}{I}_r\left(\mathbf{x}\right)=I_r\left(\mathbf{x}\right)+({\overline{I}}_g\left(\mathbf{x}\right)-{\overline{I}}_r\left(\mathbf{x}\right))·(1-I_r\left(\mathbf{x}\right)·I_g\left(\mathbf{x}\right)),\hfill \\ I_g\left(\mathbf{x}\right)=I_g\left(\mathbf{x}\right),\hfill \\ I_b\left(\mathbf{x}\right)=I_b\left(\mathbf{x}\right)+({\overline{I}}_g\left(\mathbf{x}\right)-{\overline{I}}_b\left(\mathbf{x}\right))·(1-I_b\left(\mathbf{x}\right)·I_g\left(\mathbf{x}\right)).\end{array}\right.$$

(1)

where ${\overline{I}}_r\left(\mathbf{x}\right)$, ${\overline{I}}_g\left(\mathbf{x}\right)$, and ${\overline{I}}_b\left(\mathbf{x}\right)$ denote the mean values of the degraded color components of the underwater image $I_c\left(\mathbf{x}\right)$.

Although additive adjustments can compensate for color distortions in red and blue channels, our study reveals that this compensation may worsen the color composition in many cases, leading to inferior quality in restored images. As demonstrated in Figure 1, two distinct outcomes are observed: Type-I images benefit from color correction, with spectral intensities approaching the ground truth, enhancing visual quality. Conversely, Type-II images experience worsened color discrepancies, resulting in suboptimal restoration. This necessitates a dual restoration approach. Our method uses a classifier to categorize images, followed by the application of the DLM for Type-I and the DCM for Type-II. This strategy ensures precise, adaptive restoration aligned with the specific requirements of each image category.

3. Proposed Method

The proposed methodology restores images through a two-phase process. Initially, an image classifier categorizes each image as either Type-I or Type-II. Subsequently, Type-I images are processed using the Deep Line Model (DLM), while Type-II images undergo enhancement through the Deep Curve Model (DCM). The complete framework depicted in Figure 2 showcases the complete process, from initial classification to the final output, highlighting the effectiveness and adaptability of both DLM and DCM in underwater image restoration.

3.1. Image Classifier

Based on the observations in images’ profile intensity and metrics, images have been categorized into two distinct types. Type-I images are those that retain or improve in quality following linear transformation for color compensation, while Type-II images are characterized by a decline in quality after the same transformation. Following to this classification, we applied linear transformations and computed metrics such as RMSE and PSNR and labelled the images accordingly. For the classification task, DenseNet121 [26] is employed to analyze the intrinsic properties of an input image and to guide it towards the most appropriate restoration pathway—DLM for Type-I and DCM for Type-II images—thereby enhancing feature extraction and image restoration. The subsequent sections delve into the detailed frameworks of each model.

3.2. Deep Line Model

Mostly, underwater image enhancement methods are executed in two linear operations. Initially, a color-corrected image ${\acute{I}}_c\left(\mathbf{x}\right)$ is obtained by compensating the channels through additive adjustment factors [9,27]. The generalized form of this operation is as follows:

$${\acute{I}}_c\left(\mathbf{x}\right)=I_c\left(\mathbf{x}\right)+{\eta }_c,$$

(2)

where ${\eta }_c$ represents the additive adjustment factors for each channel $c\in \{r,g,b\}$. In the second step, the restored image ${\widehat{I}}_c\left(\mathbf{x}\right)$ is obtained by improving contrast. It is achieved by applying another linear operation that stretches the pixel values of the color-corrected image.

$${\widehat{I}}_c\left(\mathbf{x}\right)={\alpha }_c·{\acute{I}}_c\left(\mathbf{x}\right)+{\beta }_c,$$

(3)

where, ${\alpha }_c,{\beta }_c$, are constants utilized to represent weights for each channel that are mostly applied globally.

Instead of performing two separate linear operations and using global weights, we suggest a deep line model that combines two steps and uses per-pixel weights. The proposed model is expressed as;

$${\widehat{I}}_c\left(\mathbf{x}\right)={\alpha }_c\left(\mathbf{x}\right)·I_c\left(\mathbf{x}\right)+{\beta }_c\left(\mathbf{x}\right)·{\eta }_c,$$

(4)

where, ${\alpha }_c\left(\mathbf{x}\right),{\beta }_c\left(\mathbf{x}\right)$, are weight matrices and ${\eta }_c$ color compensation factor for each channel $c\in \{r,g,b\}$. The color compensation factors ${\eta }_c$ are computed by using the mean guided compensations [24] through the following expressions.

$$\left\{\begin{array}{c}{\eta }_r=({\overline{I}}_g\left(\mathbf{x}\right)-{\overline{I}}_r\left(\mathbf{x}\right))(1-I_r\left(\mathbf{x}\right)I_g\left(\mathbf{x}\right)),\hfill \\ {\eta }_g=0,\hfill \\ {\eta }_b=({\overline{I}}_g\left(\mathbf{x}\right)-{\overline{I}}_b\left(\mathbf{x}\right))(1-I_b\left(\mathbf{x}\right)I_g\left(\mathbf{x}\right)).\hfill \end{array}\right.$$

(5)

where, ${\overline{I}}_r\left(\mathbf{x}\right)$, ${\overline{I}}_g\left(\mathbf{x}\right)$, and ${\overline{I}}_b\left(\mathbf{x}\right)$ denote the mean values of degraded color components of underwater image $I_c\left(\mathbf{x}\right)$.

Now, the computational challenge exist in in the derivation of dynamic weight matrices. To overcome this, we employ a lightweight deep neural network, as illustrated in Figure 2b. The network’s architecture encompasses seven convolutional layers; the first six are equipped with 16 filters each, kernel $3\times 3$ and utilizing ReLU activation functions, while the seventh layer adopts a Tanh activation to produce the required weight matrices. This setup is specifically designed to facilitate localized adjustments, empowering the deep line model delineated in Equation (4) to competently address the complex characteristics of underwater images and effectuate precise, adaptive enhancements. Figure 2c exemplifies this capability, depicting the generic behavior of deep lines with varying random parameters $\alpha ,\beta$, spanning a range from -1 to 1. Our model is not only effective but also efficient, comprising a mere 18,390 trainable parameters and requiring only 54.07 MB of memory, rendering it an optimal solution for resource-constrained systems.

3.3. Deep Curve Model

The images that are not restored through linear operations require non-linear transformations. Inspired by the work [28] for low light image enhancement, we propose a deep curve model for underwater image restoration. A second-order polynomial is a simple non-linear mapping between an input image $I_c\left(\mathbf{x}\right)$ and the output image ${\widehat{I}}_c\left(\mathbf{x}\right)$ which is also differential.

$${\widehat{I}}_c\left(\mathbf{x}\right)={\gamma }_{c,1}\left(\mathbf{x}\right)·{\left(I_c\left(\mathbf{x}\right)\right)}^2+{\gamma }_{c,2}\left(\mathbf{x}\right)·I_c\left(\mathbf{x}\right)+{\gamma }_{c,3}\left(\mathbf{x}\right),$$

(6)

where ${\gamma }_{c,1}\left(\mathbf{x}\right),{\gamma }_{c,2}\left(\mathbf{x}\right),{\gamma }_{c,3}\left(\mathbf{x}\right)$ are pixel-wise coefficients for each channel $c\in \{r,g,b\}$. By setting ${\gamma }_{c,1}\left(\mathbf{x}\right)=-{\gamma }_c\left(\mathbf{x}\right),{\gamma }_{c,2}\left(\mathbf{x}\right)=ones\left(\mathbf{x}\right),{\gamma }_{c,3}\left(\mathbf{x}\right)={\gamma }_c\left(\mathbf{x}\right)$, the non-linear mapping can be simplified and re-written as

$${\widehat{I}}_c\left(\mathbf{x}\right)=I_c\left(\mathbf{x}\right)+{\gamma }_c\left(\mathbf{x}\right)·(1-I_c\left(\mathbf{x}\right))$$

(7)

where ${\gamma }_c\left(\mathbf{x}\right)$ are the weight matrices for the non-linear mapping (curves) for each channel $c\in \{r,g,b\}$. It means that the curves are applied separately to each of the three RGB channels allowing for better restoration by preserving the inherent color and by reducing the risk of over-saturation. Further, the image is restored by applying mapping inside the network, so it should be differentiable for forward and back-word passes. While the second-order curves can provide satisfactory restoration results, however, they can further be improved by applying higher-order curves. One simple way to achieve higher-order mapping is to apply second-order mapping iterative fashion. The iterative version of the deep curve model can be expressed as;

$${\widehat{I}}_c^{\left(n\right)}\left(\mathbf{x}\right)={\widehat{I}}_c^{(n-1)}\left(\mathbf{x}\right)+{\gamma }_c^{(n-1)}\left(\mathbf{x}\right)·(1-{\widehat{I}}_c^{(n-1)}\left(\mathbf{x}\right)),$$

(8)

where n indicates the iteration number and ${\gamma }_c^{(n-1)}\left(\mathbf{x}\right)$ are the weight matrices for the $(n-1)th$ iteration. For $n=1$, ${\widehat{I}}_c^{\left(1\right)}\left(\mathbf{x}\right)$ is computed through (6). Further, for n iterations, $n\times 3$ weight matrices are required. In this work, we have set the eight i.e.$n\in \{1,2,...,8\}$, so $(8\times 3=24)$ dynamic weight matrices are to learn. Now, the problem is how to compute dynamic weight matrices.

To learn the weight matrices (curve parameter maps), we adopted a technique similar to that used in the deep line model discussed in the previous section. A lightweight deep neural network, as shown in Figure 2e, is employed to compute these dynamic weight matrices. The network takes the input image $I_c\left(\mathbf{x}\right)$ and learns a set of pixel-wise curve parameter maps corresponding to higher-order curves. The behavior of such curves is illustrated in Figure 2f, for instance, ${\lambda }_3$, with ${\lambda }_1$ and ${\lambda }_2$ set to -1 and the number of iterations n equal to 3, showcasing the advanced adjustment capabilities with these curves. The network’s architecture comprises seven convolutional layers, with the first six layers each containing 32 kernels of size $3\times 3$ with a stride of 1, followed by a ReLU activation function to introduce non-linearity into the model. The final layer consists of 24 convolutional kernels of the same size and stride but employs a Tanh activation function, ensuring that the output values are constrained within the range of -1 to 1. This layer produces a total of 24 curve parameter maps (dynamic weight matrices) across eight iterations, with each iteration providing three curve parameter maps for each channel $c\in \{r,g,b\}$.

4. Results and Discussion

4.1. Implementation Details

In this study, we utilized two primary datasets: EUVP [29] and UIEBD [30]. Both datasets comprise subsets containing paired and unpaired images. We aggregated paired images from subsets of EUVP, including underwater_dark, underwater_scenes, and underwater_imagenet, resulting in a combined total of $11,950$ images. These images were used for both training and testing within our proposed method. Similarly, from the UIEBD dataset, we selected 890 images from the raw-890 subset for the same purpose. For performance evaluation, we used the test_samples subsets from both EUVP and UIEBD, which consist of 515 and 240 images respectively.

In our implementation, we constructed the framework using PyTorch and executed on an NVIDIA GeForce RTX3090 GPU. The proposed method integrates three core models: (1) DenseNet121, which differentiates between Type-I and Type-II images. It utilizes a learning rate of $1\times {10}^{-4}$ and a batch size of 2 across both datasets; (2) DLM targets the enhancement of Type-I images, with configuration settings such as a batch size of 64 and 80 epochs for EUVP, and a batch size of 2 and 200 epochs for UIEBD; (3) DCM focuses on the restoration of Type-II images. This model adopts optimizer configurations similar to the DLM but with a specific batch size of 2 and 100 epochs for EUVP and solely 200 epochs for UIEBD. Throughout their training phases, both the DLM and DCM models employ the RMSE (L2) as loss function. Using the assembled dataset, classifier was trained to categorize the images into two types: Type-I and Type-II. This classification was carried out on $11,435$ images from EUVP and 890 images from UIEBD. As a result, 758 images from EUVP and 240 images from UIEBD were identified as Type-II, while the remaining images were designated as Type-I. The DLM was then trained on $10,677$ Type-I images from EUVP and 500 images from UIEBD. In contrast, the DCM was trained using 758 Type-II images from EUVP and 140 Type-II images from UIEBD. All models utilized the Adam optimizer to ensure efficient convergence. For both training and evaluation, we processed images with dimensions $256\times 256$. Importantly, our settings incorporated gradient clipping, normalized to $0.1$, to prevent gradient explosions, along with a weight decay of $0.0001$ to provide regularization.

4.2. Analysis

The proposed method was tested using 515 images from the EUVP dataset and 240 images from the UIEBD dataset. The proposed classifier designated 486 to Type-I and 29 images to Type-II out of 515 images from the EUVP dataset, setting them up as the testing sets for the DLM and DCM, respectively. Similarly, from the UIEBD dataset, test images classified 213 images in Type-I and 27 images in Type-II. All datasets were provided to each of the proposed DLM and DCM for the restoration. RMSE and PSNR metrics were computed for each image in the datasets and the average of the PSNR and RMSE are shown in Table 1. As DLM is designed for refining Type-I images and DCM is developed for the restoration of Type-II images. From the table, it can be observed that when models are applied on Type-I and Type-II images respectively, improved PSNR and RMSE measures are obtained. Whereas, declined or marginally improved metrics are obtained when models are applied on Type-II and Type-I images respectively. Hence, DLM and DCM are effective for the restoration of Type-I and Type-II images respectively.

In addition, for qualitative analysis of DLM and DCM, we selected two images from each Type, and their restored versions are shown in Figure 3. From the analysis of the figure, it is evident that the DLM performs better for Type-I images in both datasets, yielding restored images that are closer to the ground truth (GT). However, when the DCM is applied to Type-I images, although certain areas appear clearer, there is a tendency for colors to become denser. For instance, the blue color intensifies, resulting in a more bluish appearance of the image. Conversely, Type-II images restored using DCM for both datasets exhibit a cleaner look and are more closely aligned with the GT, whereas DLM shows suboptimal performance, either causing color distortion or producing blurry images.

In our detailed examination of the Deep Curve Model (DCM) performance, we observed a consistent enhancement in the restoration quality with increasing iterations. Utilizing two representative images from the UIEBD dataset, as illustrated in Figure 4, we record the progression of quality improvements. For instance, as shown in Figure 4b, the restoration quality at the fourth iteration (I4) manifests a considerable enhancement from the input, evidenced by the RMSE/PSNR values of 0.09/20.95. This trend of enhancement persists through iterations I4 and I6, as highlighted by the corresponding RMSE/PSNR figures, which signify improved clarity and overall quality of the dehazed images. The peak of visual clarity is achieved at iteration I8, registering the lowest RMSE of 0.06 and the highest PSNR of 24.66 for the top image, with the bottom image exhibiting similarly positive metrics. Notably, limiting the model to merely one or two iterations does not invariably lead to inferior dehazing outcomes; in some cases, image quality may remain stable or even slightly enhance, suggesting that the optimal number of iterations for image enhancement is variable. The correlation between the iteration count and the quality of dehazing is evident, with higher iterations yielding superior visual and quantitative results.

4.3. Comparative Analysis

In order to rigorously assess the efficacy of our proposed approach, we contrasted it against leading-edge methods in the domain. This encompasses non-learning techniques such as NLD [7], RLP [31], MMLE [10], and UNTV [8], as well as learning-driven paradigms including ACT [32], FGAN [33], DNet [12], AOD [14], and SCNet [34]. First, input images are restored through the above-mentioned methods and the proposed method, and then restoration accuracy is compared by utilizing the widely accepted quantitative metrics, Root Mean Square Error (RMSE) and Peak Signal-to-Noise Ratio (PSNR). A Lower value for RMSE and a higher value for PSNR indicates better results. Table presents the quantitative results for test samples across both datasets. The performance of the proposed networks for the EUVP dataset is denoted as “Ours”. Focusing on the EUVP dataset, a quick examination of Table 1 reveals that DLM exhibits superior performance in Type-I images, registering a RMSE of 0.08 and a PSNR of 22.30dB, which outperforms state-of-the-art (SOTA) methods. Similarly, for Type-II images, DCM surpasses competing methods by achieving 0.06 RMSE and 25.00dB PSNR. Notably, when contrasted with SCNet and ACT — the models boasting the second-best performance for Type-I and Type-II images respectively — DLM excels with a differential of 0.02 in RMSE and 2.3dB in PSNR, whereas DCM showcases a marked improvement with 0.87 RMSE and 4.4dB PSNR. Transitioning to the UIEBD dataset, SCNet leads in performance for Type-I images, registering 0.07 RMSE and 23.60dB PSNR. This is contrasted with DLM, which holds the position for the second-best performance, achieving 0.11 RMSE and 20.10dB PSNR. Conversely, for Type-II images, DCM stands out by attaining 0.08 RMSE and 22.20dB PSNR. When compared with SCNet — the runner-up in performance — our DCM establishes a superior benchmark with a difference of 0.02 in RMSE and an elevation of 1.5dB in PSNR.

Table 2. Comparison of Average RMSE and PSNR for Non-Learning & Learning-Based Methods on EUVP and UIEBD Test Images. The best results are in bold and the second best results are with underline.

Non-learning-based Methods

Learning-based Methods

Dataset

Type

Input

NLD

RLP

MMLE

UNTV

ACT

D-Net

AOD

F-GAN

SCNet

Ours

EUVP

Type-I

0.11/20.00

0.15/17.00

0.15/17.30

0.18/15.30

0.14/17.40

0.84/17.50

0.12/19.10

0.78/15.10

0.73/13.70

0.10/20.00

0.08/22.30

Type-II

0.08/22.00

0.12/18.40

0.15/16.90

0.17/15.60

0.11/19.20

0.93/20.60

0.10/20.10

0.87/13.30

0.84/15.50

0.11/19.70

0.06/25.00

UIEBD

Type-I

0.15/17.60

0.18/15.30

0.14/17.50

0.21/14.30

0.16/16.20

0.81/15.50

0.17/15.90

0.75/15.10

0.70/12.70

0.07/23.60

0.11/20.10

Type-II

0.14/18.60

0.12/18.70

0.11/19.20

0.24/12.60

0.17/15.60

0.92/19.70

0.14/17.80

0.86/13.10

0.53/9.99

0.10/20.70

0.08/22.20

Table 2. Comparison of Average RMSE and PSNR for Non-Learning & Learning-Based Methods on EUVP and UIEBD Test Images. The best results are in bold and the second best results are with underline.

Non-learning-based Methods

Learning-based Methods

Dataset

Type

Input

NLD

RLP

MMLE

UNTV

ACT

D-Net

AOD

F-GAN

SCNet

Ours

EUVP

Type-I

0.11/20.00

0.15/17.00

0.15/17.30

0.18/15.30

0.14/17.40

0.84/17.50

0.12/19.10

0.78/15.10

0.73/13.70

0.10/20.00

0.08/22.30

Type-II

0.08/22.00

0.12/18.40

0.15/16.90

0.17/15.60

0.11/19.20

0.93/20.60

0.10/20.10

0.87/13.30

0.84/15.50

0.11/19.70

0.06/25.00

UIEBD

Type-I

0.15/17.60

0.18/15.30

0.14/17.50

0.21/14.30

0.16/16.20

0.81/15.50

0.17/15.90

0.75/15.10

0.70/12.70

0.07/23.60

0.11/20.10

Type-II

0.14/18.60

0.12/18.70

0.11/19.20

0.24/12.60

0.17/15.60

0.92/19.70

0.14/17.80

0.86/13.10

0.53/9.99

0.10/20.70

0.08/22.20

To assess the robustness of the proposed method for qualitative evaluation, we selected six images from each of the dataset for type-I and type-II categories and the visual outcomes from established approaches and the proposed method shown in Figure 5. It can be observed from the figure that methods such as MMLE tend to over-darken certain areas in their dehazing results. Additionally, techniques like NLD, RLP, and UNTV exhibit notable color distortions and texture degradation. Meanwhile, ACT, D-Net, and AOD-Net struggle with remaining haze effects. F-GAN’s outcomes lean excessively toward reddish tones, and while SCNet shows an improvement over previous methods, especially for type-I images, still presents slight color distortions and tends to produce overly bright images. Remarkably, ACT performs commendably on type-II images when compared to other competitors. In contrast, our proposed method excels by restoring finer details and achieves more visually appealing restorations, outperforming both traditional and learning-based counterparts.

Similarly, to further verify the effectiveness of our approach, we extended our comparison to the six randomly selected images from the UIEBD dataset, representing both Type-I and Type-II categories. The restored images for the comparative methods along with the proposed method are presented in Figure 6. It can be observed from the figure that prevalent methods such as UDCP, ACT, D-Net, and AOD-Net struggle with lasting haze. MMLE and UNTV, in particular, introduce significant color distortions, to preserve texture details and edge sharpness, whereas F-GAN tends to bias the image restoration towards a reddish tint. RLP and SCNet offer superior visual clarity compared to their counterparts, yet their dehazed images exhibit excessive brightness when compared to our model. In contrast, our method not only restores natural colors and sharp edges but also excels in processing Type-II images, consistently surpassing both conventional and learning-based algorithms.

Further, Table enumerates the quantitative results for images depicted in Figure 5 and Figure 6. For both datasets, the performance of the proposed method is indicated as “Ours”. In examining the EUVP dataset for Type-I images, DLM consistently showcases superior performance across all images, registering a minimum RMSE of 0.04 and a PSNR of 28.30dB. Similarly, for Type-II images, DCM emerges as the top performer across all images, notching a minimum RMSE of 0.03 and a PSNR of 29.90dB. Transitioning to the UIEBD dataset, for Type-I images, DLM stands out in performance for 2 out of 3 images, achieving a minimum RMSE of 0.05 and a PSNR of 26.90dB. For Type-II images, DCM exhibits top-tier performance in 2 out of 3 images, recording a minimum RMSE of 0.04 and a PSNR of 29dB. In a comprehensive analysis, both DLM and DCM prove their efficacy across both datasets, outperforming in 10 out of 12 images. In comparison to other methods, with RLP being the second-best performer, it excelled in 4 out of 12 images.

Table 3. Comparison of Average RMSE and PSNR for non-learning and learning-based methods on EUVP and UIEBD test images. The best results are in bold and the second best results are with underline.

Non-learning-based Methods

Learning-based Methods

Dataset

Image

Input

NLD

RLP

MMLE

UNTV

ACT

D-Net

AOD

F-GAN

SCNet

Ours

EUVP

test_p84_

0.11/19.30

0.16/15.70

0.20/15.10

0.14/17.33

0.13/17.52

0.14/17.00

0.11/18.90

0.18/15.16

0.17/15.50

0.17/15.62

0.05/26.70

Type-I

test_p404_

0.14/17.20

0.18/15.00

0.20/16.00

0.11/18.79

0.16/15.96

0.18/14.99

0.14/17.10

0.17/15.24

0.19/14.60

0.20/13.96

0.04/28.30

test_p510_

0.07/22.70

0.09/21.10

0.10/22.20

0.13/17.48

0.09/20.77

0.10/20.11

0.10/20.40

0.21/13.73

0.21/13.60

0.13/17.55

0.06/24.50

EUVP

test_p171_

0.07/23.70

0.09/20.70

0.10/19.60

0.17/15.61

0.10/20.38

0.08/22.04

0.10/19.90

0.25/12.13

0.17/15.20

0.10/20.22

0.04/27.70

Type-II

test_p255_

0.05/26.50

0.11/18.90

0.40/8.45

0.30/10.45

0.12/18.61

0.06/24.88

0.05/26.40

0.16/15.73

0.21/13.50

0.06/24.27

0.03/29.90

test_p327_

0.07/23.70

0.10/20.20

0.20/16.20

0.16/15.68

0.10/20.24

0.09/21.18

0.09/20.50

0.21/13.44

0.18/15.00

0.11/18.99

0.05/26.10

UIEBD

375_img_

0.09/20.80

0.11/19.50

0.14/16.80

0.15/16.54

0.10/20.10

0.11/19.40

0.11/19.30

0.18/15.01

0.17/15.40

0.17/15.57

0.07/22.90

Type-I

495_img_

0.14/18.10

0.20/15.40

0.20/14.50

0.18/15.11

0.14/17.51

0.19/14.62

0.14/17.20

0.20/13.94

0.20/14.00

0.20/13.80

0.04/27.00

619_img_

0.07/22.70

0.09/21.10

0.10/22.20

0.13/17.49

0.09/20.78

0.10/20.12

0.10/20.41

0.21/13.74

0.21/13.61

0.13/17.56

0.06/24.51

UIEBD

746_img_

0.07/23.70

0.09/20.70

0.10/19.60

0.17/15.62

0.10/20.39

0.08/22.05

0.10/19.91

0.25/12.14

0.17/15.21

0.10/20.23

0.04/27.71

Type-II

845_img_

0.05/26.50

0.11/18.90

0.40/8.46

0.30/10.46

0.12/18.62

0.06/24.89

0.05/26.41

0.16/15.74

0.21/13.51

0.06/24.28

0.03/29.91

967_img_

0.07/23.70

0.10/20.20

0.20/16.21

0.16/15.69

0.10/20.25

0.09/21.19

0.09/20.51

0.21/13.45

0.18/15.01

0.11/19.00

0.05/26.11

Table 3. Comparison of Average RMSE and PSNR for non-learning and learning-based methods on EUVP and UIEBD test images. The best results are in bold and the second best results are with underline.

Non-learning-based Methods

Learning-based Methods

Dataset

Image

Input

NLD

RLP

MMLE

UNTV

ACT

D-Net

AOD

F-GAN

SCNet

Ours

EUVP

test_p84_

0.11/19.30

0.16/15.70

0.20/15.10

0.14/17.33

0.13/17.52

0.14/17.00

0.11/18.90

0.18/15.16

0.17/15.50

0.17/15.62

0.05/26.70

Type-I

test_p404_

0.14/17.20

0.18/15.00

0.20/16.00

0.11/18.79

0.16/15.96

0.18/14.99

0.14/17.10

0.17/15.24

0.19/14.60

0.20/13.96

0.04/28.30

test_p510_

0.07/22.70

0.09/21.10

0.10/22.20

0.13/17.48

0.09/20.77

0.10/20.11

0.10/20.40

0.21/13.73

0.21/13.60

0.13/17.55

0.06/24.50

EUVP

test_p171_

0.07/23.70

0.09/20.70

0.10/19.60

0.17/15.61

0.10/20.38

0.08/22.04

0.10/19.90

0.25/12.13

0.17/15.20

0.10/20.22

0.04/27.70

Type-II

test_p255_

0.05/26.50

0.11/18.90

0.40/8.45

0.30/10.45

0.12/18.61

0.06/24.88

0.05/26.40

0.16/15.73

0.21/13.50

0.06/24.27

0.03/29.90

test_p327_

0.07/23.70

0.10/20.20

0.20/16.20

0.16/15.68

0.10/20.24

0.09/21.18

0.09/20.50

0.21/13.44

0.18/15.00

0.11/18.99

0.05/26.10

UIEBD

375_img_

0.09/20.80

0.11/19.50

0.14/16.80

0.15/16.54

0.10/20.10

0.11/19.40

0.11/19.30

0.18/15.01

0.17/15.40

0.17/15.57

0.07/22.90

Type-I

495_img_

0.14/18.10

0.20/15.40

0.20/14.50

0.18/15.11

0.14/17.51

0.19/14.62

0.14/17.20

0.20/13.94

0.20/14.00

0.20/13.80

0.04/27.00

619_img_

0.07/22.70

0.09/21.10

0.10/22.20

0.13/17.49

0.09/20.78

0.10/20.12

0.10/20.41

0.21/13.74

0.21/13.61

0.13/17.56

0.06/24.51

UIEBD

746_img_

0.07/23.70

0.09/20.70

0.10/19.60

0.17/15.62

0.10/20.39

0.08/22.05

0.10/19.91

0.25/12.14

0.17/15.21

0.10/20.23

0.04/27.71

Type-II

845_img_

0.05/26.50

0.11/18.90

0.40/8.46

0.30/10.46

0.12/18.62

0.06/24.89

0.05/26.41

0.16/15.74

0.21/13.51

0.06/24.28

0.03/29.91

967_img_

0.07/23.70

0.10/20.20

0.20/16.21

0.16/15.69

0.10/20.25

0.09/21.19

0.09/20.51

0.21/13.45

0.18/15.01

0.11/19.00

0.05/26.11

4.4. Limitations

Though the proposed framework typically yields improved restoration outcomes, its efficacy can be compromised by the misclassification of images. Such misclassifications are more likely when images possess mixed characteristics or when the changes in their characteristics are slight. These occurrences underline the need to refine the classifier’s accuracy to ensure dependable performance in real-world applications. An improved classifier can be designed by cultivating better image labeling procedures and exploiting deep image features.

5. Conclusions

In this paper, we presented an underwater image restoration method that initially, categorizes input images into Type-I or Type-II. Afterward, based on the classification, the DLM is applied to restore Type-I images, while the DCM is used for the restoration of Type-II images. Both models utilize lightweight neural networks for learning per-pixel weights based on the input image’s characteristics. Experimental results and comparative analysis demonstrate the efficacy of the proposed method.