Underwater Image Restoration and Enhancement: A Comprehensive Review of Recent Trends, Challenges, and Applications

1. Introduction

An underwater environment is any area immersed in water, such as the ocean floor, a reservoir, a basin, and a riverbed. Such environments are also found in lakes, ponds, dams, canals, and even aquifers. Underwater environments are important because water covers almost 71 % of the earth’s surface and provides [1] natural habitats for most living organisms. In addition, they are considered a potential resource for the extraction of various minerals, such as silver, gold, copper, manganese, and zinc. Therefore, exploring, developing, and protecting underwater resources have become active research topics.

The clear interpretation and analysis of underwater videos and images offer important and valuable information about the underwater world. They are important for domains such as underwater archaeology, marine ecological research, naval military applications, and telecommunication cable handling [2]. Consequently, the processing and analysis of underwater images are crucial in the research on developing, exploring, and protecting underwater resources [3,4,5,6,7,8,9].

During image acquisition, the poor visibility conditions in an underwater environment reduce the obtained image quality, resulting in highly degraded, low-contrast, and noisy images. This limits its use in many practical scenarios. Two solutions for obtaining clear underwater images are available. One requires expensive specialized image acquisition hardware and the other applies image preprocessing techniques for image enhancement and restoration allowing the generated image to be better displayed when used in various applications.

Underwater images are captured using diverse methods. The moorings and buoys method is used to monitor the water quality, the sea earth’s circumstances, and microorganisms in the water. The basics of this method are to use cameras mounted on remotely operated vehicles (ROVs), unmanned underwater vehicles (UUVs), autonomous underwater vehicles (AUVs), or an ocean sensor network [10,11,12]. For better image quality, these vehicles are equipped with sensors, such as GPSs and cameras to collect information about subaquatic minerals, coral reef ecosystems, or the deep sea habitat. Figure 1 presents a selection of the types of equipment used in collecting and observing ocean data.

Figure 1. Concept map of the ocean observation network.

Figure 1. Concept map of the ocean observation network.

There are other image capturing methods that depend on sonar and their quality depends on the wavelength of the sounds used. Sonar emits these, then capture the underwater sound reflections and converts them into images [13]. Using this image-capturing method helps researchers to study underwater images efficiently.

Due to the above-mentioned circumstances with the underwater image acquisition process, the obtained images must be preprocessed to better display during underwater image analysis. This is accomplished by developing new underwater image processing and computer vision techniques [14]. Computer vision algorithms can effectively analyze and interpret underwater visual data, but this is restricted by the limited visibility conditions resulting in low contrast and noisy images. Preprocessing techniques are required to avoid these challenges and obtain clear, high-quality images. Image preprocessing techniques are classified into two main groups; underwater image restoration and underwater image enhancement [15].

The implementation of techniques for underwater image restoration mainly depends on physical models. These physical models are important for many tasks, such as building the underwater image degradation model, computing parameters for the model (e.g., diffusion, attenuation, or water turbidity coefficients), and tackling the inverse problem. Performing these tasks requires prior knowledge and assumptions about the environmental conditions. Mathematical models can be used to estimate the model parameters, but they are very complicated and computationally challenging. Conversely, the underwater image enhancement techniques for providing clear images of suitable quality are based on qualitative criteria. These techniques can improve image color and contrast much more simply and quickly, without using physical models.

Many reviews on underwater image enhancement and restoration have been published. However, these investigations focus on specific aspects related to underwater image analysis. [16] presented a brief survey on underwater image enhancement. Other surveys such as [6,12,15,17], reviewed many methods for enhancing and restoring underwater images, but focused only on the techniques used, their limitations, quality assessment measures, and future directions. More recent surveys have been published such as those by [18,19,20]; these concentrate only on the methods used for underwater enhancement while ignoring restoration. Finally, the survey presented by [21] lacks a complete discussion of existing enhancement methods besides quality evaluation metrics. Although these reviews have been produced, they do not comprehensively discuss several issues. They present incomplete classifications of enhancement and restoration techniques and ignore the latest developments with deep learning techniques. They also lack a clear discussion and investigation about how to increase the images quality. Therefore, this survey intends to review the most prominent approaches for underwater image restoration and enhancement techniques and overcome the previously listed limitations.

The following are the basic contributions of this survey:

• The basic concepts related to the underwater environment, including image formation and light degradation models, are explained.
• Recent underwater image enhancement and restoration methods are comprehensively discussed to identify their working methodologies, strengths, and limitations.
• The datasets applied for improving underwater image analysis and the existing evaluation metrics are discussed and compared.
• Different enhancement and restoration techniques are experimentally evaluated by using images from underwater images datasets.
• The main limitations that researchers face in underwater image analysis are summarized. These limitations are classified into two categories: those related to the underwater environment and those related to the underwater images.
• Several open issues for underwater image enhancement and restoration are presented to highlight potential future research directions.

The remainder of this survey is organized as follows. Section 2 indicates the methodology implemented in this study, such as the keywords used for the searches, data sources, and criteria for selecting, including, and excluding articles. Section 3 presents the background of imaging in subaquatic environments, identifying the image formation model and types of light degradation. Section 4 introduces the classification of the underwater image processing methods into two main categories (enhancement and restoration) and presents a review of important previous studies. Section 5 presents the techniques used for underwater image analysis. Section 6 highlights the limitations faced by researchers in this field. Section 7 provides a comparison of the existing underwater imaging datasets. Section 8 presents the metrics used for evaluating the quality of underwater imaging techniques. Section 9 details performance evaluation for the qualitative, quantitative, and computational time assessment. Section 10 presents a discussion on several applications in the field of underwater image enhancement and restoration. Section 11 elaborates on future research directions. Finally, the conclusion is presented. Figure 2 shows the survey structure and Table 1 provides all the abbreviations used in this survey.

Table 1. Used abbreviations.

Abbreviation	Definition	Abbreviation	Definition
AD	Average Difference	AG	Average Gradient
AHE	Adaptive Histogram Equalization	AMBE	Absolute Mean Brightness Error
AUVs	Autonomous Underwater Vehicles	BBHE	Brightness Preserving Bi-Histogram Equalization
CCF	Colourfulness Contrast Fog density index	CEF	Colour Enhancement Factor
CRBICMRD	Color Restoration depended on the Integrated Color Model with Rayleigh Distribution	CLAHE	Contrast Limited Adaptive Histogram Equalization
CNN	Convolutional Neural Network	CNR	Contrast to Noise Ratio
DCP	Dark Channel Prior	DCT	Discrete Cosine Transform
DOP	Degrees of Polarization	DL	Deep Learning
DSNMF	Deep Sparse Non-negative Matrix Factorization	DWT	Discrete Wavelet Transform
EAs	Evolutionary Algorithms	EME	Measure of Enhancement
EMEE	Measure of Enhancement by Entropy	EUVP	Enhancement of Underwater Visual Perception
FR	Full Reference	GANs	Generative Adversarial Networks
GUM	Generalized Unsharp Masking	HE	Histogram Equalization
HIS	Hue-Saturation-Intensity	HR	High Resolution
HSV	Hue-Saturation Value	HVS	Human Visual System
IEM	Image Enhancement Metric	ICM	Integrated Color Model
IFM	Image Formation Model	JTF	Joint Trigonometric Filtering
LFR	Light Field Rendering	MARI	Marine Autonomous Robotics for Interventions
MAI	Maximum Attenuation Identification	MCM	Multi-Color Model
MD	Maximum Difference	MILP	Minimum Information Loss Principal
MIP	Maximum Intensity Prior	MLP	Multilayer Perceptron
MSRCR	Multiscale Retinox with Color Restoration	MSE	Mean Square Error
MTF	Modulation Transfer Function	NAE	Normalized Absolute Error
NCC	Normalized Cross-Correlation	NIQA	Natural Image Quality Assessment
NR	No Reference	NR-IQA	No-referenced Image Quality Metric
PCQI	Patch based Contrast Quality Index	PDI	Polarization Differential Imaging
PSF	Point Spread Function	PSNR	Peak-Signal-to-Noise Ratio
PSO	Particle Swarm Optimization	RAHIM	Recursive Adaptive Histogram Modification
RCP	Red Channel Prior	RGB	Red-Green-Blue
RGHS	Relative Global Histogram Stretching	RIP	Range Intensity Profile
RMSE	Root Mean Square Error	RNN	Recurrent Neural Networks
ROVs	Remotely Operated Vehicles	RR	Reduced Reference
RUIE	Real-World Underwater Image Enhancement	SAUV	Sampling System-AUV
SCM	Single Color Model	SNR	Signal-to-Noise Ratio
SR	Super-Resolution	SSIM	Structure Similarity Index Measure
SSEQI	Spatial Spectral Entropy based Quality index	SVM	Support Vector Machine
TM	Transmission Map	UCIQE	Underwater Colour Image Quality Evaluation metric
UDCP	Underwater Dark Channel Prior	UHTS	underwater task-oriented test suite
UIEB	Underwater Image Enhancement Benchmark	UIE	Underwater Image Enhancement
UIQS	Underwater Image Quality Set	UIEB	Underwater Image Enhancement Benchmark
UIQM	Underwater Image Quality Measure	UISM	Underwater Image Sharpness Measurement
UICM	Underwater Image Color Measurement	ULAP	Underwater Light Attenuation Prior
UIConM	Underwater Image Contrast Measurement	UOI	Underwater Optical Imaging
UUVs	Unmanned Underwater Vehicles	WCID	Wavelength Compensation and Image Defogging

Table 1. Used abbreviations.

Abbreviation	Definition	Abbreviation	Definition
AD	Average Difference	AG	Average Gradient
AHE	Adaptive Histogram Equalization	AMBE	Absolute Mean Brightness Error
AUVs	Autonomous Underwater Vehicles	BBHE	Brightness Preserving Bi-Histogram Equalization
CCF	Colourfulness Contrast Fog density index	CEF	Colour Enhancement Factor
CRBICMRD	Color Restoration depended on the Integrated Color Model with Rayleigh Distribution	CLAHE	Contrast Limited Adaptive Histogram Equalization
CNN	Convolutional Neural Network	CNR	Contrast to Noise Ratio
DCP	Dark Channel Prior	DCT	Discrete Cosine Transform
DOP	Degrees of Polarization	DL	Deep Learning
DSNMF	Deep Sparse Non-negative Matrix Factorization	DWT	Discrete Wavelet Transform
EAs	Evolutionary Algorithms	EME	Measure of Enhancement
EMEE	Measure of Enhancement by Entropy	EUVP	Enhancement of Underwater Visual Perception
FR	Full Reference	GANs	Generative Adversarial Networks
GUM	Generalized Unsharp Masking	HE	Histogram Equalization
HIS	Hue-Saturation-Intensity	HR	High Resolution
HSV	Hue-Saturation Value	HVS	Human Visual System
IEM	Image Enhancement Metric	ICM	Integrated Color Model
IFM	Image Formation Model	JTF	Joint Trigonometric Filtering
LFR	Light Field Rendering	MARI	Marine Autonomous Robotics for Interventions
MAI	Maximum Attenuation Identification	MCM	Multi-Color Model
MD	Maximum Difference	MILP	Minimum Information Loss Principal
MIP	Maximum Intensity Prior	MLP	Multilayer Perceptron
MSRCR	Multiscale Retinox with Color Restoration	MSE	Mean Square Error
MTF	Modulation Transfer Function	NAE	Normalized Absolute Error
NCC	Normalized Cross-Correlation	NIQA	Natural Image Quality Assessment
NR	No Reference	NR-IQA	No-referenced Image Quality Metric
PCQI	Patch based Contrast Quality Index	PDI	Polarization Differential Imaging
PSF	Point Spread Function	PSNR	Peak-Signal-to-Noise Ratio
PSO	Particle Swarm Optimization	RAHIM	Recursive Adaptive Histogram Modification
RCP	Red Channel Prior	RGB	Red-Green-Blue
RGHS	Relative Global Histogram Stretching	RIP	Range Intensity Profile
RMSE	Root Mean Square Error	RNN	Recurrent Neural Networks
ROVs	Remotely Operated Vehicles	RR	Reduced Reference
RUIE	Real-World Underwater Image Enhancement	SAUV	Sampling System-AUV
SCM	Single Color Model	SNR	Signal-to-Noise Ratio
SR	Super-Resolution	SSIM	Structure Similarity Index Measure
SSEQI	Spatial Spectral Entropy based Quality index	SVM	Support Vector Machine
TM	Transmission Map	UCIQE	Underwater Colour Image Quality Evaluation metric
UDCP	Underwater Dark Channel Prior	UHTS	underwater task-oriented test suite
UIEB	Underwater Image Enhancement Benchmark	UIE	Underwater Image Enhancement
UIQS	Underwater Image Quality Set	UIEB	Underwater Image Enhancement Benchmark
UIQM	Underwater Image Quality Measure	UISM	Underwater Image Sharpness Measurement
UICM	Underwater Image Color Measurement	ULAP	Underwater Light Attenuation Prior
UIConM	Underwater Image Contrast Measurement	UOI	Underwater Optical Imaging
UUVs	Unmanned Underwater Vehicles	WCID	Wavelength Compensation and Image Defogging

Figure 2. Survey’s structure.

Figure 2. Survey’s structure.

2. Research Methodology

This section describes the protocols used to examine different methods and techniques proposed for solving underwater image analysis problems during 2006–2022. The search keywords, data sources, inclusion/exclusion criteria, and article selection criteria are discussed. Table 2 presents the frequency of using the techniques proposed for underwater image analysis classified into three different classes.

Table 2. Technique type: Analysis based on frequency.

No.	Method type	Method frequency %
1	Hardware-based Methods	10%
2	Underwater Image Restoration	30%
3	Underwater Image enhancement	60%

Table 2. Technique type: Analysis based on frequency.

No.	Method type	Method frequency %
1	Hardware-based Methods	10%
2	Underwater Image Restoration	30%
3	Underwater Image enhancement	60%

2.1. Search keywords

The keywords were carefully selected for the initial search. Then, many new words found in numerous related articles were used to compile a keyword selection. The main keywords used in many studies include underwater image analysis, underwater image enhancement, underwater image restoration, underwater datasets, and underwater image quality evaluation. Our understanding of the topic facilitated the selection of other keywords, such as color enhancement, light correction method, color correction, dark channel prior, deep learning, image dehazing, scattering, and absorption.

2.2. Data sources

Our survey included searching various academic databases to collect the articles, as indicated in Table 3.

Table 3. Academic databases selected for research in this survey.

Academic database name	Link
Science direct	http://www.sciencedirect.com/
Web of science	https://apps.webofknowledge.com/
PubMed	https://pubmed.ncbi.nlm.nih.gov/
IEEEXplore	https://ieeexplore.ieee.org/
Springerlink	https://link.springer.com/
PeerJ	https://peerj.com/
Scopus	https://www.scopus.com/

Table 3. Academic databases selected for research in this survey.

Academic database name	Link
Science direct	http://www.sciencedirect.com/
Web of science	https://apps.webofknowledge.com/
PubMed	https://pubmed.ncbi.nlm.nih.gov/
IEEEXplore	https://ieeexplore.ieee.org/
Springerlink	https://link.springer.com/
PeerJ	https://peerj.com/
Scopus	https://www.scopus.com/

2.3. Article Inclusion/Exclusion Criteria

Based on our research goal, the inclusion/exclusion criteria were chosen to determine which publications were suitable for the next review stage. Research relevance was assumed for articles that meet the inclusion criteria and excluded articles that do not fulfill the inclusion criteria. The inclusion/exclusion criteria set is presented in Table 4.

Table 4. Article inclusion and exclusion criteria.

Inclusion Criteria	Exclusion Criteria
Our survey only concentrates on articles on underwater image analysis and processing techniques. Only articles concerned with processing underwater images. Only articles and research in English were taken into account.	Articles not on underwater image analysis and processing techniques are excluded. Articles not focused on any other types of imaging but only underwater imaging. Articles that were not written in English were excluded.

Table 4. Article inclusion and exclusion criteria.

Inclusion Criteria	Exclusion Criteria
Our survey only concentrates on articles on underwater image analysis and processing techniques. Only articles concerned with processing underwater images. Only articles and research in English were taken into account.	Articles not on underwater image analysis and processing techniques are excluded. Articles not focused on any other types of imaging but only underwater imaging. Articles that were not written in English were excluded.

2.4. Article Selection

Inclusion and exclusion criteria were created to choose which articles were suitable for the review phase. The articles under inclusion criteria articles were considered to be related to the research, and those not meeting the inclusion criteria were excluded. The list of inclusion/exclusion criteria has been discussed in the previous section. Choosing an article for this research was a three-phase process. The first phase extracted only the titles, abstracts, and keywords of the articles. The second phase discussed the abstract, introduction, and conclusion to modify the choices from the first phase. In the final phase, the articles were perused, and thereafter, the article’s quality was evaluated according to its research relevance.

3. Basic Concepts

Life is believed to have originated in the oceans, and at present, the underwater environment is the natural habitat for most living organisms. In the accessible areas of the underwater environment, various human activities are conducted. The underwater environment is explored using underwater images that have been analyzed by applying computer vision and image processing techniques. When analyzing underwater images in computer vision, a critical and fundamental difference between images taken in water and air must be considered.

First, the light rays are attenuated and scattered as they travel through the water body. The former leads to a loss of photons while the latter leads to a gain of photons [22]. Both effects are wavelength dependent and therefore affect image coloration by producing bluish/greenish tints in underwater images. Second, light rays are refracted at the water-air interface of the camera housing, generating geometric distortions in the image. Therefore, as introduced in the following subsection, it is essential to discuss the characteristics of the [23] underwater image model to improve underwater image analysis.

3.1. Scattering

Underwater light scattering occurs when dust particles are present. When the refracted light from the object outside reaches the camera, it combines with the floating particles in the imaging medium, causing a scattering effect. Two forms of scattering affect underwater images; forward and backward scattering [12,24,25]. When light reflected from an object is scattered on its way to the camera, it is termed forward scattering. In contrast, backscattering happens when reflected light reaches the camera immediately before reaching the lighted scene. Forward scattering results in blurred images, while backscattering causes effects such as low contrast and hazy in the image [24].

3.2. Underwater Image Model

Jaffe-McGlamery is an imaging model for underwater image enhancement that depends on physical models [13,26]. This model was developed as a simulator for designing underwater image systems and evaluating the use of computer vision algorithms. Therefore, the model was adopted to incorporate several factors, such as light sources, color, and shadows. It is also based on realistically modeling the water medium and linear superposition. When the irradiance enters the camera, it contains a linear combination of three different contents: the direct component $E_D$, the forward-scattered component $E_F$, and the backscattered component $E_B$. Hence, the total irradiance $E_T$ is computed by:

$$\begin{array}{cc}\hfill E_T\left(Total\right)& =E_D\left(direct\right)+E_F\left(fowardScatter\right)+\hfill \\ \hfill & E_B\left(backScatter\right)\hfill \end{array}$$

(1)

where $E_D\left(direct\right)$ is the light reflected by the object and reaches the camera without being scattered, $E_F\left(fowardScatter\right)$ is forward scattering, and $E_B\left(backScatter\right)$ is backscattering. This model is widely applied for image restoration and requires complex computations and a longer execution time [27,28]. If the distance between the used camera and the underwater scene is very small, forward scattering can be eliminated, and only the background scattering and direct transmission are considered [29,30,31,32,33].

The simplified Image Formation Model (IFM) is a typical and effective model for restoring underwater images. It is computed by Equation 2.

$$I^c\left(x\right)=J^c\left(x\right)t^c\left(x\right)+B^c(1-t^c\left(x\right))$$

(2)

where I is the camera, J is the underwater scene, t is the residual energy ratio, x is a particular $point(i,j)$ on the image scene, c is a channel from RGB channels, $J^c\left(x\right)t^c\left(x\right)$ is the direct transmission, and $B^c(1-t^c\left(x\right))$ is the background scattering.

3.3. Underwater Light Degradation

The empirical Lambert-Beer law states that "The Decline in light intensity is based on the properties of the medium through which the light travels" [13]. As per this law, the intensity of light to create underwater images decays exponentially when traveling through water. This intensity loss is called attenuation. The absorption effects make the light to lose energy, while scattering causes a change in the electromagnetic energy direction. The absorption and scattering phenomenon leads to light attenuation [34].

Light attenuation is a major concern when dealing with underwater imaging as it causes the hazy effect that makes image processing applications difficult. It limits visibility to about 20m in clear water and 5m in murky water [33]. The light absorption in water changes by wavelength. As shown in Figure 3, the colors in the visible spectrum disappear as the water depth increases. Red light is absorbed first because of its longer wavelength. Due to its shorter wavelength, blue penetrates the deepest, leaving a bluish hue in underwater images [6,13].

Figure 3. Underwater color reduction.

Figure 3. Underwater color reduction.

4. Classification of Underwater Image Processing Techniques

Due to the increasing demand for clear good-quality images for understanding and analyzing the real-life underwater environment, many studies have discussed the analysis of underwater images. As mentioned, underwater image processing is classified into two main classes: image restoration and enhancement. The main difference between these classes is that image restoration is based on the original IFM, but image enhancement is not.

In this section, The current studies related to underwater image processing are presented. These are classified into three main classes, image restoration, enhancement, and a fusion of both. Then, each class is divided into its corresponding sub-classes as shown in Figure 4.

Figure 4. Taxonomy of underwater image analysis techniques.

Figure 4. Taxonomy of underwater image analysis techniques.

4.1. Underwater Image Restoration Techniques

The underwater image restoration method depends on physical models. It builds the physical model by understanding the physical image degradation mechanism and the core physics of light propagation. Then, it deduces the basic physical model parameters using prior knowledge and finally restores the restored image [35]. The simplified IFM is identified by Equation 2, which is a typical and effective underwater image restoration model. Underwater image restoration is classified into two groups: hardware-based restoration and software-based restoration. Hardware-based restoration is subdivided into three groups: polarization characteristic-based, stereo imaging, and range-gated imaging. Whereas software-based restoration is subdivided into three groups: optical image-based, prior knowledge-based, and deep learning-based restoration techniques. Table 5 presents a comparison of these underwater image restoration methods.

Table 5. Summary of underwater image restoration methods.

Reference	Method Based	Advantages	Disadvantages
Huang et al. (2016)	Polarization	Effective in the cases of both scattered light and object radiance	High computational complexity
Hu et al. (2017)	Polarization	Enhanced visibility and low computational complexity	Didn’t effectively remove noise and no application for color images
Han et al. (2017)	Polarization	Suppressed backscattering and extracted edges	No experiments were applied in real-life conditions
Hu et al. (2018)	Polarization	Enhanced the underwater images even in turbid media	Complex computational time
Hu et al. (2018)	Polarization	Intensity and DCP of backscattering were suppressed	Solving the $A_{\infty }$ and $P_{scat}$ spatial distribution was very difficult
Ferreira et al. (2019)	Polarization	Effective method for underwater images recovery	Complicated the cost function and time-consuming
Yang et al. (2019)	Polarization	Enhanced the contrast in underwater images	Noise wasn’t removed
Wang et al. (2022)	Polarization	Qualitatively and quantitatively improved the underwater images and removed noise	High time complexity
Jin et al. (2020)	Polarization	higher signal to noise ratio and higher contrast	Noise wasn’t removed
Fu et al. (2020)	Polarization	Enhanced visibility in underwater images	High computational complexity
Burno et al. (2010)	Stereo Imaging	Good quality underwater images	High time complexity
Roser et al. (2014)	Stereo Imaging	improved stereo estimation	Didn’t work well in shallow water due to various light conditions
Lin et al. (2019)	Stereo Imaging	Enhanced the stereo imaging system	High computational time
Luczynski et al. (2019)	Stereo Imaging	Effective method	Noise wasn’t removed
Tan et al. (2006)	Rang Gated	Enhanced underwater images contrast	Noise wasn’t removed
Li et al. (2009)	Rang Gated	Reduced speckle noise and preserved features details	High computational complexity
Liu et al. (2018)	Rang Gated	Enhanced image visibility	Didn’t effectively remove noise
Wang et al. (2020)	Rang Gated	Enhanced image contrast and visibility	High computational complexity
Wang et al. (2021)	Rang Gated	Enhanced image contrast and worked well even if the estimated depth was smaller	Complication of cost function
Trucco and Olmos-Antillon (2006)	Optical	Optimized the computed parameters values automatically	Increased the time and computation complexity
Hou et al. (2007)	Optical	Effective method that depended on point spread function	Importance of estimating the parameters of illumination scattering
Boffety et al. (2012)	Optical	An effective smoothing method was used	Low contrast in images
Wen et al. (2013)	Optical	Enhanced the perception of underwater images	Poor flexibility and adaptability
Ahn et al. (2018)	Optical	Effective and accurate method	Increased time complexity
Chao and Wang (2010)	DCP	Recovered the underwater images and removed scattering	Underwater images suffered from color distortion
Yang et al. (2011)	DCP	Fast method for underwater images restoration	Only suitable for underwater images with rich colors.
Chiang and Chen (2011)	DCP	Restored underwater images color balance and removed haze	High computational complexity
Serikawa and Lu (2014)	DCP	Improved the contrast and visibility	High computational time
Peng et al. (2015)	DCP	Exploited the blurriness of underwater image	Noise wasn’t removed
Lu et al. (2015)	UDCP	Color correction of underwater images effectively	Decreased the contrast
Lu et al. (2017)	UDCP	Effective method for recovering the underwater images	Increased noise
Galdran et al. (2015)	UDCP	Enhanced the artificial light and contrast	Colors of some restored images were unreal and incorrect
Carlevaris-Bianco et al. (2010)	MIP	Reduced the haze effects and provided color correction	Didn’t solve problems of attenuation and scattering
Zhao et al. (2015)	MIP	Removed haze effect and corrected colours	Illumination wasn’t considered
Li et al. (2016)	MIP	Increased brightness and contrast of underwater images	Noise wasn’t removed
Peng and Cosman (2017)	Other Prior	Worked well for various underwater images	Noise wasn’t removed
Peng et al. (2018)	Other Prior	Restored degraded images and increased contrast	High computational complexity
Li et al. (2016)	Other Prior	Increased brightness and contrast	Couldn’t remove noise effects
Wang et al. (2017)	Other Prior	Enhanced contrast and corrected colours	High time complexity
Song et al. (2018)	Other Prior	improved quality of underwater images and Lowest running time	Noise wasn’t removed
Ding et al. (2017)	DL	Increased contrast	Highest running time
Cao et al. (2018)	DL	Restored images effectively	Blurring and low visibility of underwater images
Barbosa et al. (2018)	DL	Increased the underwater images quality	Noise wasn’t removed
Hou et al. (2018)	DL	Increased contrast, and restored natural appearance	Noise and some blurring

Table 5. Summary of underwater image restoration methods.

Reference	Method Based	Advantages	Disadvantages
Huang et al. (2016)	Polarization	Effective in the cases of both scattered light and object radiance	High computational complexity
Hu et al. (2017)	Polarization	Enhanced visibility and low computational complexity	Didn’t effectively remove noise and no application for color images
Han et al. (2017)	Polarization	Suppressed backscattering and extracted edges	No experiments were applied in real-life conditions
Hu et al. (2018)	Polarization	Enhanced the underwater images even in turbid media	Complex computational time
Hu et al. (2018)	Polarization	Intensity and DCP of backscattering were suppressed	Solving the $A_{\infty }$ and $P_{scat}$ spatial distribution was very difficult
Ferreira et al. (2019)	Polarization	Effective method for underwater images recovery	Complicated the cost function and time-consuming
Yang et al. (2019)	Polarization	Enhanced the contrast in underwater images	Noise wasn’t removed
Wang et al. (2022)	Polarization	Qualitatively and quantitatively improved the underwater images and removed noise	High time complexity
Jin et al. (2020)	Polarization	higher signal to noise ratio and higher contrast	Noise wasn’t removed
Fu et al. (2020)	Polarization	Enhanced visibility in underwater images	High computational complexity
Burno et al. (2010)	Stereo Imaging	Good quality underwater images	High time complexity
Roser et al. (2014)	Stereo Imaging	improved stereo estimation	Didn’t work well in shallow water due to various light conditions
Lin et al. (2019)	Stereo Imaging	Enhanced the stereo imaging system	High computational time
Luczynski et al. (2019)	Stereo Imaging	Effective method	Noise wasn’t removed
Tan et al. (2006)	Rang Gated	Enhanced underwater images contrast	Noise wasn’t removed
Li et al. (2009)	Rang Gated	Reduced speckle noise and preserved features details	High computational complexity
Liu et al. (2018)	Rang Gated	Enhanced image visibility	Didn’t effectively remove noise
Wang et al. (2020)	Rang Gated	Enhanced image contrast and visibility	High computational complexity
Wang et al. (2021)	Rang Gated	Enhanced image contrast and worked well even if the estimated depth was smaller	Complication of cost function
Trucco and Olmos-Antillon (2006)	Optical	Optimized the computed parameters values automatically	Increased the time and computation complexity
Hou et al. (2007)	Optical	Effective method that depended on point spread function	Importance of estimating the parameters of illumination scattering
Boffety et al. (2012)	Optical	An effective smoothing method was used	Low contrast in images
Wen et al. (2013)	Optical	Enhanced the perception of underwater images	Poor flexibility and adaptability
Ahn et al. (2018)	Optical	Effective and accurate method	Increased time complexity
Chao and Wang (2010)	DCP	Recovered the underwater images and removed scattering	Underwater images suffered from color distortion
Yang et al. (2011)	DCP	Fast method for underwater images restoration	Only suitable for underwater images with rich colors.
Chiang and Chen (2011)	DCP	Restored underwater images color balance and removed haze	High computational complexity
Serikawa and Lu (2014)	DCP	Improved the contrast and visibility	High computational time
Peng et al. (2015)	DCP	Exploited the blurriness of underwater image	Noise wasn’t removed
Lu et al. (2015)	UDCP	Color correction of underwater images effectively	Decreased the contrast
Lu et al. (2017)	UDCP	Effective method for recovering the underwater images	Increased noise
Galdran et al. (2015)	UDCP	Enhanced the artificial light and contrast	Colors of some restored images were unreal and incorrect
Carlevaris-Bianco et al. (2010)	MIP	Reduced the haze effects and provided color correction	Didn’t solve problems of attenuation and scattering
Zhao et al. (2015)	MIP	Removed haze effect and corrected colours	Illumination wasn’t considered
Li et al. (2016)	MIP	Increased brightness and contrast of underwater images	Noise wasn’t removed
Peng and Cosman (2017)	Other Prior	Worked well for various underwater images	Noise wasn’t removed
Peng et al. (2018)	Other Prior	Restored degraded images and increased contrast	High computational complexity
Li et al. (2016)	Other Prior	Increased brightness and contrast	Couldn’t remove noise effects
Wang et al. (2017)	Other Prior	Enhanced contrast and corrected colours	High time complexity
Song et al. (2018)	Other Prior	improved quality of underwater images and Lowest running time	Noise wasn’t removed
Ding et al. (2017)	DL	Increased contrast	Highest running time
Cao et al. (2018)	DL	Restored images effectively	Blurring and low visibility of underwater images
Barbosa et al. (2018)	DL	Increased the underwater images quality	Noise wasn’t removed
Hou et al. (2018)	DL	Increased contrast, and restored natural appearance	Noise and some blurring

4.1.1. Hardware based Restoration

Monitoring and exploring the underwater environment requires many hardware devices. These devices are also used to improve underwater images. The methods that hardware-based need hardware components for underwater image restoration. This includes using lasers, sensors, polarizers, ROVs, polaricams, and stereo imaging. Polarization processing has been used to reduce backscattering precisely. The polarization process is executed by applying a polarized light source for taking pictures or using polarization cameras. The laser-based methods have been used to eliminate backscattering by using a camera that closes the flash gate at a particular moment. Waterproof sensors have been applied for sensing marine snow, macroparticles, and swimming organisms to prevent reflections. Aqua tripods are used for capturing underwater images more effectively, and these devices are placed on the seafloor. Underwater image restoration, depending on hardware, can be classified into three categories, namely, polarization characteristics, stereo imaging, and range gate imaging.

4.1.1.1. Polarization characteristic-based

Light polarization is the property of light waves that describes their direction of oscillation. Polarization vibrates light in only one direction [29,36,37,38]. In air, reflected light is partially polarized, while in water, the light is visible in most directions. Therefore, it is much weaker, and this scattering along multiple paths degrades the polarization through meters. Because of the advantages of avoiding the scattering and absorption of light, polarization imaging has become a more significant underwater image restoration technique. [39] presented a technique that depended on the effect of polarization on objects. This method recovered the objects’ radiance based on the target signal’s estimated polarization and enhanced the underwater image quality in cases where backscatter and object radiance was found. It has been used in many applications, such as artifact objects.

Hu et al. [40] solved underwater vision problems such as signal attenuation and backscatter veiling. They developed an underwater image recovery method that depended on transmittance correction. It transformed the transmittance of low depolarization objects from negative to positive values, optimizing underwater images’ quality with the simple polynomial fitting algorithm. This method was very effective for underwater images with a high or low degree of depolarization. Han et al. [41] enhanced low-resolution and low-contrast underwater images resulting from light attenuation and scattering in water. They depended on the PSFs that were estimated using a slant-edge method. Subsequently, the modulation transfer function (MTF) was proposed for evaluating resolution variation with spatial frequencies. This method reduced the effect of underwater image scattering.

Hu et al. [42] proposed a method for polarimetric image restoration in turbid media using the circular polarization arising from illumination. The restored underwater images contain linear and circular polarization information. This method produced more effective experimental results than the previous methods. The results of this method confirmed that it enhanced the quality of recovered underwater images recorded in turbid water.

Hu et al. [43] developed a restoration method that estimated the polarization degree and the backscatter intensity at different positions in the underwater images. This method considered the field of non-uniform optics in underwater image retrieval. Recovering the radiance of objects uses an estimation of backscatter intensity at different image positions and degrees of polarization (DOP) and was highly effective in enhancing underwater images. Sanchez et al. [44] developed a method for restoring underwater images through the estimation of model parameters using the bioinspired optimization metaheuristic with a cost function: a no-referenced image quality metric (NR-IQA). This method could restore the underwater images, but with a complicated cost function.

Yang et al.[45] developed an underwater image restoration method that relied on polarimetric images using active non-polarized illumination. The non-polarized illumination indicated that the polarization effect could be discounted, and it did not matter whether the degree of polarization was low or high. This method improved the visibility and image contrast. Wang et al. [46] presented a new technology for restoring underwater images that depended on the periodic integration of polarization images. It replaced one or two pairs of orthogonal polarization images by integrating a series of polarization images into the polarization differential imaging (PDI) system. This method captured images at different positions during a complete cycle of image intensity. Then, these images were combined, and the result was calculated based on integrating the polarized light’s intensity. Finally, the polarization degree at each pixel was computed, and a clear image was restored.

Jin et al. [47] developed a new method for removing polarization scattering based on automatically executing polarimetric calculations of the target light at each pixel, which helped restore the underwater image. The polarization degree of the target light in this method was constant. This method was very effective in retrieving underwater images and enhanced the visibility and contrast in underwater images. Fu et al.[48] proposed a new underwater image restoration method consisting of scattering and absorption compensation. It depended on the wavelength and depth of the scene in the underwater signals. In the scattering method, an automatic map was used to estimate the backlight without considering the existence or not of any object. In absorption, a new compensation strategy was introduced in color restoration.

4.1.1.2. Stereo imaging

The stereo imaging method simulates the human visual system. This method uses traditional cameras to take pictures of the same target from various views and perspectives and then computes the depth of the field from these stereo images. Due to the emergence of the charged-coupled device (CCD), this method consists of a binocular vision device that obtains the depth information. Higher resolutions and refresh rates along with lower costs make this method of stereo imaging more popular in AUV systems.

Bruno et al. [49] proposed a structured illumination and light in the stereo imaging method with various conditions of water turbidity. This method applied 3D underwater reconstructions that depended on the combination of stereo-photogrammetry and structured light. The patterns of structured light were projected using a video projector and acquired by the stereo-vision system. This method achieved effective results even if in turbid conditions.

Roser et al. [50] developed a method for improving stereo perception in AUV systems. This method was applied for enhancing and restoring underwater images to improve the stereo range resolution using natural, dynamic lighting under turbid conditions. This method used a model for underwater light attenuation to estimate the visibility parameters. First, contrast enhancement was performed by employing visibility estimation and computing disparate densities. Second, the light attenuation model was used for ocean water to obtain color enhanced images.

Lin et al. [51] proposed an image restoration method for AUVs that depended on an object-recognition and stereo-imaging system. The Hough transform used with the optical flow method for linear features and movement speeds in dynamic underwater imaging and used the Harris corner detector for target distance estimation. The AUV had a binocular camera with wide-angle lenses. This method was highly effective and produced accurate results.

Luczynski et al. [52] proposed a method for improving stereo imaging hardware for deep sea operations. The method had the computation power for processing onboard stereo vision and also for tasks of computer vision such as inspection, object recognition, mapping, navigation, and intervention. They formalized a method for stereo component selection that included optimizing and validating the pressure in cameras using the finite element method (FEM).

4.1.1.3. Range gated imaging

The system of range-gated imaging includes a fast camera that uses a CCD image sensor, a timing control unit (TCU), and a pulsed laser. It controls the gate of camera that intakes the reflected light directly and prevents backscattering from reaching the sensor. The camera gate’s switching intervals depend on prior information, manual settings, various sensors, and a laser range finder. This gate is opened for a short time until the pulses return after hitting an object and then immediately close.

For an ROV, tan et al. [53] presented a hardware optimization method for range-gated imaging in highly turbid conditions. They advanced hardware for a range-gated imaging system and the optimization stages of tailgating and preprocessing techniques. The tail gating system was applied by a camera delay to the tail of the reflected image temporal profile (RITP) and this was followed by contrast limited adaptive histogram equalization (CLAHE) for image enhancement.

Li et al. [54] used a range-gated system for restoring underwater image visibility and quality in turbid conditions. It utilized time discrimination for enhancing the ratio of signal-to-backscattering noise by rejecting the backscattered light in the medium. It consisted of a synchronous and control system, a pulsed laser system, and a camera with a high-speed gate. This method efficiently reduced speckle noise in the underwater images and preserved the details of features.

Liu et al. [55] proposed a system for constructing the scattering model and developed an optimal pulse through coordinated gate control. This method used a 532 nm narrow-pulse laser with a self-built gain CCD system to form the range-gated imaging system. This method was verified by simulation and computing the relative ratio for the images that were acquired through the laser distance gating system.

Wang et al. [56] developed a 3D dehazing range-gated system for removing the scattering. This method greatly advanced underwater target navigation, detection, and marine scientific research because of the excellent suppression of backscatter. This method depends on the characteristics of how light propagates in water. The reference image and coefficient of water attenuation were needed for computing the depth-noise maps (DNMs). The experiments on this method were conducted under various water conditions.

Wang et al. [57] proposed a method to decrease the input images number and restore their clarity. This method was used for dehazing underwater images using only a single-gated underwater image. It depended on the prior that target intensity distributes due to the range intensity profile (RIP) in RGI. The depth noise map and depth transmission were computed from the scene depth. Finally, the high quality of the images was restored and enhanced.

4.1.2. Software based Restoration

Software-based restoration is a non-physical restoration approach that aims to create the imaging model and compute the parameters used in this imaging model. These methods use restoration software algorithms to recover underwater images. Underwater image restoration depended on software can be classified into three groups, namely, optical imaging, prior knowledge, and deep learning methods. Compared to hardware-based methods, those based on software- have many advantages such as lower computational time, easy modulation, better design, and reduced costs.

4.1.2.1. Optical

The Underwater Optical Imaging (UOI) model can obtain natural and clear underwater images by establishing a rough optical imaging model and reversing the degradation process [58]. This model is defined by Equation 2. There are many underwater optical imaging applications, such as detectors for onboard underwater optics, aerial, ocean-surface, and underwater optical cameras [59].

Trucco et al. [60] developed a method for self-tuning image restoration that depends on the Jaffe-McGlamery UOI model [58,61]. The optimal filter parameters are automatically computed for each underwater image by optimizing the quality depended on the global contrast measure. The simplified physical model is suitable for diffused light with poor backscatter and various imaging conditions. This technique depends on the basic assumptions that the underwater images were affected both by forward scatter and homogeneous illumination.

Hou et al. [62] presented a framework for underwater image restoration that depended on the UOI model. They assumed that the blurring in underwater images resulted from the scattering by suspended particles and organisms. The restoration was done by considering underwater image proprieties from different domains (i.e., spatial and frequency). From the spatial domain, they used the point spread function (PSF) and modulation transfer function (MTF) from the frequency domain. This method restored underwater images using deconvolution depended on estimating the light scattering parameters.

Boffety et al. [63] developed a valuable simulation tool for color restoration that depended on underwater optical images. This method is based on studying the spectral discretization influence of the model parameters on color rendering. They demonstrated that if just RGB data from the simulation scene is available, the reconstruction step improves the image color.

Wen et al. [64] presented an underwater optical technique for describing underwater image formation that depends on the physical process. Then, after using this model, an enhancement algorithm was applied to enhance the images. The new underwater dark channel prior was proposed to compute the scattering rate and the backlight in the UOI. The results showed that this method was efficient at restoring underwater images. As part of the sampling missions, ahn et al. [65] presented an image transmission system as a sampling system-AUV (SAUV) and demonstrated its effectiveness on the high seas. This method applied underwater optical imaging for autonomous vehicles and increased underwater detection accuracy.

4.1.2.2. Prior knowledge-based Image Restoration

Light absorption, suspended particles, and scattering are the reasons for underwater image degradation. Many restoration methods depend on prior knowledge applied for underwater image restoration, such as the dark channel prior (DCP) [66,67], underwater dark channel prior (UDCP) [68,69], maximum intensity prior (MIP) [70], red channel prior (RCP) [71], and underwater light attenuation prior (ULAP) [72]. The following subsections discuss the various types of these prior-based methods applied for underwater image restoration.

• Dark Channel Prior (DCP) Method
  [66] presented the DCP method that is used for dehazing the images. Haze is a normal phenomenon that reduces visibility, obscures scenes, and changes colors. It is a problem for photographers as it causes the degradation of image quality. It threatens the reliability of many applications, such as object detection, outdoor surveillance, and aerial imagery. Therefore, removing the haze from images is crucial in computer graphics/vision. The DCP-based dehazing technique is used for enhancing underwater images. This method depends on the observation that good quality and clear underwater images have some pixels at very low intensities in at least one color channel.
  For restoring clear underwater images, chao et al. [31] proposed an effective DCP-based method, which was used to reduce the effects of water scattering and attenuation in underwater images. DCP was used to compute the turbid water depth by assuming that multiple patches in water-free underwater images consist of a few pixels with very low intensities in at least one color channel. yang et al. [73] developed a low-complexity and efficient DCP-based method for restoration of underwater image. They calculated the depth maps of images by employing a media filter instead of soft matting. Color correction was also used to improve the contrast in the underwater image. This method was highly effective images restoration and reduced the execution time.
  Chiang et al. [33] presented a method for enhancing underwater images by applying Wavelength Compensation and Image Defogging (WCID). They used the dehazing algorithm to reduce for the attenuation discrepancy across the propagation path and to remove the possible light source influence presence. This method performed well in enhancing the underwater images objectively and subjectively. Serikawa et al. [74] proposed a new method that compensates for the attenuation discrepancy across the propagation path and used a fast dehazing algorithm named joint trigonometric filtering (JTF). JTF improves the transmission map (TM), which, estimated by the DCP affords many improvements, such as scatter reduction, edge information, and image contrast. This algorithm is characterized by noise reduction, better exposure to dark regions, and improved contrast.
  Peng et al. [75] developed a method for computing depth maps for underwater image restoration. It depended on the observation that an object that was further from the camera was more blurred. They combined image blurriness with the image formation model (IFM) to compute the distance between the scene points and the camera. It was much more effective than any other IFM-based enhancement method. The DCP is affected by selective light attenuation in the underwater environment, so various underwater enhancement methods based on DCP were developed and used.
• Underwater Dark Channel Prior (UDCP) Method
  The underwater image red channel will dominate the dark channel because red light attenuates more rapidly than blue and green light as it travels through the water. To avoid the red influence, [68] introduced the UDCP, which evaluates only the green and blue (GB) channels to determine the underwater DCP. [76] proposed a new technique that compensates for the attenuation discrepancy in underwater images through the propagation path. They developed color lines depended on an ambient light estimator and adaptive filtering in shallow oceans for underwater image enhancement. They also presented a color correction algorithm for color restoration.
  Lu et al. [12] proposed a new technique for super-resolution (SR) and scattering in underwater images. First, based on self-similarity, a high resolution (HR) of the scattered and the de-scattered image is obtained through the SR algorithm. Then, the convex fusion rule is used for retrieving the HR image. This algorithm is highly effective in restoring underwater images. Galdran et al. [71] developed a new, automatic method for the restoring of underwater images that depends on RCP. This RCP extracts the dark channel in which the blue and red channels are reserved. Their experimental results indicate that this method effectively enhances degraded underwater images.
• Maximum Intensity Prior (MIP) Method
  Suspended particles that cause turbidity or fogging degrade the underwater images quality. The difference in attenuation between the underwater images’ red (R) and GB channels is significant. Carlevaris et al. [70] developed an effective algorithm that removes light scattering, known as dehazing, in underwater images. They presented a prior for estimating scene depth termed the maximum intensity prior (MIP). The MIP is the difference value between the R channel intensity and the maxima of the G and B channels. The closest point shift in the foreground represents the most significant difference between the color channels.
  Zhao et al. [77] developed a new method that derives the water’s optical properties. This method estimated the background light (BL) that depended on the DCP and MIP. First, it took the brightest 0.1% of the dark channel pixels and then chose the pixel that differed maximally in the B-G or G-R channels. Li et al. [78] developed a new method for restoring underwater images that determines the selected background light using its maximally different pixels. This method depends on dehazing the blue-green channels and correcting the red channel. First, by using a blending strategy as Li et al. [79,80], a flat background region was selected in the quad-tree subdivision. Then, 0.1% of the region candidate’s brightest pixels from the dark channel were taken. Finally, a pixel with the greatest difference in the R-B channel was selected as the global backlight.
• Other Prior-based Method
  In addition to those listed above, some priors are not commonly applied but are helpful in underwater image restoration. For example, Peng et al. [81] developed a new technique for computing the underwater scenes depth that depended on light absorption and image blurring. This method was used in the IFM for image restoration and its experimental results were much more accurate and effective than any other.
  Peng et al. [82] developed a method for enhancing and restoring underwater images by reducing light absorption, scattering, low contrast, and color distortion caused by light traveling through a turbid medium. First, ambient light was computed by color change that depended on depth. Then, the scene transmission was computed by the differences between the observed intensity and the ambient light. In addition, adaptive color correction was calculated. Li et al. [79] developed a method for enhancing and restoring underwater images that depends on the minimum information loss principle (MILP). The dehazing algorithm was applied to recover underwater images’ color, natural appearance, and visibility. An effective contrast enhancement algorithm was applied to enhance underwater images’ contrast and brightness. It improved visual quality, accuracy, and other valuable information.
  Wang et al. [83] proposed the maximum attenuation identification (MAI) technique for deriving the depth map and backlight from degraded underwater images. Region background estimation was simultaneously applied to ensure optimal performance. Experiments were conducted on three image types: calibration plate, natural underwater scene, and colormap board. Song et al. [72] presented an accurate, effective, and rapid scene depth estimation model that depended on ULAP. It assumed that the differences between the R intensity value and the G and B intensity values in only one pixel of the underwater image were strongly related to depth changes in the scene. In estimating the R-G-B channels, this model was applied for the BL and TMs.

4.1.2.3. Deep Learning

Restoring degraded and hazy underwater images is a challenge. Existing prior-based methods have inferior and limited performance in many situations because of their hand-designed features. Therefore, the tendency toward deep learning algorithms is critical. Due to the deep learning rapid development in underwater image restoration, researchers have seen a major shift from complete parameter selection using artificial optimization models to automatic and effective training models. They depend on instance data to extract valuable feature vectors using deep learning.

Ding et al. [84] developed a technology for solving the problem of underwater images that were degraded due to light scattering and color casts. This method featured underwater enhancement that included color correction and an image dehazing method that depended on the atmospheric scattering model. First, the transmission map was derived from the color-corrected image. Then, a convolutional neural network (CNN) was used to the image patches extracted from the color-corrected image to predict the depth map of the scene. This method was exceptionally effective and accurate and was used in many applications, such as underwater object detection and recognition. Cao et al. [85] developed a method for restoring underwater images that depended on two neural network techniques for estimating scene depth and backlight. This method solved problems such as color distortion and low contrast resulting from light scattering and absorption. The method’s effectiveness was confirmed by its experimental results.

Barbosa et al. [86] developed a CNN-based technique for underwater image enhancement and restoration. This method did not require any ground truth data as it used image quality metrics to support underwater image restoration. The results of these experiments showed a notable improvement in the underwater images’ visual quality and preserved edges. Hou et al. [87] developed a new framework for performing residual learning in the transmission and image domains. This method consisted of a data-driven residual model for transmission estimation and residual formulation based on the knowledge-driven illumination balance in the underwater environment. Qualitative and quantitative analyses both confirmed the method’s effectiveness.

4.2. Underwater Image Enhancement Techniques (IFM-free)

Studies related to enhancing underwater images often use the techniques of enhancement directly to the images [88,89]. These methods enhance the color and contrast of images depended on pixel intensity redistribution and do not depend on the principles of underwater imaging. Further enhancement methods are applied especially associated with the underwater image characteristics, such as low contrast, and haze. These methods make changes for the pixel values in the spatial or transformation domain. Deep learning methods, especially CNNs, have been applied for underwater image enhancement that relies on hidden features that can be learned for quality improvement. Underwater image enhancement is categorized into four groups: spatial domain-based image enhancement, frequency domain-based image enhancement, color constancy-based image enhancement, and deep learning-based image enhancement. Table 6 presents a comparison of underwater image enhancement methods.

Table 6. The summary of the underwater image enhancement methods.

Reference	Method Based	Advantages	Disadvantages
Ancuti et al. (2012)	Spatial Domain (SCM)	Increased contrast of underwater images	Didn’t work well with poor artificial light
Ancuti et al. (2016)	Spatial Domain (SCM)	High accuracy in underwater images enhancement	Noise wasn’t removed
Liu et al. (2017)	Spatial Domain (SCM)	Enhanced underwater images contrast and visibility	Low accuracy
Torres-M´ endez and Dudek (2008)	Spatial Domain (MCM)	Depended on learned constraints for underwater images enhancement	Some noise and blurring in underwater images
Iqbal et al. (2007)	Spatial Domain (MCM)	Solved the problem of light	Low contrast in underwater images
Ghani and Isa (2017)	Spatial Domain (MCM)	Enhanced underwater images qualitatively and quantitatively	High time complexity
Hitam et al. (2013)	Spatial Domain (MCM)	Highest PSNR values and lowest MSE	Blurring in underwater images
Huang et al. (2018)	Spatial Domain (MCM)	Enhanced the visibility of underwater images	Not suitable for all types of underwater images
Petit et al. (2009)	Frequency Domain	Light attenuation was removed	Low contrast and visibility
Cheng et al. (2015)	Frequency Domain	Better Contrast and Higher visibility	Highest time running
Sun et al. (2011)	Frequency Domain	Removed the noise from underwater images	Poor quality in low light conditions
Ghani et al. (2018)	Frequency Domain	Highest contrast and visibility	Highest run time
Priyadharsini et al. (2018)	Frequency Domain	Better PSNR and SSIM results	Some Noise wasn’t removed
Joshi et al. (2008)	Color Constancy	Balance between machine and human vision	Low color and contrast distortion
Fu et al. (2014)	Color Constancy	Enhanced contrast, color, and edges and details	High time complexity
Zhang et al. (2017)	Color Constancy	Enhanced edges and reduced noise	Couldn’t enhance the underwater images contrast
Wang et al. (2018)	Color Constancy	Increased image quality and balanced color	Noise and high time complexity
Zhang et al. (2019)	Color Constancy	Good denoising and edge-preserving	Low contrast
Tang et al. (2013)	Color Constancy	Intensity channel was applied in multi-scale Retinex	Filtering techniques were in efficient
zhang et al. (2021)	Color Constancy	Increased contrast	Noise wasn’t removed
Dixit et al. (2016)	Contrast	Removed noise and preserved details	Low efficiency and highest time
Wang et al. (2016)	Contrast	Increased contrast and precision value	Didn’t remove noise
Bindhu and Maheswari (2017)	Contrast	Noise was reduced	High computational complexity
Guraksin et al. (2019)	Contrast	Visual information is more important	Didn’t remove haze
Sankpal and Deshpande (2019)	Contrast	Increased images’ contrast	Entropy was still less than other researches
Azmi et al. (2019)	Contrast	Improved images details and reduced color cast	Low efficiency and highest time
Wang et al. (2017)	Deep Learning	Enhanced contrast and color correction	Low efficiency and highest time
Fabbri et al. (2018)	Deep Learning	Enhanced contrast	Noise and Light not solved
Anwar et al. (2018)	Deep Learning	Enhanced contrast	Didn’t remove haze values.
Li et al. (2018)	Deep Learning	Corrected color cast	Low contrast
Li et al. (2019)	Deep Learning	Enhanced contrast	Effects of attenuation and backscatter weren’t solved
Pritish et al. (2019)	Deep Learning	Enhanced contrast and visibility of underwater images	Noise wasn’t removed
Li et al. (2020)	Deep Learning	Enhanced brightness and visibility	Low contrast and noise wasn’t removed
Hu et al. (2021)	Deep Learning	Enhanced contrast of underwater images	Clarity of the image was far lower than that of the truth image
Tanget al. (2023)	Deep Learning	Enhanced contrast	The network was more weaker

Table 6. The summary of the underwater image enhancement methods.

Reference	Method Based	Advantages	Disadvantages
Ancuti et al. (2012)	Spatial Domain (SCM)	Increased contrast of underwater images	Didn’t work well with poor artificial light
Ancuti et al. (2016)	Spatial Domain (SCM)	High accuracy in underwater images enhancement	Noise wasn’t removed
Liu et al. (2017)	Spatial Domain (SCM)	Enhanced underwater images contrast and visibility	Low accuracy
Torres-M´ endez and Dudek (2008)	Spatial Domain (MCM)	Depended on learned constraints for underwater images enhancement	Some noise and blurring in underwater images
Iqbal et al. (2007)	Spatial Domain (MCM)	Solved the problem of light	Low contrast in underwater images
Ghani and Isa (2017)	Spatial Domain (MCM)	Enhanced underwater images qualitatively and quantitatively	High time complexity
Hitam et al. (2013)	Spatial Domain (MCM)	Highest PSNR values and lowest MSE	Blurring in underwater images
Huang et al. (2018)	Spatial Domain (MCM)	Enhanced the visibility of underwater images	Not suitable for all types of underwater images
Petit et al. (2009)	Frequency Domain	Light attenuation was removed	Low contrast and visibility
Cheng et al. (2015)	Frequency Domain	Better Contrast and Higher visibility	Highest time running
Sun et al. (2011)	Frequency Domain	Removed the noise from underwater images	Poor quality in low light conditions
Ghani et al. (2018)	Frequency Domain	Highest contrast and visibility	Highest run time
Priyadharsini et al. (2018)	Frequency Domain	Better PSNR and SSIM results	Some Noise wasn’t removed
Joshi et al. (2008)	Color Constancy	Balance between machine and human vision	Low color and contrast distortion
Fu et al. (2014)	Color Constancy	Enhanced contrast, color, and edges and details	High time complexity
Zhang et al. (2017)	Color Constancy	Enhanced edges and reduced noise	Couldn’t enhance the underwater images contrast
Wang et al. (2018)	Color Constancy	Increased image quality and balanced color	Noise and high time complexity
Zhang et al. (2019)	Color Constancy	Good denoising and edge-preserving	Low contrast
Tang et al. (2013)	Color Constancy	Intensity channel was applied in multi-scale Retinex	Filtering techniques were in efficient
zhang et al. (2021)	Color Constancy	Increased contrast	Noise wasn’t removed
Dixit et al. (2016)	Contrast	Removed noise and preserved details	Low efficiency and highest time
Wang et al. (2016)	Contrast	Increased contrast and precision value	Didn’t remove noise
Bindhu and Maheswari (2017)	Contrast	Noise was reduced	High computational complexity
Guraksin et al. (2019)	Contrast	Visual information is more important	Didn’t remove haze
Sankpal and Deshpande (2019)	Contrast	Increased images’ contrast	Entropy was still less than other researches
Azmi et al. (2019)	Contrast	Improved images details and reduced color cast	Low efficiency and highest time
Wang et al. (2017)	Deep Learning	Enhanced contrast and color correction	Low efficiency and highest time
Fabbri et al. (2018)	Deep Learning	Enhanced contrast	Noise and Light not solved
Anwar et al. (2018)	Deep Learning	Enhanced contrast	Didn’t remove haze values.
Li et al. (2018)	Deep Learning	Corrected color cast	Low contrast
Li et al. (2019)	Deep Learning	Enhanced contrast	Effects of attenuation and backscatter weren’t solved
Pritish et al. (2019)	Deep Learning	Enhanced contrast and visibility of underwater images	Noise wasn’t removed
Li et al. (2020)	Deep Learning	Enhanced brightness and visibility	Low contrast and noise wasn’t removed
Hu et al. (2021)	Deep Learning	Enhanced contrast of underwater images	Clarity of the image was far lower than that of the truth image
Tanget al. (2023)	Deep Learning	Enhanced contrast	The network was more weaker

4.2.1. Spatial Domain-based Image Enhancement

The spatial domain process depends on the intensity histogram, which expands the gray levels depended on the grayscale mapping theory [90]. Due to the nature of underwater images, histograms indicate a more concentrated and important pixel-value distribution than is found in natural images. The dynamic range expansion of the underwater image histogram improves the visibility, detailed information, and contrast of images. The spatial domain completes the intensity histogram in various standard color models, e.g., red-green-blue (RGB), hue-saturation-intensity (HSI), hue-saturation-value (HSV), and CIE-Lab. The spatial domain approach has significantly advanced in the area of image enhancement [91,92]. The spatial domain is divided into two subgroups: The Single-Color Model (SCM) and the Multi-Color model (MCM), as introduced in the following paragraphs.

• Single Color Model (SCM) Method
  Ancuti et al. [93] presented a fusion-based method for underwater image enhancement. First, the two fused images were created from the input image. The first image was corrected by white balancing, and the contrast was improved for the second image using adaptive histogram equalization. Thereafter, the four fusion weights were defined relative to the salient features, contrast, and two fused image exposure. Finally, the two fused images and weights were combined to obtain the enhanced images. Ancuti et al. [34] proposed a method for color balance and the enhancement of underwater images. This method used a single image and did not need specified hardware, knowledge, or information about scene structure or underwater conditions. It relied on the fusion of two images derived from a white-balanced, color-compensated image of the original degraded and hazed image. This method improved the underwater images’ contrast, edge sharpness, and visibility.
  Liu et al. [94] developed an effective and accurate underwater image enhancement method. This method is known as Deep Sparse Non-negative Matrix Factorization (DSNMF) for estimating the underwater image illumination. First, the images were divided into small blocks. Each channel of this small block was an [R, G, B] matrix, then each depth of the matrix was divided into several layers using the DSNMF sparsity constraint. The last layer of this factoring matrix was applied as illumination, and the image was enhanced with sparse constraints.
• Multiple Color Model (MCM) Method
  Torres et al. [95] depended on the Markov random field (MRF) to represent the relations between underwater images before and after distortion and used maximum a posterior (MAP) estimation for enhancing the colors in underwater images. While computing the dissimilarity between image patches, the underwater images were transformed to the CIE-Lab color space to represent equally perceived differences. This method’s experimental results indicated its efficacy and feasibility. Iqbal et al. [96] developed a new and effective method for underwater image enhancement depended on the integrated color model (ICM). This method solved the problems of image degradation through light scattering and absorption. First, it applied the RGB contrast stretching algorithm for color contrast equalization. Second, HSI saturation and intensity stretch were applied to increase the true color and improve the brightness and saturation in the degraded underwater images.
  Ghani et al. [97] developed a technique for underwater image enhancement that depended on recursive adaptive histogram modification (RAHIM). This technique tended to enhance underwater images’ backgrounds to increase the contrast. They modified the brightness and the underwater image saturation in the HSV color model using the human visual system and the Rayleigh distribution. Then, the enhanced underwater image was converted to the RGB color model. Hitam et al. [98] developed a technique for enhancing underwater images that depended on contrast limited adaptive histogram equalization (CLAHE) and built the CLAHE mix to enhance the visibility of the underwater image. CLAHE was used for the RGB color model and the HSV color model for two underwater images. Thereafter, these images were merged using the Euclidean norm. This method enhanced the image contrast and reduced noise.
  Huang et al. [99] presented a method for underwater image enhancement that depended on relative global histogram stretching (RGHS) in two color models, RGB and CIE-Lab. First, the underwater images were preprocessed using gray world adaptive histogram stretching in the RGB color model with the help of RGB channels and selectively attenuated light propagation in the underwater images. Thereafter, the CIE-Lab color model, applied the brightness L and color a, b components for curve and linear adaptive stretching optimization.

4.2.2. Frequency Domain-based Image Enhancement

The frequency domain technique processes images using spatial, or convolution transform to enhance these images [100]. There are two components in the frequency domain: the high frequency, which represents the edge region where pixel values show significant changes; the low frequency represents the flat region in the image [101]. The frequency domain improves the underwater image quality through high-frequency amplification and by suppressing the low-frequency component [102]. The problem of Degraded underwater images is that the difference between the low- and high-frequency components is minimal [103]. Therefore, many techniques, such as homomorphic filtering [104], transformation domain methods [105], wavelet transform, and high-boost filtering are used to improve underwater images.

Petit et al. [106] presented an effective method that depended on quaternions to improve object contrast and color reproduction. This method requires the preprocessing of color space contraction and inversion light attenuation. A low-pass filter was used to remove noise through the high-frequency suppressor component, and a high-pass filter was used to preserve details by reducing the low-frequency components. The results of this method were very accurate and effective. Cheng et al. [107] developed a method for underwater image enhancement that designed the Jaffe-McGlamery optical model and proposed an accurate and effective algorithm for underwater image recovery. This algorithm used a prior dark red channel to compute the transmission and background light. They developed a simple low-pass filter to blurred and degraded underwater images by analyzing the physical property of the point scattering function. The experimental results confirmed that this method was highly effective.

Feifei et al. [108] presented a method for underwater image enhancement that depends on wavelet decomposition and a high-pass filter. This highly effective and accurate method was developed to reduce noise in underwater images and solved the wavelet shortcoming when processing backscatter noise. Ghani et al. [109] presented a technique to increase the visibility in deep underwater images that depended on homomorphic filtering, recursive superimposed CLAHE, and dual-image wavelet fusion. Homomorphic filtering was used to provide whole image illumination. The recursive overlapping CLAHE algorithm was used to stretch and separate overlapping blocks and adjacent overlapping blocks of the image channel. After that, these two images were fused using wavelet transform.

Priyadhars et al. [110] developed a method for underwater image enhancement to solve the imperfections in these images, such as low contrast and visibility. These problems caused objects in underwater images to be obscure. This method used the stationary wavelet transform (SWT) to divide the input image into four components; high-high, high-low, low-high, and low-low. The results showed that it was highly effective and increased contrast.

4.2.3. Color Constancy-based Image Enhancement

The human visual system is based on color constancy and ensures that colored objects are perceived predictably under various lighting conditions. Color constancy consists of white balancing and Retinex. White balancing is applied to ensure that the color of objects under various conditions of lighting is recorded accurately. Retinex is a precise and automatic application that relies on color constancy theory and enables humans to explore the world under various lighting conditions.

Joshi et al. [111] proposed a method to resolve imprecise coloration and low contrast in underwater images resulting from degradation. Retinex was used to achieve a balance between human and machine vision by applying color constancy. This method includes color rendering, dynamic range compression, and color constancy theory to produce highly effective and accurate results. Fu et al. [112] developed a technique for enhancing underwater images to address problems, as visual fuzz, insufficient illumination, and color distortion. This method was based on retinex, which was used to improve a single underwater image. First, color correction was used to resolve color distortion. After that, retinex was used to analyze illumination and reflectance. Finally, illumination and reflectance we enhanced to eliminate fuzz and underexposure problems.

Zhang et al. [113] developed a technique for underwater image enhancement to solve image problems, such as blurring, low contrast, and low visibility. This method depended on the Retinex framework that simulated the human visual system. Retinex is a portmanteau of "Retina" and "Cortex" and its function depends on a combination of trilateral and bilateral filters. This method effectively solved the degradation problem under various turbidity conditions. Yong et al. [114] developed a new and effective method for enhancing underwater images by converting them from the RGB color space to HSV. Then, Retinex was used to divide the v channel into a detail layer and a lighting layer that relied on various methods for image enhancement. Finally, the improved V, H, and S channels were converted to an RGB color model to improve and enhance images.

Zhang et al. [115] developed an underwater image enhancement technique to solve image degradation problems. They relied on a multiscale retinex with color restoration (MSRCR), which consisted of four main components: illumination estimation, guided operation filter, fog-free image reconstruction, and white balance operation. This highly effective method was used to improve image contrast and detail and produced excellent results. Tang et al. [116] developed a more advanced technique for underwater image enhancement that relied on Retinex and was suitable for multi-scene images. First, the images were pre-corrected to edit the pixel distribution and decrease the dominant color. Then, a multiscale Retinex with an intensity channel was applied. Finally, they applied infinite impulse response and down-sampled using Gaussian filtering to increase the processing speed.

Zhang et al. [117] developed a technique that resolved inferior image quality by enhancing the low contrast and color cast prevalent in underwater imaging. Their developments in color correction used the adaptive contrast enhancement technique for underwater images. First, the dedicated fractions were used to compensate for the lower color channels computed by considering the ratio of the difference between the lower and upper channel to the lower color channel. Then, the adaptive contrast enhancement technique was used to generate the underwater images with a stretched foreground and background. Finally, they applied an unwrap mask for sharpness.

4.2.4. Contrast-based Image Enhancement

Contrast contributes significantly to the subjective evaluation of underwater image quality. It refers to the brightness difference between dark and light areas in images. The luminance disparity reflected from two neighbouring surfaces creates contrast, and this deviation is the visual property that makes certain objects more distinguishable than others.

Dixit et al. [118] presented a method for image enhancement that was depended on the DCP with ACCLAHE and HF. The DCP computed the blur region and removed them. ACCLAHE estimated the maximum bin height in a local histogram of the images and redistributed the pixels equally to every gray level. The HF algorithm was used for enhancing underwater images.

Wang et al. [119] presented a method for enhancement of underwater image that contributes significantly to ocean research. This method depended on the model of a virtual retina and image quality assessment (IQA). The virtual retina is highly correlated with the human vision system and is applied for improving the contrast of underwater images and removing noise. After this, the adaptive enhancement of underwater images was measured with a type of no-reference image quality assessment. This method achieved higher performance than those produced by other research.

Bindhu et al. [120] proposed a method for solving underwater image problems such as low contrast, color loss, and haze. This method enhanced the underwater images’ quality using interpolation enhancement that was based on increasing the underwater images’ contrast. This method produced better entropy, a lower mean square error (MSE), and peak signal-to-noise ratio (PSNR) values.

Guraksin et al. [121] presented a method for underwater images that was depended on a wavelet transform algorithm and the differential evolution algorithm. First, contrast adjustment on underwater images. Then, homomorphic filtering was applied for the image’s brightness normalization. The images were divided into R, G, and B components. wavelet transform and Haar wavelet decomposition were applied to each channel. The method’s performance was tested by determining the PSNR, entropy, and MSE.

Sankpal et al. [122] proposed a method for solving light attenuation in water that caused degradation in underwater images. The method improved underwater images by correcting the backward scattering effect using Rayleigh stretching for every color channel’s maximum likelihood computation of the scale parameter. Correcting the signal corrected the underwater images.

Azmi et al. [123] proposed a method for underwater image color enhancement that consisted of four steps. First, a method was introduced to neutralize the color cast. The color channels were improved depending on the gain factors that were computed through the differences value between inferior and superior color channels. Second, the fusion of dual-intensity depended on the mean and median average. Third, the swarm intelligence depended on the equalization mean for enhancing images. Finally, the technique of unsharp masking was applied for enhancing images.

4.2.5. Deep Learning-based Image Enhancement

Deep learning methods produce superior feature extraction results more rapidly because of the deep network structure. These methods are widely used for defogging images [124], target detection [125], and image segmentation [126]. For instance, Wang, Zhang, Cao, and Wang (2017c) presented an effective and novel technique for underwater image enhancement that depended on a CNN. This technique, named UIE-Net, enhanced the contrast and brightness of underwater images degraded by dispersion and absorption. The UIE-Net framework’s tasks included haze removal and color correction.

[127] presented an underwater image enhancement technique to solve underwater image problems, caused by suspended particles, light absorption, and refraction. This highly accurate method improved the image quality using a generative adversarial network (GAN) to increase the reliability and safety of using visual perception. [128] developed a CNN-based method to improve underwater images. The UWCNN is a very effective and accurate model of an automatic mechanism for reconstructing clear and high-contrast underwater images. The UWCNN was efficiently trained using a synthetic underwater image database.

To solve imaging problems such as scattering and attenuation through water, Li et al. [129] proposed a correction method that depends on the supervised color transfer model. This model designed the multi-term loss function that included the measure of cycle consistency loss, similarity index loss, and adversary loss and its results were very effective and accurate. Li et al. [130] presented comprehensive research and analysis on the enhancement of underwater images that have been degraded because of light absorption and scattering. Using this technique, they compiled the underwater image enhancement benchmark (UIEB), a real dataset that contains 950 images that were trained using CNNs. The comprehensive study was analyzed quantitatively and qualitatively.

Uplavikar et al. [131] developed a technique for underwater image enhancement to resolve light scattering and attenuation that reduces image detail and contrast. This method solved a water-type diversity problem that hindered underwater image enhancement. This was done by learning and defining the content features of underwater images using untangling the annoyances of water types. Li et al. [132] developed an effective method to improve underwater images based on using a CNN that processed the underwater scene prior. The method combined the underwater image’s physical model and the underwater scene’s optical properties. This method was used to solve imaging problems such as light absorption and scattering that degraded the contrast and visibility in images. This method directly reconstructed clear images with high contrast.

Hu et al. [133] developed a method for enhancing underwater images degraded because of light scattering and absorption. A GAN that efficiently completes high-quality underwater image style conversions was applied to underwater image enhancement. Despite being widely used, GANs are affected by the quality of underwater images. This research added the natural image quality assessment (NIQE) index to the GAN algorithm to better compare underwater images. Tang et al. [134] proposed a more generative network based on attention U-Net that had attention gate mechanism. This gate filtered invalid features and saved texture, contour, and style information. This paper used three different loss functions to evaluate image quality for color, global content, and structural information.

4.3. Fusion of Restoration and Enhancement

Recently, many studies have tended to work on restoring and enhancing underwater images rather than working on just one of them. The fusion approach takes advantage of two models to increase brightness, contrast, clarify details, increase visibility, and remove noise using many filters. For example, Gao et al.[135] developed a method for restoring and enhancing underwater images. First, it drew on the prior dark channel in the image dehazing field to rectify and estimate the bright channel image, transmittance image, and atmospheric light. After applying these methods, restoration was performed. Second, these restored images were enhanced very effectively through histogram equalization with excellent results.

Zhou et al. [136] proposed a technique for underwater image restoration and enhancement to solve image problems, such as the lack of details and color deviation. This technique enhanced the visual effect and quality in underwater images. First, this method applies color restoration by adjusting the pixel value. Then, for color enhancement, the histogram is applied on the H channel to the source of the underwater images. Finally, for image enhancement, the edge preservation method is used. This method is very effective and accurate. Luo et al. [137] presented a technique for restoring and enhancing underwater images. Three techniques were applied: contrast optimization, color balancing, and histogram stretching. For color balancing, the scalar values of the R, G, and B channels were renewed to match the three channels’ distributions. Then, the optimized contrast algorithm was applied. The histogram stretching technique depends on the red channel for contrast and brightness improvement. This method enhances the underwater image quality and increases the contrast.

Dewangan et al. [138] developed a method for restoring and enhancing underwater images. It improves the underwater image clarity. They used HSV filters to enhance the images. It does not need any segmentation. The applied restoration and enhancement were accurate for object detection. Also, this method applied haze removal from underwater images that helped produce depth information from the vision techniques. This method includes many steps for underwater image enhancement that do not eliminate unwanted noise but enhance an image’s illumination, contrast, and visual quality.

Sequeira et al. [139] presented a method for enhancing and restoring underwater images that depends on processing them. This method applies the innovative algorithm for underwater image restoration and enhancement in a single underwater image processing technique. Their restoration depends on an effective red channel algorithm for the blue channel, as the red color channel has no intensity underwater. After that, the integrated color model is applied for underwater image enhancement. The results of these algorithms showed high contrast and were more realistic.

Daway et al. [140] developed a method for the restoration and enhancement of underwater images. Images were improved by applying color correction. The color restoration depended on the integrated color model with Rayleigh distribution (CRBICMRD). which was used for color restoration in the RGB and YCbCr model for color transformation. This method applied the multiscale retinex technique with unsupervised color correction, color restoration, and Rayleigh stretching. It was highly effective and improved the images quality.

Zhou et al. [21] developed a technology for resolving the low contrast and color distortion problems. It depended on the Jaffe-McGlamery model. The maximum bright proportions were applied for color correction of the underwater images. After that, a histogram was applied for contrast enhancement. Finally, two-level wavelet decomposition of the color-corrected and contrast-stretched underwater images was performed.

5. Underwater Image Analysis Techniques

This section reviews the most prominent image analysis techniques, including histogram equalization, adaptive histogram equalization, CLAHE, histogram sliding, brightness preserving bi-histogram equalization, generalized unsharp masking, contrast stretching, noise filtering, discrete wavelet transform, discrete cosine transform, and dark channel prior.

5.1. Histogram Equalization (HE)

The histogram equalization (HE technique) is applied to improve underwater images and increase the contrast in images [141]. It extends the intensity of the gray-level across the entire range in underwater images and enhances the underwater images’ contrast using a histogram. This technique is highly effective in cases of low-contrast images. If the image has high contrast, this method may aggravate the condition. It is calculated by Equation 3.

$$H\left(i\right)=round(\frac{cdf\left(i\right)-cd{f}_{min}}{(M\ast N-cd{f}_{min})}\ast (L-1))$$

(3)

where i is the pixel value. $M\ast N$ represents image pixels. L is the gray level number. $cd{f}_{min}$ is the value of minimum non-zero of cumulative distribution function.

5.2. Adaptive Histogram Equalization (AHE)

This modified version of the HE technique [142] applies multiple histograms to various sections of the same underwater image to improve the contrast. In HE, the same transform functions are applied to transform each pixel in the underwater image, consequently this technique is inadequate for image enhancement. AHE uses different transform functions to transform each pixel in the image to improve contrast. AHE solves the problems of HE, but its computational cost is high.

5.3. Contrast Limited Adaptive Histogram Equalization (CLAHE)

CLAHE is a modified version of the AHE technique [143]. AHE causes excessive noise amplification in underwater images but CLAHE limits this noise by decomposing the underwater image into several sub-blocks and performing HE on each part of the entire image. The disadvantages of CLAHE are that it generates ring and noise artifacts in the flat regions in images [144].

$$J=(j_{max}-j_{min})\ast P\left(f\right)+j_{min}$$

(4)

where j is the new value of pixel and $P\left(f\right)$ is the cumulative probability distribution [145].

5.4. Histogram Sliding

Histogram sliding is a technique for graphically illustrating pixel intensity values [146]. It is applied to manipulate brightness in underwater images. It adjusts the darkness or brightness in images to maintain the relationships between gray-level values. When sliding it, the entire histogram is shifted only either to the right or left. This makes underwater images clearer. It is calculated by adding or subtracting a fixed number of gray-level values.

$$F^{\sim }(i,j)=F(i,j)+offset$$

(5)

where the offset is the amount of histogram sliding. If the offset is positive, the contrast increases and the image is brighter, but if it is negative, the image is darkened.

5.5. Brightness Preserving Bi-Histogram Equalization (BBHE)

BBHE splits the degraded underwater image into two different and distinct images depended on the average of the input image [147]. For every image, histogram equalization is computed to improve it. If the first enhanced image has an intensity lower than the mean, it is scaled with intensity values between 0 and the mean. If the second equalized image has intensity values higher than the mean, t is scaled with the intensities values between the mean and 256. Although this method increases the image contrast, it requires complicated and specified hardware.

5.6. Generalized Unsharp Masking (GUM)

GUM is applied to enhance the sharpness and underwater image contrast [17]. It enhances underwater images by processing the residual and the model component. It reduces the halo effect by applying edge-preserving filter techniques. It also solves the out-of-range problem using tangent and logarithmic ratio methods. Although this method solves the halo effect and range problems, edge preservation is reduced.

5.7. Contrast Stretching

Contrast stretching is an effective and simple technique that improve the contrast of images by stretching the intensity values range [148]. It adjusts every image’s pixel value to apply the structure visualization to both the lighter and the darker regions of the underwater image. Image contrast is the difference value between the minimum and maximum pixel intensity. It has some disadvantages; in low-contrast images, specific details are very difficult to compute.

5.8. Noise Filtering

Noise filtering is a set of filters and processes for removing noise in underwater images. It is applied in many image processing applications. Many filters are applied to remove noise from underwater images. Filters are chosen according to the noise type and filter behavior. For example, to remove Gaussian noise, the Gaussian/Bilateral filter is applied, and the median filter is applied to remove salt and pepper noise. Figure 5 shows a classification of the various types of image noise filters.

Figure 5. Classification of image noise filters.

Figure 5. Classification of image noise filters.

5.9. Discrete Wavelet Transform (DWT)

The DWT divide an image into several sets, where each is a time series of coefficients that describe the time evaluation of the image in the frequency domain [149]. Function $F_n\left(x\right)$ is classified into the weighted sum of the base functions ${\varphi }_{jo,k}\left(x\right)$ and ${\psi }_{j,k}\left(x\right)$ by applying the DWT.

$$\begin{array}{cc}\hfill F_n\left(x\right)=& \frac{1}{\sqrt{M}}\sum _{k}w\varphi (jo,k){\varphi }_{jo,k}\left(x\right)+\hfill \\ & \frac{1}{\sqrt{M}}{\displaystyle \sum _{j=jo}^{\infty }}\sum _k.W\psi (j,k){\psi }_{j,k}\left(x\right),j,kϵZ\hfill \end{array}$$

(6)

where the $jo$ is the starting scale, M is the signal length. $w_{\varphi }(jo,k)$ and $W_{\psi }(j,k)$ are the approximation coefficients. Images have two-dimensional, but DWT has one dimension. Therefore, the tensor product is computed for wavelet and scaling functions. For the size M and N, the decomposition of images f(x, y), are calculated by Equation 8.

$$\begin{array}{cc}\hfill \varphi (x,y)=& \varphi \left(x\right)\varphi \left(y\right),{\psi }^H(x,y)=\psi \left(x\right)\varphi \left(y\right)\hfill \\ & {\psi }^v(x,y)=\varphi \left(x\right)\psi ,{\psi }^D(x,y)=\psi \left(x\right)\psi \left(y\right)\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill f(x,y)=& \frac{1}{\sqrt{M}}\sum _m\sum _{n}w\varphi (jo,m,n){\varphi }_{jo,m,n}(x,y)+\hfill \\ & \frac{1}{\sqrt{MN}}\sum _{i=H,V,D}{\displaystyle \sum _{j=jo}^{\infty }}\sum _m\sum _n{w}_{\psi }^i(j,m,n){\psi }_{i,m,n}^i(x,y)\hfill \end{array}$$

(8)

where $j,m,nϵZ$ decomposes the signal through high pass filter and low pass filter. X and Y are the signal directions and are applied for making the four sub-bands of the underwater image $w_{\varphi }(j,.,.)$, $W_{\psi }^H(j,.,.)$, $W_{\psi }^V(j,.,.)$, and $W_{\psi }^D(j,.,.)$. This approach decomposes the underwater image into four sub-bands: LH, LL, HH, and HL. For recomposing the underwater image, the inverse of DWT is based on approximation and detail coefficients.

5.10. Discrete Cosine Transform (DCT)

The DCT is the simplest transform technique applied in image compression and image processing applications [150]. It characterizes the underwater image as the sum of sinusoids having various frequencies and magnitudes. The purpose of the DCT is to focus most information in the signal’s low-frequency components owing to its powerful energy compression. It uses interpixel redundancies to increase the high-quality decorrelation of most images. The DCT decomposes the underwater image into sub-bands; each band is critical. It is calculated by using Equation 9.

$$F_n(u,v)=\alpha \left(u\right)\alpha \left(v\right){\displaystyle \sum _{x=0}^{N-1}}{\displaystyle \sum _{x=0}^{M-1}}.cos\left[\frac{\pi .u}{2.m}(2y+1)\right]$$

(9)

where $u=0,1,...,M-1$; and $v=0,1,...,N-1$ is the pixel intensity is $f(x,y)$ in row x and column y. The $\alpha \left(u\right)$ and $\alpha \left(v\right)$ functions calcuated by Equation 10.

$$\alpha \left(u\right),\alpha \left(v\right)\left\{\begin{array}{cc}\hfill \sqrt{\frac{1}{N}}& foru,v\le 0\hfill \\ \hfill \sqrt{\frac{2}{N}}& foru,v\ne 0\hfill \end{array}\right.$$

(10)

5.11. Role of Evolutionary Algorithms in Contrast Enhancement

EAs are robust and stochastic metaheuristics from evolutionary computing and are applied to solve optimization problems in image processing applications. These algorithms, such as particle swarm optimization (PSO) [151], artificial bee colony [152], and genetic algorithms [153] are applied to enhance the contrast of underwater images. These algorithms aim to maximize the suitability criterion for underwater image enhancement and are applied to compute the optimal gamma corrections parameters. Gamma correction is a simple and important technique that produces normal-looking images while retaining brightness. However, selecting the optimal value for gamma is a difficult task.

5.12. Dark Channel Prior (DCP)

DCP depends on the key observation and most of the local spots in haze-free outdoor images. These images include a few pixels with very low intensities in at least one color channel. For example, in the RGB channels, any of the three colors red, green, or blue, may have an intensity value of less than, or close to 0. This means that the minimum intensity value in the region has the smallest quantity value [66]. This method is very easy to implement. In addition, it requires less processing time and negates the halo effect.

6. Limitations

Exploring the underwater environment by capturing images is critical and is conducted by employing skilled divers, optical cameras, specialized hardware, or underwater ROVs. With the exception of optical cameras, all other systems have many disadvantages, such as limited field of view, depth limits, and complex processes. Due to the unexpected nonlinear hydrodynamic effects and the lack of an accurate model, the ROV control system is complicated.

Underwater exploration is expensive as it requires the use of highly skilled divers. For a single investigation, standby divers and supervisors may be required. Moreover, a limited time amount can be spent underwater, especially when a diver conducts inspections. Consequently, the time required for investigation has increased. Using underwater image enhancement techniques, this drawback can be considerably mitigated. Based on the aforementioned challenges, underwater image analysis limitations can be classified into environment-based and image-based limitations, as discussed in the following subsections.

6.1. Underwater Environment-based Limitations

These limitations are related to factors encountered in the underwater environment, such as equipment, refraction, non-uniform illumination, motion, scattering, and absorption. These limitations negatively affect underwater images and videos and make them hazy and degraded.

6.1.1. Equipment

Underwater images are captured using two camera equipment options. The first involves using an existing land camera with a housing unit. This housing unit must be enclosed with diving silicon to maintain a waterproof seal. This solution is the best option for photographers who have a high-quality land camera and are unable to buy an expensive underwater camera. The second option is using a specialized underwater camera. These specialized cameras differ in quality and price.

6.1.2. Refraction

Refraction describes how light bends as it passes from one medium to another [154]. When reflected from an object light travels through water and passes through the underwater camera’s glass and air, and the object appears approximately 25% larger and closer than they actually are. Refraction makes it difficult to focus sharply on the subject, leading to blurred photos. Refraction can be used to reconstruct the underwater scene [155,156].

6.1.3. Non-uniform Illumination

During underwater light propagation, light levels weaken as the depth increases. Natural light is not always available and varies depending on the time of the day. When the sun is directly overhead, the surface of the water reflects the least amount of light. The weather also influences the light availability. If the weather is stormy, turbulent water will significantly affect light conditions. There are many algorithms for solving non-uniform lighting [157,158,159].

6.1.4. Motion

Motion occurs when the relative position between the imaging device and the target object changes owing to movement. Such movements between the imaging device and target objects, in addition to the movement of underwater currents, cause motion blur [160,161,162], which affects underwater images. Motion blur produces distortions in the underwater image and degrades the luminance spectrum. Therefore, capturing images of stationary subjects such as corals or rock formations is easier than moving underwater objects. Many algorithms deblur underwater images for better clarity and quality [163].

6.1.5. Scattering

Scattering denotes the angular light distribution deflected by suspended particles in one direction at a specified wavelength [12,164,165]. It occurs by light falls on objects and is deflected and reflected many times by particles in the water before reaching the camera. Scattering degrades the contrast and visibility, blurs detail, and causes fogging in underwater images.

6.1.6. Absorption

Absorption occurs when one substance is absorbed or inextricably blended into another [83,166] and is divided into light absorption and color. Because water is a good natural light filter, it absorbs a fraction of the light that passes through it. The general formula for underwater light loss is that half the light is lost for every 10 meters of depth. Additionally, the available light is not constant and changes depending on the time of day and other factors such as the weather and surface conditions. The images do not exhibit a hue but green or blue for color absorption. As a result, the longer wavelengths of light, yellow, red, and orange are absorbed by the water.

6.2. Underwater Image-based Limitations

The limitations of underwater imaging have a degrading and hazy effect on the images’ quality. Therefore, it is critical to use restoration and enhancement algorithms to underwater images and videos. These limitations include low contrast, noise, blurring, poor visibility, and haze. Solving the limitations in underwater imaging is easier than changing the environmental limitations.

6.2.1. Low Contrast

Contrast is the computed difference in color or luminance that allows objects to be distinguishable from other objects through the same field of view [167]. Poor illumination, scattering, and absorption reduce contrast. This low contrast degrades the underwater images and reduces visibility and details thus contrast enhancement is a critical process and deciding whether the contrast is global or local is crucial. Local contrast means dividing the image into small regions, and contrast enhancement is performed on each independently. Global contrast indicates an increase of contrast in the entire image.

6.2.2. Noise

Image noise denotes random variations or changes in color or brightness [13,168]. This noise affects the resolution of the underwater images. There are several types of noise, such as:

• Salt and pepper noise: This signifies the smaller and larger grayscale values of a specified pixel or region.
• Gaussian noise: This is the most common noise type and is a statistical noise with a probability density function equal to normal distribution.
• Fixed mode noise: It is the underwater clutter that degrades the image.

Such noise degrades underwater images, therefore, noise removal methods are critical for image enhancement.

6.2.3. Blurring

Blurring is used for removing the edges to smooth an image. In the case of underwater images, blurring degrades them and obscures detail [169,170,171]. Because light is scattered and absorbed, underwater images are severely distorted by blurring and color cast, which decreases the image quality. The low quality makes it difficult to process images for object detection, classification, and segmentation. Due to the blurring effects in underwater images, deblurring methods are in high demand for image enhancement and restoration.

6.2.4. Poor visibility

Visibility means whether objects are detectable by sight or the distance at which light or objects can be discerned. The difficulty associated with ensuring the objects visibility at long or short distances in underwater scenes poses a challenge for the image processing community. Due to the backscatter and absorption caused by noise and suspended particles, such images suffer from poor visibility, which is a major problem for oceanic applications in computer vision. Light attenuation limits the viewing distance to about twenty meters in clear water and five meters or less in turbid and murky water. The poor objects visibility at short or long distances in underwater scenes causes image processing problems. Many studies have improved the visibility of underwater images [13,172].

6.2.5. Hazy

Hazy images are captured in foggy or hazy weather conditions and degraded by absorption and scattering. These hazy images have weak contrast and low visibility, rendering it more difficult to identify the objects in images by human vision. Due to the many hazy effects on underwater images, the enhancement of hazy underwater images is important [173,174]. Figure 6 indicates some examples from the datasets of underwater images with different limitations.

Figure 6. Some examples of underwater images from available datasets with various scenes limitation: (a) low light, (b) low contrast, (c) haze, and (d) blur.

Figure 6. Some examples of underwater images from available datasets with various scenes limitation: (a) low light, (b) low contrast, (c) haze, and (d) blur.

7. Underwater Image Datasets

In this section, we present the underwater imaging datasets that researchers use for enhancing and restoring underwater images. There is no complete dataset for underwater imaging because collecting underwater images is very difficult. Most underwater datasets have limitations such as few categories, single target objects, and imperfect information for labeling. Figure 7 shows some examples of images from underwater images datasets.

Figure 7. Samples of underwater images from the available underwater datasets.

Figure 7. Samples of underwater images from the available underwater datasets.

• Real-World Underwater Image Enhancement (RUIE) Dataset
  The RUIE dataset [175] is a large-scale dataset that contains 4000 underwater images from multiple views. According to the underwater image enhancement network (UIE) algorithms, the RUIE dataset is classified into three subsets: the underwater image quality set (UIQS), underwater color cast set (UCCS), and underwater task-oriented test suite (UHTS), as presented in Table 9. These subsets are used to restore color cast, enhance visual appearance, and aid in computer vision detection/classification at a higher level.
• Underwater Image Enhancement Benchmark (UIEB) Dataset
  The UIEB dataset [130] contains 950 real-world underwater images, 890 of which have a corresponding reference image. The remaining 60 were retained as testing data. This dataset is used in qualitative and quantitative underwater image enhancement algorithms. The UIEB dataset includes many levels of resolution and covers several scene/main object categories.
• Enhancement of Underwater Visual Perception (EUVP) Dataset
  The EUVP dataset [176] is a large-scale dataset that includes a paired and unpaired collection of low and good-quality underwater images used for adversarial supervised learning. These images were collected using seven different cameras in different situations. The unpaired data was collected by six human assistants and the paired data was collected by relying on human perception. This dataset includes 12K paired and 8K unpaired images, as shown in Table 7 and Table 8.

  Table 7. EUVP paired dataset.

  Dataset Name	Training Images	Validation Images	Total Images
  Underwater Dark	5550	570	11670
  Underwater ImageNet	3700	1270	8670
  Underwater Scenes	2185	130	4500

  Table 7. EUVP paired dataset.

  Dataset Name	Training Images	Validation Images	Total Images
  Underwater Dark	5550	570	11670
  Underwater ImageNet	3700	1270	8670
  Underwater Scenes	2185	130	4500

  Table 8. EUVP unpaired dataset.

  Poor quality	Good quality	Validation	Total Images
  3195	3140	330	6665

  Table 8. EUVP unpaired dataset.

  Poor quality	Good quality	Validation	Total Images
  3195	3140	330	6665

• U-45 Dataset
  The U-45 dataset [177] is an effective public underwater test dataset that includes 45 underwater images chosen from among real underwater images. This dataset contains the low contrast, color casts, and haze-like effects that contribute to image degradation.
• Jamaica Port Royal Dataset
  The Jamaica Port Royal dataset [178] was gathered in Port Royal, Jamaica, at the site of a submerged city containing both natural and artificial structures. These images were collected using a handheld diver rig. Sixty-five hundred images were collected during a single dive at a maximum depth of 1.5 m above the seabed.
• Marine Autonomous Robotics for Interventions (MARI) Dataset
  The MARI dataset [179] aims to improve the development of cooperative AUVs for underwater interventions in offshore industries, rescue, search, and various types of scientific exploration tasks. This dataset presents diverse underwater videos and images captured underwater by a stereo vision system.
• MOUSS
  The MOUSS dataset [24] was obtained by using a stationary camera on the ocean floor. At 1–2 m, with sufficient ambient lighting, 159 images of fish and other relevant objects were acquired. The test dataset was a combination of images from training and new collections.
• MBARI Dataset
  The MBARI dataset [24] was collected from different regions and consisted of 666 images of fish and other relevant objects. This dataset was obtained by the Monterey Bay Aquarium Research Institute.
• AFSC Dataset
  The AFSC dataset [24] was collected from the ROV that was placed underwater and equipped with an RGB video camera. It consisted of numerous videos from various ROV missions and contained 571 images.
• NWFSC Dataset
  The NWFSC dataset [24] was collected using a remotely operated vehicle and looking downward at the ocean floor. The first dataset contained 123 images of fish and other objects near the seabed.
• RGBD Dataset
  The RGBD dataset [24] collected for underwater image restoration and enhancement contained a waterproof color chart in the underwater environment. It consisted of over 1100 images.
• Fish4knowledge Dataset
  The Fish4knowledge dataset [180] consisted of fish data collected from a live video dataset. It had 27370 fish images. The entire dataset was divided into 23 clusters, with each distinct cluster representing a particular species.
• Wild Fish Marker Dataset
  The Wild Fish Marker dataset [181] was collected using a remotely operated vehicle under different ocean conditions. This dataset contained fish images depending on the cascade classifiers of Haar-like features. These images were not unconstrained as the underwater environment was variable because of the moving recording platform. It included an annotated training and validation dataset and independent test data.
• HabCam Dataset
  The HabCam dataset [24] was collected from underwater images on the seafloor. The HabCam vehicle was used for recording. It flew over the ocean taking six images in one second. These images are critical for studying the ecosystem and advancing the marine sciences.
• Port Royal Underwater Image Dataset
  The Port Royal underwater image dataset [178] was collected using a GAN to create realistic underwater images. These images were taken using a camera onboard autonomous as well as operated vehicles. This method is capable of recording high-resolution underwater images.
• OUCVISION Dataset
  The OUCVISION dataset [182] is a large-scale underwater image enhancement and restoration dataset which is used for recognizing and detecting salient objects. It contains 4400 images of 220 objects. Each object was taken with four pose variations (right, left, back, and front) and five spatial regions (bottom right, bottom left, center, top right, top left) to obtain 20 images.
• Underwater Rock Image Database
  The underwater rock image database [24] was collected to enhance and restore underwater images. It depended on the GAN to generate realistic underwater images.
• Underwater Photography Fish Database
  The underwater photography fish database [24] was collected from reef- and fish-life photographs taken in locations all over the world, such as the Indian Ocean, Red Sea, etc. This dataset contained many reef fish species, including Parrotfish, Butterflyfish, Angelfish, Wrasse, and Groupers. It also includes non-fish subjects like nudibranchs, corals, and octopi.
  Some underwater image datasets are available in https://github.com/xahidbuffon/Awesome_Underwater_Datasets. Table 9 indicates comparison between all datasets.

Table 9. List of underwater imaging datasets.

Dataset	Source	No. of Images	Objects	Resolution
RUIE  [175]	Dalian Univ. of Technology	UIQS 3630 (726 × 5) UCCS 300 (100 × 3) UHTS 300 (60 × 5)	Sea cucmbers, scallops, and urchins	400 × 300
UIEB [130]	—–	950	Diverse objects	Variable
EUVP [176]	—–	31505	Diverse objects	256 × 256
U-45 [177]	Nanjing Univ. of Information Science and Technology, China	45	Diverse objects	256 × 256
Jamaica Port Royal [178]	—–	6500	Fishes and other related objects	1360 × 1024
MARI [179]	—–	variable	Fished and other related objects	1292 × 964
MOUSS [24]	CVPR AAMVEM workshop	159	Fishes	968 × 728
MBARI [24]	Monterey Bay Aquariam Research Insitute	666	Fishes	1920 × 1080
AFSC [24]	CVPR AAMVEM workshop	571	Fishes and other related objects	2112 × 2816
NWFSC [24]	Integrated by CVPR AAMVEM workshop	123	Fishes and other related objects	2448 × 2050
RGBD [24]	Tel Aviv Univ.	1100	Diverse objects	1369 × 914
Fish4knowledge [180]	The Fish4knowledge team	images from underwater videos	Diverse objects	Variable
Wild Fish Marker [181]	NOAA Fisheries	1934 positive images and 3167 negative images, 2061 fish images	Fishes and other related objects	Variable
HabCam [24]	Integrated by CVPR AAMVEM workshop	10465	Sand dollars, scallops, rocks, and fishes	2720 × 1024
Port Royal Underwater image [178]	Real scientific surveys in Port Royal	18091	Artificial and natural structures	1360 × 1024
OUCVISION [182]	Ocean Univ. of china	4400	Artificial targets or rocks	2592 × 1944
Underwater Rock Image Database [24]	Univ. of Michigan	15057	Rocks in pool	1360 × 1024
The underwater Photography Fish Database [24]	Amateur contribution	8644	Reef fishes, Coral, and others	variable

Table 9. List of underwater imaging datasets.

Dataset	Source	No. of Images	Objects	Resolution
RUIE  [175]	Dalian Univ. of Technology	UIQS 3630 (726 × 5) UCCS 300 (100 × 3) UHTS 300 (60 × 5)	Sea cucmbers, scallops, and urchins	400 × 300
UIEB [130]	—–	950	Diverse objects	Variable
EUVP [176]	—–	31505	Diverse objects	256 × 256
U-45 [177]	Nanjing Univ. of Information Science and Technology, China	45	Diverse objects	256 × 256
Jamaica Port Royal [178]	—–	6500	Fishes and other related objects	1360 × 1024
MARI [179]	—–	variable	Fished and other related objects	1292 × 964
MOUSS [24]	CVPR AAMVEM workshop	159	Fishes	968 × 728
MBARI [24]	Monterey Bay Aquariam Research Insitute	666	Fishes	1920 × 1080
AFSC [24]	CVPR AAMVEM workshop	571	Fishes and other related objects	2112 × 2816
NWFSC [24]	Integrated by CVPR AAMVEM workshop	123	Fishes and other related objects	2448 × 2050
RGBD [24]	Tel Aviv Univ.	1100	Diverse objects	1369 × 914
Fish4knowledge [180]	The Fish4knowledge team	images from underwater videos	Diverse objects	Variable
Wild Fish Marker [181]	NOAA Fisheries	1934 positive images and 3167 negative images, 2061 fish images	Fishes and other related objects	Variable
HabCam [24]	Integrated by CVPR AAMVEM workshop	10465	Sand dollars, scallops, rocks, and fishes	2720 × 1024
Port Royal Underwater image [178]	Real scientific surveys in Port Royal	18091	Artificial and natural structures	1360 × 1024
OUCVISION [182]	Ocean Univ. of china	4400	Artificial targets or rocks	2592 × 1944
Underwater Rock Image Database [24]	Univ. of Michigan	15057	Rocks in pool	1360 × 1024
The underwater Photography Fish Database [24]	Amateur contribution	8644	Reef fishes, Coral, and others	variable

8. Underwater Image Quality Evaluation Metrics

Assessing the underwater images quality is an essential task that can be used automatically and accurately. Image quality assessment (IQA) approaches are categorized to include: (a) objective and (b) subjective methods [183,184] for automatically assessing images’ quality.

Subjective image quality metrics are time-consuming, expensive, and not not sufficient for most real-time applications. Objective image quality assessment techniques apply mathematical and statistical models that rely on the human visual system (HVS) to compute images’ quality.

Objective IQA techniques are classified into three groups; full reference (FR), reduced reference (RR), and no reference (NR), as indicated in Figure 8. with FR IQA the underwater reference image is available. With the RR AQI, partial information from underwater images is available. With NR IQA, the reference image is not applicable. In addition to the conventional evaluation metric, to evaluate the underwater image quality effectively, specialized metrics are presented in the literature as defined below and listed in Figure 9.

Figure 8. Classification of objective image quality assessment methods.

Figure 8. Classification of objective image quality assessment methods.

Figure 9. Classification of specialized underwater image quality assessment metrics.

Figure 9. Classification of specialized underwater image quality assessment metrics.

• Mean Square Error (MSE): calculates the squared error between the original and enhanced images [98]. The lower the MSE, the better the quality and the less error. The MSE is computed mathematically using Equation 11.

  $$MSE=\frac{1}{MN}{\displaystyle \sum _{i=1}^M}{\displaystyle \sum _{j=1}^N}{[F(i,j)-E(i,j)]}^2$$

  (11)

  where the $M\times N$ is the image size, $F(i,j)$ is the original image, and $E(i,j)$ is the enhanced image.
• Peak-Signal-to-Noise Ratio (PSNR): computes the peak error and is the percentage of the quality measurement between the original, and enhanced images [98]. The greater the PSNR, the good the reconstructed or enhanced image quality. It is calculated from the MSE using Equation 12.

  $$PSNR=20{log}_{10}\frac{MA{X}_F}{\sqrt{MSE}}$$

  (12)

  where $MA{X}_F$ is the maximum pixel value in an image and is 255 in case of gray level image.
• Entropy: represents a statistical value of the information in the image. It represents the randomness degree in the image that can be applied to characterize the texture of the image [185,186]. The higher entropy value indicates that the image has minimal information loss. It is computed by using Equation 13.

  $$H\left(F\right)=-{\displaystyle \sum _{i=0}^{255}}{p}_i{log}_2{p}_i$$

  (13)

  where i is the gray level number in a pixel in the image F and $p_i$ is the probability of intensity i.
• Structure Similarity Index Measure (SSIM): is applied to compute the similarity value between the original and enhanced images. It is presented by Wang [187] and formulated in [188], [189]. x and y are the patch locations of two different images. The SSIM involves three measures: contrasts C(x, y), brightness B(x, y), and structure S(x, y). The greater the SSIM value, the better the enhancement and the less distortion. The SSIM is computed by using Equation 14.

  $$SSIM(F,E)=\frac{(2{\mu }_x{\mu }_y+C_1)+(2{\sigma }_{xy}+C_2)}{({\mu }_x^2+{\mu }_y^2+C_1)({\sigma }_x^2+{\sigma }_y^2+C_2)}$$

  (14)

  where ${\mu }_x$, ${\mu }_y$ are the values of means and the ${\sigma }_x^2$, ${\sigma }_y^2$ are the values of standard deviation of x and y patches of pixels. ${\sigma }_{xy}$ is the covariance value of x and y patches of pixels and $C_1={\left(k_{1}L\right)}^2$ and $C_2={\left(k_{2}L\right)}^2$ are the small constant values to prevent the instability. L is the dynamic range value of pixels, $K_{1}0.01$ and $K_2=0.03$.
• Colour Enhancement Factor (CEF): This is used to represent the enhancement effect. The greater the CEF, the better quality of the enhanced image. It is calculated by using Equation 15.

  $$CEF=\frac{CM\left(F^{\sim }\right)}{CM\left(F\right)}$$

  (15)

  where $CM\left(F^{\sim }\right)$ and $CM\left(F\right)$ denote enhanced and original images. The $CM\left(F\right)=\sqrt{{\sigma }_{\alpha }^2+{\sigma }_{\beta }^2}+0.3\sqrt{{\mu }_{\alpha }^2+{\mu }_{\beta }^2}$. ${\sigma }_{\alpha }^2$ and ${\sigma }_{\beta }^2$ are the standard deviation values, and ${\mu }_{\alpha }^2$ and ${\mu }_{\beta }^2$ are average values of $\alpha$ and $\beta$.
• Contrast to Noise Ratio (CNR): This is used to compute the underwater image quality [190]. It is the signal amplitude associated with the surrounding noise in underwater images.

  $$CNR(F,F^{\sim })=\frac{{\mu }_i-{\mu }_n}{{\sigma }_n}$$

  (16)

  where ${\mu }_i$ represents the original image average value, ${\mu }_n$ is the enhanced image average value, and ${\sigma }_n$ is the standard deviation.
• Image Enhancement Metric (IEM): computes the sharpness and contrast in an underwater image by classifying the image into non-overlapping blocks [191]. It represents the mean value ratio of the center pixel’s absolute difference from eight neighbors in the original and enhanced images. It is calculated by using Equation 17.

  $$IEM=\frac{{\displaystyle \sum _{l=1}^{k1}}{\displaystyle \sum _{m=1}^{k2}}{\displaystyle \sum _{n=1}^{k3}}{F}_{e,c}^{m,l}-F_{F,c}^{m,l}}{{\displaystyle \sum _{l=1}^{k1}}{\displaystyle \sum _{m=1}^{k2}}{\displaystyle \sum _{n=1}^{k3}}{F}_{F,c}^{m,l}-F_{F,n}^{m,l}}$$

  (17)

  where $k_1$ and $k_2$ are non-overlapping blocks. SFS and e are original and improved images. $F_{e,c}^{m,l}$ and $F_{F,c}^{m,l}$ are the intensities of the center pixel. $F_{e,n}^{m,l}$ and $F_{F,c}^{m,l}$ are the neighbours intensities from the center pixel.
• Absolute Mean Brightness Error (AMBE): This indicates the brightness that is preserved after image enhancement [192]. It is the value of the absolute difference between the average of the original and improved underwater images. A median AMBE value denotes good brightness.

  $$AMBE(F,e)={\mu }_F-{\mu }_e$$

  (18)

  where ${\mu }_F$ and ${\mu }_e$ are the average values of the original and improved image.
• Spatial Spectral Entropy based Quality index (SSEQ): This is an efficient and accurate image NR IQA model presented by [193]. It computes the underwater image quality when it is affected by many distorting factors. It is computed by using Equation 19.

  $$E=-\sum _{i}P(i,j){log}_{2}P(i,j)$$

  (19)

  where P(i, j) represents the spectral probability map that is computed by Equation 20.

  $$P(i,j)=\frac{C{(i,j)}^2}{{\sum }_i{\sum }_{j}C{(i,j)}^2}$$

  (20)
• Measure of Enhancement (EME): This computes the contrast in underwater images and assists in selecting the processing parameters [194,195]. It is computed using Equation 21.

  $$EM{E}_{m_1{m}_2}=max(\frac{1}{m_1{m}_2}{\displaystyle \sum _{i=1}^{m_1}}{\displaystyle \sum _{n=1}^{m_2}}20log\frac{X_{max;n,1}^w}{X_{min;n,1}^w})$$

  (21)

  where $X_{max;n,1}^w$ and $X_{min;n,1}^w$ are the maximum and minimum values of image within the block $w_{n,1}$.
• Measure of Enhancement by Entropy (EMEE): It computes the contrast in underwater images [194,195].The greater the EMEE value, the better quality of the image. It is calculated by using Equation 22.

  $$\begin{array}{cc}\hfill EM{E}_{m_1{m}_2}=& max(\frac{1}{m_1{m}_2}{\displaystyle \sum _{i=1}^{m_1}}{\displaystyle \sum _{n=1}^{m_2}}\alpha \frac{X_{max;n,1}^w{\left(\theta \right)}^{\alpha }}{X_{min;n,1}^w\left(\theta \right)}\ast log\frac{X_{max;n,1}^w\left(\theta \right)}{X_{min;n,1}^w\left(\theta \right)})\hfill \end{array}$$

  (22)

  where $m_1,m_2$ are the blocks into which the underwater image is divided.
• Root Mean Square Error (RMSE): It is applied to compute the difference value between the original and enhanced images. It calculates the square root of MSE. The lower the RMSE value, the better contrast value for underwater images. It is calculated by using Equation 23.

  $$RMSE=\sqrt{\frac{}{MN}\sum _{i=1}^M\sum _{j=1}^N{][F(i,j)-e(i,j)]}^2}$$

  (23)

  where F and e represent the original and improved images.
• Underwater Colour Image Quality Evaluation metric (UCIQE): It is a linear combination of saturation, contrast, and chroma [196]. It computes the effects of low contrast, non-uniform color cast, and blur issues that degrade underwater images. It converts the RGB space into the CLELAB color space as it approximates the human eye’s visual perception. The higher UCIQE value means that underwater images have a good balance between contrast, chroma, and saturation. It is computed using Equation 24.

  $$\begin{array}{cc}\hfill UCIQE& =c_1\times {\sigma }_{chromo}\hfill \\ \hfill & +c_2\times contras{t}_1+c_3\times {\mu }_{saturation}\hfill \end{array}$$

  (24)

  where $c_1$, $c_2$, and $c_3$ represent the weighted coefficients, ${\sigma }_{chromo}$ indicates the standard deviation, $contras{t}_1$ is the contrast, and ${\mu }_{saturation}$ represents the average value of saturation.
• Underwater Image Quality Measure (UIQM): measures the quality of underwater images and depends on the model of the human visual system and functions without the reference image [197]. It relies on the feature or measuring component of the underwater images to represent the visual quality. It consists of three measurements, the underwater image sharpness measurement (UISM), the underwater image color measurement (UICM), and the underwater image contrast measurement (UIConM). A higher UIQM value denotes a higher quality value for underwater images.

  $$\begin{array}{cc}\hfill UIQM=& Coef{f}_1\times UICM+Coef{f}_2\times UISM+\hfill \\ & Coef{f}_3\times UIConM\hfill \end{array}$$

  (25)
• Colourfulness Contrast Fog density index (CCF): This computes the color quality of underwater images and is the non-referenced IQA model [198]. It is a weighted combination of contrast, the colorfulness index, and fog density. It is calculated using Equation 26.

  $$\begin{array}{ccc}\hfill CCF=& w_1\times Colorfulness+w_2\times contrast+\hfill & \\ & w_3\times Fogdensity\hfill \end{array}$$

  (26)
  The colorfulness results from absorption and blurring, whereas low contrast, caused by forwarding scattering and fog density, is due to backward scattering.
• Average Gradient (AG): This is a full reference method that measures the underwater images sharpness. It computes the rate change per minute as it presents in underwater images. It is computed using Equation 27.

  $$\begin{array}{cc}\hfill AG=& \frac{1}{(L-1)(M-1)}{\displaystyle \sum _{i=1}^{L-1}}{\displaystyle \sum _{J=1}^{M-1}}\hfill \\ & \ast \sqrt{{\left({\nabla }_{x}F(i,j)\right)}^2+{\left({\nabla }_{y}F(i,j)\right)}^2}\hfill \end{array}$$

  (27)

  where L and M represent the width and height of the underwater image, ${\nabla }_x$, and ${\nabla }_y$ are the gradient in the x and y directions.
• Patch-based Contrast Quality Index (PCQI): This predicts the perceived distortion of contrast to the human eye [199]. It is based on the patch model instead of relying on global statistics. It is based on three independent image quantities: structure, signal strength, and average. The greater the PCQI value, the better contrast values in underwater images. It is computed using Equation 28.

  $$PCQI(i,j)=\frac{1}{P}{\displaystyle \sum _{k=1}^P}{l}_r(i_k,j_k)l_s(i_k,j_k)l_t(i_k,j_k)$$

  (28)

  where P represents the patch number in the underwater image. $l_r$, $l_s$, and $l_t$ are the comparison functions.
• Normalized Cross-Correlation (NCC): This evaluates the underwater images quality by calculating similarities between the enhanced and original images. It represents the correlation value in the image group [200]. The brightness of an underwater image varies due to lighting conditions, so this is the essential reason for normalizing the image. NCC produces a result value between -1 and 1. If the underwater images are uncorrelated, the value is 1; if the underwater images are perfectly correlated, the value is -1.

  $$NCC=\frac{{\sum }_{i=1}^M{\sum }_{j=1}^{N}F(i,j)\times F^{\sim }(i,j)}{{\sum }_{i=1}^M{\sum }_{j=1}^N{\left(F^{\sim }(i,j)\right)}^2}$$

  (29)

  where $F(i,j)$ is the original image and $F^{\sim }(i,j)$ is the enhanced image. i and j are the image coordinates. M and N are the pixel numbers in horizontal and vertical coordinates.
• Average Difference (AD): The average difference value calculates the differences between filtered and low-quality images [200]. It calculates the mean value between the original and the processed image. This measurement is quantitative and is applied for object detection and recognition applications. Many image processing applications find the average value of the difference value between images through this quantitative measure. The image quality is very poor when the AD value is too high. It is computed using Equation 30.

  $$AD=\frac{1}{MN}{\displaystyle \sum _{i=1}^M}{\displaystyle \sum _{j=1}^N}{F}^{\sim }(i,j)-F(i,j)$$

  (30)

  where $F^{\sim }(i,j)$ is the enhanced image and $F(i,j)$ is the original image at i,j coordinates. M and N are the number of image pixels in the horizontal and vertical coordinates.

  $$AD=\frac{1}{MN}{\displaystyle \sum _{i=1}^M}{\displaystyle \sum _{j=1}^N}{F}^{\sim }(i,j)-F(i,j)$$

  (31)

  where $F^{\sim }(i,j)$ is the enhanced image and $F(i,j)$ is the original image at i and j coordinates. M and N are pixels of image in horizontal and vertical direction.
• Maximum Difference (MD): This computes the maximum error signals by calculating the difference between the original and enhanced underwater images [201]. It uses a low-pass filter for the sharp edges of underwater images. It is similar to AD. The higher the MD value, the poorer the underwater images.

  $$MD=Max{F}^{\sim }(i,j)-F(i,j)$$

  (32)

  where $F(i,j)$ is the original image and $F^{\sim }(i,j)$ is the enhanced image.
• Normalized Absolute Error (NAE): This computes the underwater images’ quality [202]. The NAE value is inversely proportional to the image quality. If the NAE value is higher, the quality of the underwater image is poorer.

  $$NAE=\frac{{\sum }_{i=1}^N{\sum }_{j=1}^{M}Abs\left(\left(F^{\sim }(i,j)\right)\right)}{{\sum }_{i=1}^N\left(\left(F(i,j)\right)\right)}$$

  (33)

  where $F(i,j)$ is the original image and $F^{\sim }(i,j)$ is the enhanced image. $Abs$ is the absolute error in the underwater image.

The evaluation of algorithms used for enhancing and restoring underwater images of different categories is very important. These evaluation measures provide the scores that represent the similarity or distortion between the original and the enhanced images. These evaluations help to estimate the best parameters for use in different applications. Underwater image quality metrics (IQM) help estimate the quality of the underwater enhancement and restoration algorithms.

9. Performance Evaluation

Experiments were tested on several of the 890 images in the UIEB dataset for evaluating the quality, quantity and computational complexity of the enhancement and restoration. Several restoration and enhancement algorithms were tested on these selected images using six common evaluation metrics. This section has been divided into four subsections: Experimental Setting, Qualitative Evaluation, Quantitative Evaluation, and Computational Complexity.

9.1. Experimental Setting

Extensive experiments concerning subjective and objective evaluation were conducted on various techniques for enhancing and restoring underwater images. The computer configuration for these experiments was an Intel(R) core (TM) i7-9750H CPU @2.60 GHZ (Lenovo, Beijing, China); 16 GB RAM; Microsoft Windows 10 (Microsoft, Redmond, WA, USA); MATLAB R2018a and python 3.6.

9.2. Qualitative Evaluation

Subjective evaluation is critical for visualizing the underwater image restoration and enhancement effects. Figure 10 presents the subjective results for the restoration of five raw images selected from the UIEB dataset using the following restoration algorithms: DCP [75], UDCP [78], MIP [70], IBLA [81], and ULAP [72]. This figure indicates that the IBLA and ULAP algorithms restoration characteristics results were superior because they adopt all underwater light attenuation to create the correct depth map. Recent restoration methods only dehaze underwater images and cannot deal with color restoration effectively for multiple underwater images. Therefore, the color corrections algorithms can be used in preprocessing to enhance the color, brightness and contrast in restored images.

Figure 11 presents the subjective evaluation results for images enhancement on the same five images using the following enhancement algorithms: HE [203], CLAHE [204], ICM [96], UCM [205] and RGHS [99]. From these results, we note that the HE enhancement results are inferior. CLAHE equally distributes red, green, and blue pixels thus enhancing underwater images and outperforming HE. ICM indicates the equalization in the color casts. The UCM enhancement results are superior to those of ICM. RGHS is based on adaptive parameters to avoid blind pixel redistribution or global histogram stretching to reduce sharpness. RGHS exhibits a greater dehazing effect.

Figure 10. Subjective comparative results for underwater images restoration on UIEB dataset. The results are generated by using DCP, UDCP, MIP, IBLA and ULAP.

Figure 10. Subjective comparative results for underwater images restoration on UIEB dataset. The results are generated by using DCP, UDCP, MIP, IBLA and ULAP.

Figure 11. Subjective comparative results for underwater images Enhancement on UIEB dataset. The results are generated by using HE, CLAHE, ICM, UCM and RGHS.

Figure 11. Subjective comparative results for underwater images Enhancement on UIEB dataset. The results are generated by using HE, CLAHE, ICM, UCM and RGHS.

9.3. Quantitative Evaluation

A quantitative evaluation confirms the visual quality of the resultant underwater images by objective evaluation and also validates the methods’ effectiveness. Objective analysis is computed using image quality evaluation metrics such as SSIM, MSE, PSNR, PIQE, UCIQE, and UIOM. Table 10 and Table 11 respectively present the quality metrics for five restoration algorithms and five enhancement algorithms computed on five raw images from the UIEB dataset. To understand the obtained results, we should take into consideration the following points. The lower MSE value indicates noise or errors in the underwater images’ content. A higher PSNR indicates lower noise. A high PSNR and low MSE indicate a good resultant image. When the SSIM values are nearer to 1 it indicates a good similarity value. The higher the UIQM, the more balance in saturation contrast and the sharper are the underwater images. The lower the PIQE, the more the underwater images are enhanced. The higher the UCIQE, the more enhanced are the underwater images.

Table 10. Quantitative evaluation of different restoration algorithms using underwater images from UIEB dataset.

Image	Algorithm	MSE	SSIM	PSNR	PIQE	UCIQE	UIQM
(a)	DCP	838	0.6	18.89	18.43	0.48	3.11
	UDCP	775	0.9	19.23	20.92	0.51	2.71
	MIP	1198	0.6	17.34	27.59	0.53	1.28
	IBLA	1116	0.8	17.65	31.60	0.51	1.57
	ULAP	237	0.8	24.38	26.30	0.56	2.35
(b)	DCP	5886	0.7	10.43	29.68	0.40	2.25
	UDCP	12963	0.4	7.03	30.40	0.36	1.98
	MIP	1655	0.2	15.94	25.18	0.51	2.65
	IBLA	10530	0.3	7.90	30.57	0.42	1.33
	ULAP	3703	0.3	12.44	17.13	0.60	2.92
(c)	DCP	314	0.7	23.16	28.61	0.55	2.15
	UDCP	3655	0.6	12.50	28.35	0.52	2.03
	MIP	4661	0.5	11.44	45.58	0.63	0.96
	IBLA	1600	0.8	16.08	26.26	0.60	2.40
	ULAP	3675	0.8	12.47	27.10	0.63	2.46
(d)	DCP	2807	0.6	13.64	51.66	0.47	2.30
	UDCP	4617	0.5	11.48	53.30	0.46	1.79
	MIP	2025	0.7	15.06	50.96	0.50	1.76
	IBLA	974	0.6	18.24	47.02	0.50	1.37
	ULAP	1359	0.7	16.79	49.24	0.49	2.12
(e)	DCP	1864	0.8	15.42	22.43	0.52	2.30
	UDCP	4160	0.6	11.93	25.66	0.52	1.79
	MIP	7004	0.5	9.67	36.25	0.68	1.76
	IBLA	5152	0.5	11.01	35.72	0.54	1.37
	ULAP	1550	0.6	16.22	25.05	0.58	2.12

Table 10. Quantitative evaluation of different restoration algorithms using underwater images from UIEB dataset.

Image	Algorithm	MSE	SSIM	PSNR	PIQE	UCIQE	UIQM
(a)	DCP	838	0.6	18.89	18.43	0.48	3.11
	UDCP	775	0.9	19.23	20.92	0.51	2.71
	MIP	1198	0.6	17.34	27.59	0.53	1.28
	IBLA	1116	0.8	17.65	31.60	0.51	1.57
	ULAP	237	0.8	24.38	26.30	0.56	2.35
(b)	DCP	5886	0.7	10.43	29.68	0.40	2.25
	UDCP	12963	0.4	7.03	30.40	0.36	1.98
	MIP	1655	0.2	15.94	25.18	0.51	2.65
	IBLA	10530	0.3	7.90	30.57	0.42	1.33
	ULAP	3703	0.3	12.44	17.13	0.60	2.92
(c)	DCP	314	0.7	23.16	28.61	0.55	2.15
	UDCP	3655	0.6	12.50	28.35	0.52	2.03
	MIP	4661	0.5	11.44	45.58	0.63	0.96
	IBLA	1600	0.8	16.08	26.26	0.60	2.40
	ULAP	3675	0.8	12.47	27.10	0.63	2.46
(d)	DCP	2807	0.6	13.64	51.66	0.47	2.30
	UDCP	4617	0.5	11.48	53.30	0.46	1.79
	MIP	2025	0.7	15.06	50.96	0.50	1.76
	IBLA	974	0.6	18.24	47.02	0.50	1.37
	ULAP	1359	0.7	16.79	49.24	0.49	2.12
(e)	DCP	1864	0.8	15.42	22.43	0.52	2.30
	UDCP	4160	0.6	11.93	25.66	0.52	1.79
	MIP	7004	0.5	9.67	36.25	0.68	1.76
	IBLA	5152	0.5	11.01	35.72	0.54	1.37
	ULAP	1550	0.6	16.22	25.05	0.58	2.12

Table 11. Quantitative evaluation for enhancement of underwater images on UIEB dataset.

Image	Algorithm	MSE	SSIM	PSNR	PIQE	UCIQE	UIQM
(a)	HE	2472	0.7	14.19	19.61	0.60	3.31
	CLAHE	1055	0.8	17.89	18.74	0.57	3.53
	ICM	276	0.9	23.71	27.29	0.50	3.41
	UCM	146	0.9	26.47	18.05	0.52	3.17
	RGHS	156	0.9	26.19	19.93	0.53	2.08
(b)	HE	1530	0.6	16.28	15.96	0.61	3.30
	CLAHE	815	0.7	19.01	18.01	0.50	3.22
	ICM	1333	0.7	16.88	26.65	0.46	3.56
	UCM	3826	0.2	12.30	23.52	0.48	3.03
	RGHS	515	0.9	21.01	15.24	0.57	3.12
(c)	HE	2672	0.7	13.86	24.23	0.60	2.91
	CLAHE	1812	0.9	15.54	21.51	0.58	3.17
	ICM	594	0.9	20.39	16.76	0.56	2.71
	UCM	2781	0.8	13.68	18.25	0.61	2.88
	RGHS	531	0.9	20.87	23.42	0.58	2.27
(d)	HE	1575	0.7	16.15	45.38	0.59	2.70
	CLAHE	735	0.8	19.46	49.06	0.53	2.44
	ICM	1705	0.7	15.81	48.33	0.49	2.03
	UCM	1005	0.8	18.10	47.62	0.54	2.59
	RGHS	1274	0.7	17.07	49.73	0.55	2.71
(e)	HE	948	0.7	18.36	22.33	0.61	2.83
	CLAHE	506	0.9	21.08	23.63	0.56	2.98
	ICM	700	0.9	19.67	21.03	0.54	2.72
	UCM	654	0.8	19.96	20.65	0.56	2.78
	RGHS	413	0.9	21.96	21.13	0.58	2.71

Table 11. Quantitative evaluation for enhancement of underwater images on UIEB dataset.

Image	Algorithm	MSE	SSIM	PSNR	PIQE	UCIQE	UIQM
(a)	HE	2472	0.7	14.19	19.61	0.60	3.31
	CLAHE	1055	0.8	17.89	18.74	0.57	3.53
	ICM	276	0.9	23.71	27.29	0.50	3.41
	UCM	146	0.9	26.47	18.05	0.52	3.17
	RGHS	156	0.9	26.19	19.93	0.53	2.08
(b)	HE	1530	0.6	16.28	15.96	0.61	3.30
	CLAHE	815	0.7	19.01	18.01	0.50	3.22
	ICM	1333	0.7	16.88	26.65	0.46	3.56
	UCM	3826	0.2	12.30	23.52	0.48	3.03
	RGHS	515	0.9	21.01	15.24	0.57	3.12
(c)	HE	2672	0.7	13.86	24.23	0.60	2.91
	CLAHE	1812	0.9	15.54	21.51	0.58	3.17
	ICM	594	0.9	20.39	16.76	0.56	2.71
	UCM	2781	0.8	13.68	18.25	0.61	2.88
	RGHS	531	0.9	20.87	23.42	0.58	2.27
(d)	HE	1575	0.7	16.15	45.38	0.59	2.70
	CLAHE	735	0.8	19.46	49.06	0.53	2.44
	ICM	1705	0.7	15.81	48.33	0.49	2.03
	UCM	1005	0.8	18.10	47.62	0.54	2.59
	RGHS	1274	0.7	17.07	49.73	0.55	2.71
(e)	HE	948	0.7	18.36	22.33	0.61	2.83
	CLAHE	506	0.9	21.08	23.63	0.56	2.98
	ICM	700	0.9	19.67	21.03	0.54	2.72
	UCM	654	0.8	19.96	20.65	0.56	2.78
	RGHS	413	0.9	21.96	21.13	0.58	2.71

9.4. Computational Complexity

Although the quality of underwater image enhancement and restoration algorithms is critical, the computational time, especially for real-time applications, should also be considered. Computational time indicates how long an algorithm runs when enhancing or restoring images. Lower computational times mean that the algorithm is more effective. Each algorithm’s running time for restoration and enhancement is shown in seconds in Table 12.

Table 12. Computational time in seconds(secs) for restoration and enhancement algorithms of underwater images on UIEB dataset.

Image	DCP	UDCP	MIP	IBLA	ULAP	HE	CLAHE	ICM	UCM	RGHS
(a)	13	11.88	15.2	30	0.02	0.03	0.03	3	7.2	4
(b)	13	12	24	90	0.1	0.04	0.04	3.3	7	4
(c)	48	43	48	130	0.6	0.07	0.16	11.19	24	14
(d)	91	85	60	190	0.12	0.13	0.12	22.2	40	28
(e)	7	14	7	50	0.1	0.02	0.02	3	4	3

Table 12. Computational time in seconds(secs) for restoration and enhancement algorithms of underwater images on UIEB dataset.

Image	DCP	UDCP	MIP	IBLA	ULAP	HE	CLAHE	ICM	UCM	RGHS
(a)	13	11.88	15.2	30	0.02	0.03	0.03	3	7.2	4
(b)	13	12	24	90	0.1	0.04	0.04	3.3	7	4
(c)	48	43	48	130	0.6	0.07	0.16	11.19	24	14
(d)	91	85	60	190	0.12	0.13	0.12	22.2	40	28
(e)	7	14	7	50	0.1	0.02	0.02	3	4	3

10. Applications of Underwater Image Analysis

For increasing numbers of applications, capturing clear underwater videos and images is essential. Researchers use underwater images or videos to gain valuable and useful information while studying the underwater environment. In this section, the most common applications of this topic have been introduced.

10.1. Underwater Navigation

Autonomous navigation by underwater vehicles for exploring underwater resources is a popular research topic [17]. The main cause is the increasing need to collect underwater data such as mine detection and environmental monitoring. Some underwater vehicles are focused on improving underwater images.

10.2. Fish Detection and Identification

Boudhane et al. [206] developed a new method for preprocessing underwater images and detecting and locating fish. This method consists of three steps. First, the noise was removed by estimating the Poisson-Gaussian mixture and enhancing the underwater images. After that, they applied the mean shift technique to decompose the underwater image into regions. Finally, these regions were combined through an estimation that depended on the log-likelihood test.

Li et al. [207] presented a method for detecting fish food rests in underwater videos and images by applying adaptive thresholding. With the greatest accuracy, i.e., 95.6%. this approach was applied to minimize financial losses and waste. The expectation-maximization (EM) that depended on the Gaussian mixture algorithm was applied for histogram fitting and the histogram type identification for adaptive threshold computation.

Villon et al. [208] developed a new method to count and identify the types of coral reef fish in underwater videos and images by applying a CNN. This method was trained and its quality and performance were tested on several photographic databases with different post-processing decision rules to identify 20 types of fish. This method effectively and accurately detected either the whole or partial body of the fish.

Cui et al. [209] presented a new method for fish detection that uses a CNN with three optimization algorithms to increase the learning samples number and simplify it. The training process was made more efficient by accelerating it. Loss and training time were decreased by applying the dropout method and refining the loss function. The improvement in accuracy and the decrease in processing time showed the potential for the AUV implementation method.

10.3. Corrosion Estimation of Subsea Pipelines

Khan et al. [210] presented a new and effective method for estimating underwater pipeline corrosion through color information. First, underwater image enhancement and restoration were developed to improve degraded images. The offshore oil and gas industry has severe pipeline corrosion problems. which causes leaks and cracks. It was very difficult for human divers to follow the pipeline because of unfavorable conditions. In this work, corrosion estimation depended on wavelet transform was used to restore and enhance the underwater images.

10.4. Coral-reef Monitoring

Underwater digital imaging has improved data collection for monitoring benthic communities, but analysis of these underwater images remains difficult. A new and effective method by [211] was based on a deep learning CNN to analyze underwater images. This method used a global coral reef monitoring dataset and artificial intelligence for simulation, data processing, and decision-making. Several layers of the CNN were used for learning through feature extraction. Probabilistic inference was used to interpret the output to the network. The experimental accuracy of this method was 97%.

10.5. Sea Cucumber Image Enhancement

The products of sea cucumber are very rich in low-fat, high-quality protein, and vitamins. Such products contribute significantly to meeting people’s dietary needs since they depend on animal protein for nutrition. Li et al. [212] developed an effective and novel method for enhancing blurred, degraded, and color distorted underwater images. This method depended on the fusion of the prior dark channel and retinex. First, preprocessing that depended on the prior dark channel was used. Then, the Gaussian template and the underwater images were convolved to produce improved underwater images. Finally, the brightness and saturation were enhanced in the HSV color space.

Qiao et al. [213] developed the technique for the automatic segmentation of sea cucumber images taken underwater. First contrast enhancement was applied through the fusion of the CLAHE algorithm and the RGB color model. Then, rectangular edges and the sea cucumber edges were extracted and distinguished using active contour segmentation.

10.6. Other applications

Fatan et al. [214] developed a new method for tracking and detecting cables using an autonomous underwater vehicle. First, the edges of the underwater images were computed. Then, they were categorized based on the texture information using a support vector machine (SVM) and a multilayer perceptron (MLP) neural network. Subsequently, only the edges applied for the next processes remained. Finally, for tracking and detecting the cables, the filtered edges were processed using the Hough transform.

Zhou et al. [215] developed a method for detecting motion in underwater videos that is critical for different computer vision applications such as target tracking and recognition. This method depends on the enhancement of underwater images. It enhances the clarity and the target contrast through adaptive underwater color imaging algorithms and then extracts moving objects through the background model.

11. Future Directions

Underwater image analysis is expected to be an active research topic in various disciplines, as computer vision, pattern recognition, and machine learning, owing to its extensive and complex functional applications. Despite several effective and good studies and many trials, several promising research directions can still be suggested. The performance in many domains remains low compared with that of other underwater image restoration and enhancement techniques, causing problems such as deblurring, super-resolution, and dehazing. In the following paragraphs, some of the potential future directions are discussed.

• Efforts should be directed into noise removal as some research experiments cause high noise in underwater videos and images.
• Studies should be dedicated to real-time object tracking and detection from enhanced underwater images.
• Reducing the high computational cost and execution time required for restoring and enhancing underwater images.
• The performance of the enhancement methods of contrast is still poor in many aspects. Therefore, increasing the contrast in underwater images and videos is critical research that has attracted considerable attention in recent years.
• Underwater image datasets are primarily used for model testing rather than training. Although there are many underwater image datasets, a limited number of them contain a finite number of underwater images. Therefore, a more compact dataset that can enhance underwater images is needed.
• Evaluation metrics must be developed to consider more features in underwater videos and images, such as texture, noise, and depth estimation.
• Lightweight instruments and tools must be constructed to capture and take underwater images in challenging conditions.
• The computational efficiency and robustness of underwater imaging methods must be improved. The desired methods must be adaptable to diverse underwater conditions and effective strategies for different types of underwater applications should be developed. For recovering realistic scenes, the fusion of restoration and enhancement techniques improves the computational efficiency of underwater imaging. However, it is time-consuming to compute the two main parameters. Conversely, IFM-free methods can improve image quality by redistributing the pixel values and produce optimal color distributions.
• Several deep learning techniques, such as GAN to create the white balance and RNN to increase detail and decrease noise, should be used for underwater image enhancement. Learning-based underwater image enhancement methods depend heavily on datasets. These datasets require multiple numbers of paired and referenced images. Therefore, compiling a public benchmark dataset of various enhanced and hazed underwater images is essential.
• In the future, high-level tasks such as target detection through visibility degradation will be applied to evaluate underwater image enhancement methods. Current methods for underwater imaging focus on enhancing the perceptual effects but ignore whether enhanced imaging increases the accuracy and quality of analysis of high-level features for classification and detection. Therefore, the relationships between low-level underwater image enhancement and high-level classification and detection should be advanced.
• The methods for the enhancement of deep-sea underwater images differ from those used for shallow-water environments. The natural light propagated underwater is absorbed below 1000 meters; therefore, only artificial light sources strongly affect the images. The existing underwater image restoration and enhancement methods cannot recover deep sea underwater images. Therefore, to improve image quality and reduce halo effects, a new and effective imaging model for deep-sea imaging is required to resolve uneven illumination, light attenuation, scatter interference, and low brightness.