Deep Learning for Point-of-Care Ultrasound Image Quality Enhancement: A Review

1. Introduction

The use of handheld devices suitable for point-of-care ultrasound (POCUS) has been on the rise in recent years. This increase in popularity can be attributed to some key characteristics of these devices. Firstly, their portability makes them more convenient compared to conventional cart-based devices. Moreover, these handheld POCUS devices are more affordable than traditional ultrasound machines [1,2,3,4,5], making ultrasound technology more accessible and expanding its application beyond the radiology departments. This is particularly useful in situations where larger, more expensive ultrasound equipment is impractical, such as in bedside emergency settings, general practitioner offices, home care environments, and rural medicine facilities [6,7,8,9,10,11,12].

However, one of the primary drawbacks of ultrasound examination with a handheld device is the reduced imaging quality due to hardware limitations and the absence of sophisticated post-processing algorithms. These limitations can potentially lead to less accurate diagnoses [2,4]. Compared to conventional high-end ultrasound systems, handheld POCUS devices typically exhibit reduced resolution and contrast, less distinct texture or edges of structures, and increased noise levels [6,13,14,15]. Despite the advancements in POCUS technology in recent years, a trade-off remains between imaging quality and the benefits of cost and portability [14,16,17].

Efforts to enhance the quality of POCUS can be categorized into three main approaches. The first approach involves advancements in hardware. However, this approach is constrained by rising costs or compromised portability. Another option for quality improvement involves refinements in the ultrasound beam-forming algorithm [18,19]. Nevertheless, the accessibility of raw radio frequency (RF) signals required for these improvements is limited in most commercial ultrasound systems. Therefore, this systematic review opts to center its focus on a third alternative: modifications to the image post-processing methods, eliminating the need for hardware remodeling or operations on the raw RF signal.

Traditional post-processing techniques, such as filtering and deconvolution, have been employed for ultrasound image enhancement for some time, as described in the review by Ortiz et al. [20]. Over the last few years, deep learning has emerged as a powerful tool, achieving state-of-the-art performance in various image processing tasks, including image quality enhancement [21,22,23]. Lepcha et al. recently conducted a systematic survey on existing state-of-the-art image enhancement techniques, including deep learning [24]. However, to the best of the authors’ knowledge, there has been no recent literature review on the current status of deep learning-based image enhancement specifically focusing on ultrasound. This gap in the literature presents a compelling area for investigation, particularly given the affordability and flexibility of POCUS, alongside its inherent challenges related to image quality.

The aim of this systematic review is, therefore, to explore the current state-of-the-art progress in ultrasound image enhancement using deep learning for point-of-care ultrasound applications. In this review, we will categorize the quality enhancement methods used in the selected articles, provide an overview of the improvements in performance achieved by these methods, and assess the practical benefits and limitations of these deep learning algorithms in enhancing ultrasound image quality for clinical practice.

2. Materials and Methods

This systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [25].

2.1. Literature Search

2.1.1. Search Strategy

A literature search was conducted in PubMed and Scopus on January 29, 2024, covering publications from 2018 onwards. This decision was made due to the substantial growth of deep learning applications in medical contexts observed only in more recent years, especially in tasks related to image generation [21,26]. The search strategy was formulated to encompass three primary concepts: “Quality enhancement”, “Ultrasound” and “Deep learning”. It should be noted that even though this review focuses on the POCUS applications, the search included methods developed for general ultrasound imaging to ensure a comprehensive coverage of relevant algorithms. Synonyms and related keywords for each concept were identified and included in the search string, such as specific aspects of quality enhancement like denoising or specific types of deep learning networks like a convolutional neural network. The complete search strings for both databases are reported in Appendix A. Additionally, a snowballing search was conducted to identify related articles, and duplicates were removed during the screening process.

2.1.2. Eligibility Criteria

Studies were included if they met the following criteria: 1) focused on medical image quality enhancement, defined as increased spatial resolution, contrast enhancement, denoising, or enhancement of structure boundaries; 2) used post-image processing techniques on B-mode ultrasound images, i.e. image-to-image methods; 3) proposed a deep learning algorithm; 4) proposed algorithm was specifically developed for ultrasound images; 5) full-text original article was written in English.

Exclusion criteria included: 1) other quality enhancement methods like restoration and inpainting, or different study aims like domain conversion and 3D reconstruction; 2) hardware changes were required; 3) RF ultrasound data was used as input; 4) research on (microbubble) contrast-enhanced ultrasound, ultrasound computed tomography, elastography, color Doppler ultrasound, quantitative ultrasound, and high-intensity focused ultrasound; 5) non-journal publications (e.g. reviews, comments, dissertations, newspapers, and books); 6) non-accessible full-text publications.

2.1.3. Selection Procedure

The title and abstract of all studies were screened. Studies were excluded if they did not meet the eligibility criteria. For the remaining studies, the full text was retrieved and evaluated comprehensively. Each study was classified according to the quality enhancement aspects it addressed, which is explained in more depth in the paragraph below. This classification resulted in a final selection of articles for further assessment and quantification.

2.2. Categorization by Quality Enhancement Aspects

Papers published on quality enhancement in ultrasound were further grouped based on the specific distortions addressed, which are particularly relevant to POCUS imaging, namely: 1) spatial resolution; 2) contrast; 3) texture or detail enhancement; and 4) noise. The definitions of these quality enhancement aspects as implemented in this review are further specified in Table 1.

Given the multifaceted nature of distortions in handheld ultrasound and the necessity for real-time quality enhancement, this review focused on deep learning algorithms simultaneously addressing multiple quality enhancement aspects. However, these quality enhancement aspects can be closely related. For instance, the presence of noise reduces image contrast and resolution, thereby affecting edges and fine details [27,28]. Therefore, improving the quality of the ultrasound image by addressing one or more of the quality enhancement aspects should be specifically described and evaluated through a suitable performance metric.

Furthermore, studies were also included if they reported on the process of mapping low-quality images to high-quality reference images. This had to be achieved by obtaining ultrasound images that naturally showed a disparity in quality as a result of differences in the capture process, such as a different number of piezoelectric elements or the number of plane waves used, and not by artificially inducing quality reduction or improvement. Consequently, this led to the identification of a final category: 5) general quality improvement. The articles addressing either multiple quality enhancement aspects or quality improvement, in general, were selected for further descriptive and quantitative assessment.

Table 1. Definitions of quality enhancement aspects.

Quality enhancement aspect	Definition
1. Spatial resolution	The ability of differentiating two adjacent structures as being distinct from one another, either parallel (axial resolution) or perpendicular (lateral resolution) to the direction of the ultrasound beam [29].
2. Contrast resolution	The ability to distinguish between different echo amplitudes of adjacent structures through image intensity variations [29].
3. Detail enhancement of structures	Enhancement of texture, edges, or boundaries between structures.
4. Noise	Minimization of random variability that is not part of the desired signal.
5. General quality improvement	Mapping low-quality images to high-quality reference images, where the quality disparities are inherent to differences in the capture process and not artificially induced.

Table 1. Definitions of quality enhancement aspects.

Quality enhancement aspect	Definition
1. Spatial resolution	The ability of differentiating two adjacent structures as being distinct from one another, either parallel (axial resolution) or perpendicular (lateral resolution) to the direction of the ultrasound beam [29].
2. Contrast resolution	The ability to distinguish between different echo amplitudes of adjacent structures through image intensity variations [29].
3. Detail enhancement of structures	Enhancement of texture, edges, or boundaries between structures.
4. Noise	Minimization of random variability that is not part of the desired signal.
5. General quality improvement	Mapping low-quality images to high-quality reference images, where the quality disparities are inherent to differences in the capture process and not artificially induced.

2.3. Data Extraction

Two performance metrics were evaluated; the Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity Index Measure (SSIM). The PSNR and SSIM are commonly used metrics for assessing image quality, which can quantitatively show the effectiveness of the proposed networks. Both are full-reference metrics, which evaluate the quality of an image, by comparing it to a high-quality reference image. Data was extracted if the article reported either the PSNR or SSIM with standard deviation for both the low-quality input images as well as for the generated images by the proposed algorithm.

The PSNR is defined as the ratio of the maximum power of a signal and the power of the distorting noise [30]. It reflects the pixel-based similarity between the reconstructed image and the corresponding high-quality reference [14]. This ratio between two images is expressed in decibels (dB), thus following a $lo{g}_{10}$ scale. A higher value indicates that the reconstructed image contains more details and provides a higher image quality [6,31]. And vice versa, a small value of the PSNR implies high numerical differences between images [31]. Given a reference image f and a test image g, both of size MxN with maximum intensity $MA{X}_I$ and the Mean Squared Error ($MSE$) between f and g, the PSNR is calculated as follows:

$$\mathrm{PSNR}=10·{log}_{10}\left({\displaystyle \frac{{\mathrm{MAX}}_I^2}{\mathrm{MSE}(f,g)}}\right)$$

(1)

where MSE represents,

$$\mathrm{MSE}(f,g)={\displaystyle \frac{1}{MN}}\sum _{i=1}^M\sum _{j=1}^N{(f_{ij}-g_{ij})}^2$$

(2)

The SSIM evaluates the perceived quality and assesses the perceptual-based similarity between paired images [14,30]. It is considered to be correlated with the quality perception of the human visual system. Instead of using traditional error summation methods, the SSIM is designed by modeling any image distortion as a combination of three factors that are loss of correlation, luminance distortion, and contrast distortion. The SSIM index ranges between -1 and 1, with a value of 1 indicating perfect correlation, 0 indicating no correlation, and -1 indicating anti-correlation between the images [31].

For a reference image f and a test image g, where $\mu$ denotes the mean, ${\sigma }_f^2$ denotes the variance of f, ${\sigma }_{fg}$ denotes the covariance of f and g and $C1$ and $C2$ are two positive constants used to avoid a null denominator, the SSIM is defined as:

$$\mathrm{SSIM}(f,g)={\displaystyle \frac{(2{\mu }_f{\mu }_g+C_1)(2{\sigma }_{fg}+C_2)}{({\mu }_f^2+{\mu }_g^2+C_1)({\sigma }_f^2+{\sigma }_g^2+C_2)}}.$$

(3)

Additionally, the extracted data was grouped by set type (in vivo, phantom, or simulation).

2.4. Statistical Analysis

Statistical analysis were performed using IBM SPSS Statistics, Version 29.0.2 (Released 2023; IBM Corp., Armonk, New York, United States), using a random effects model. Statistical heterogeneity was evaluated by calculating I2 statistics, with high heterogeneity defined as >75% and statistical significance defined as p < 0.05. Mean differences were calculated by subtracting the quality performance values of the low-quality input image from the values of the generated image by the proposed algorithm. The results were summarized in forest plots.

3. Results

3.1. Study Selection

The systematic literature research identified 457 articles from the two databases after duplicate removal. Snowballing did not identify any additional relevant articles. The initial screening based on title and abstract resulted in 69 articles being selected for full-text review [6,14,15,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97]. Following an initial analysis focusing on quality enhancement aspects, 15 articles were identified that addressed multiple quality enhancement aspects. These articles were selected for further descriptive and quantitative assessment. Finally, 6 articles were selected for a meta-analysis based on their reported outcomes. The study selection process is illustrated in Figure 1.

3.2. Quality Enhancement Aspects

The distribution of quality enhancement aspects across the 69 included articles was analyzed, for which the findings are depicted in Figure 2. This figure shows a predominant focus on denoising, followed by resolution enhancement. Of the 69 articles, the majority (n=54) focused on a single quality enhancement aspect, while a smaller part (n=15) addressed a combination of aspects or quality enhancement in general. Consequently, these 15 articles [6,14,15,32,33,34,35,36,37,38,39,40,41,42,43,46,47,86,87,88,89] were selected for further analysis.

3.3. Study Characteristics

The datasets used in the 15 selected articles for further assessment were categorized into three types: 1) in vivo data; 2) phantom data (including in vitro, ex vivo, and tissue-mimicking datasets, both self-made and commercial); and 3) simulation data. For studies comparing multiple deep learning algorithms or loss functions, the best-performing algorithm or loss function and their corresponding performances were reported. Characteristics of the included articles are shown in detail in Table 2. Most articles [33,37,38,41,42,43] used plane wave imaging (PWI) for data collection, by mapping low-quality images from one or a few angles to high-quality compounded images from multiple angles. The next most common ultrasound mode for data collection involved low-quality input data from handheld POCUS and high-quality reference images from high-end ultrasound devices [6,14,15,35]. Additionally, all articles used (a combination of) a CNN and/or a GAN as a deep learning network. More variety in loss functions was observed, even though often (albeit in combination with other loss functions) the MSE and SSIM loss were used.

3.4. Study Outcomes

The outcomes of the selected studies are reported in Table 3, including the computation time, source code availability, number of images in test set, performance metrics for low-quality input images, and performance metrics for enhanced generated images. The performance of the proposed algorithms were categorized by dataset type (in vivo, phantom or simulation). Both full-reference metrics and non-reference metrics are reported.

Additionally, the quantitative outcomes for the most commonly reported performance metrics (PSNR and SSIM) are visualized in Figure 3 and Figure 4. Both the baseline performance metrics for the low-quality input images and the obtained performance metrics for the images generated by the proposed algorithm are shown. In these figures, each bar represents a dataset and the color represents the corresponding study. The baseline PSNR low-quality input images ranged from 8.65 to 29.24, while the generated images had PSNR values ranging from 13.99 to 36.59. Similarly, SSIM values ranged from 0.03 to 0.71 for low-quality input images and from 0.30 to 0.93 for the enhanced images.

3.5. Meta-Analysis Results

Six studies reported PSNR values[6,14,32,34,37,39] and five reported SSIM values[6,14,34,37,39] with standard deviations for both the low-quality input images and the generated images. Consequently, these studies were included in a meta-analysis. The meta-analysis revealed a mean increase in PSNR between the generated and low-quality input images of 6.60 ± 2.99 (Figure 5). The mean increase in SSIM was 0.28 ± 0.15 (Figure 6). Both increases were statistically significant (p=0.00). However, high heterogeneity was observed in both meta-analyses (I² = 100%), indicating substantial variability among the included studies.

4. Discussion

Point-of-care ultrasound (POCUS) is recognized for its affordability and convenience, but it suffers from a lower image quality compared to conventional high-end, cart-based ultrasound systems. Recent advances in deep learning have achieved state-of-the-art performance in various problems of image processing, including the enhancement of image quality. This systematic review provides a overview of research focused on ultrasound image enhancement using deep learning methods, suitable for real-time POCUS applications. A comprehensive description of the methods used, as well as a further analysis of the performance of the proposed algorithms, is given.

It was observed that the majority of studies utilized GANs incorporating CNNs in both the generator and discriminator networks. The emergence of GANs in the medical imaging field, as described by Liu et al. [23], is noteworthy as these models are capable of generating highly realistic medical images, effectively bridging the gap between supervised learning and image generation. Despite this trend, there was considerable variation in GAN and CNN architectures, loss functions, and evaluation methods across studies. Some studies compared their proposed network with existing networks to benchmark quality enhancement, while others compared the generated images to the original low-quality input images and/or paired high-quality reference images. For the methods that quantitatively assessed the networks’ performance, a variety of image quality metrics was reported. In addition to metric-based assessments, some studies incorporated visual assessments or tested the effect of quality enhancement on downstream tasks such as segmentation or diagnosis. Often, a combination of these evaluation methods was utilized to provide a more comprehensive overview of the proposed algorithm’s performance.

Variability in evaluation methods and performance metrics poses a challenge for direct comparisons among all articles. However, those reporting the most commonly reported performance metrics (PSNR and SSIM) for the low-quality input images or generated images allowed for some comparisons. Figure 3 and Figure 4 reveal substantial disparities in low-quality input image performance, indicating varying baseline qualities across studies. These differences may be explained by heterogeneity in ultrasound devices and dataset types. Nevertheless, consistent improvements in image quality were observed when comparing enhanced images to the original inputs, as shown by Figure 3 and Figure 4. The meta-analysis further supports these findings, showing a statistically significant increase in PSNR and SSIM. This indicates the potential of the proposed deep learning algorithms for enhancing the quality of ultrasound images. However, variability in the extent of performance gain across datasets and articles is notable. This is further supported by the I2 score of both meta-analyses (I2 = 100%), indicating high heterogeneity. This variability complicates the determination of achievable quality gain. Notably, simulated datasets generally exhibited higher performance gains compared to in vivo and phantom datasets, suggesting that simulation results may not fully represent clinical scenarios.

This review focused on ultrasound enhancement for POCUS applications but included studies for ultrasound in general as well to ensure comprehensive coverage of relevant algorithms. A key selection criterion was the simultaneous addressing of multiple distortion types, which led to the inclusion of 15 articles. Interestingly, despite the importance of computation time for real-time applications, most articles did not report this aspect. Furthermore, the lack of source code availability hinders the reproducibility of the conducted research. Studies focusing specifically on enhancing POCUS images commonly paired low-quality POCUS images with high-quality images from high-end machines. Although these image pairs are expected to contain the same locational information, they often suffer from locational differences due to acquisition challenges, which can only be partially mitigated by registration methods and consequently impact network training. In contrast, studies using Plane Wave Imaging (PWI) did not encounter this issue as they used the same device with different numbers of angles, resulting in nearly identical locational information. Future research could benefit from developing more accurately paired datasets, particularly using ex vivo data, to improve image-to-image translation techniques for POCUS.

Several limitations of this review should be noted. First, the selection of articles that addressed multiple quality enhancement aspects might have excluded relevant studies focusing on single aspects. Second, the heterogeneous nature of the included studies, with varying datasets and ultrasound devices, complicates direct and fair comparisons. Although we attempted to group datasets into in vivo, phantom, and simulation categories, diversity remained within these subgroups. Lastly, performing a meta-analysis for machine learning-based research presents unique challenges, as this method was originally designed for comparing cases and controls in medical treatments. Aspects such as the number of images in "case" and "control" groups, use of cross-validation, and dataset similarities due to augmentation were not consistently accounted for. Therefore, the meta-analysis should be seen primarily as an illustrative tool, and caution is needed when drawing firm conclusions about the precise effects of ultrasound image enhancement in terms of expected PSNR and SSIM gain.

5. Conclusions

This review thoroughly examined the progress in ultrasound image quality enhancement using deep learning, with a focus on applications suitable for POCUS. Ultrasound image enhancement through deep learning is a vibrant research field. However, the majority of performed studies focus on single aspects of quality enhancement, which is less effective for POCUS that suffers from multiple distortion types. Studies addressing multiple quality aspects demonstrate the potential for substantial image quality improvements across various ultrasound devices. PSNR values for low-quality input images ranged from 8.65 to 29.24, improving to 13.99 to 36.59 for the enhanced images. Similarly, SSIM values ranged from 0.03 to 0.71 and 0.30 to 0.93 for the low-quality input images and the enhanced images, respectively. However, quantifying the expected performance gain precisely remains challenging due to the heterogeneous nature of the studies. It is important to note that studies often neglect to report computation times, a factor crucial for enabling real-time applications. Future research should prioritize the development of standardized evaluation metrics, report computational efficiency, and ensure reproducibility by sharing source code. Additionally, creating accurate paired datasets with POCUS and high-end US images is essential for advancing this field and achieving reliable real-time image enhancement.