MFNR: Multi-Frame Method for Complete Speckle Noise Removal in Ultrasound images using KDE

Speckle noise is one of the most difficult noises to remove especially in medical applications. It is a nuisance in ultrasound imaging systems which is used in about half of all medical screening systems. Thus, noise removal is an important step in these systems, thereby creating reliable, automated, and potentially low cost systems. Herein, a generalized approach MFNR (Multi-Frame Noise Removal) is used, which is a complete Noise Removal system using KDE (Kernal Density Estimation). Any given type of noise can be removed if its probability density function (PDF) is known. Herein, we extracted the PDF parameters using KDE. Noise removal and detail preservation are not contrary to each other as the case in single-frame noise removal methods. Our results showed practically complete noise removal using MFNR algorithm compared to standard noise removal tools. The Peak Signal to Noise Ratio (PSNR) performance was used as a comparison metric. This paper is an extension to our previous paper where MFNR Algorithm was showed as a general purpose complete noise removal tool for all types of noises


I. INTRODUCTION
U L tra-sound imaging is used in half of all medical screening systems [1]. From imaging of brain, heart, uterus, spleen and kidneys, etc., ultrasound is a very handy and reliable tool. Such wide-spread use is due to the low cost and portability of the ultrasound machines. Frequently the results of the ultrasound images are limited by the noise in the system esp. Speckle Noise. The noise is created through constructive and destructive interference of the returned signals thus creating granularity which show up as bright and dark spots in the image. This paper discusses a newer technique known as MFNR which is a general purpose complete noise removal tool for any noise type [2]. Herein it is applied for the ultrasound systems. Herein, multiple frames are used as opposed to single frame noise removal systems in conventional systems. Multiple frame means imaging at such a high frequency such that we capture multiple images of a single instance that is practically time-invariant.
The results were compared with industry standard algorithms like Median and Lee filters for removing speckle noise. They were compared to our proposed MFNR algorithm for the removal of speckle noise. Median, Lee or other filters should be used if the fit the requirements. MFNR Algorithm should be used if practically complete noise removal is required. [3]. It is Awais   shown that MFNR Algorithm practically completely removes the noise depending on the number of image frames used. More the frames used better the Peak Signal to Noise Ratio (PSNR). The conventional wisdom in image noise removal is that there must be a compromise between noise removal and detail preservation. We showed that in multi frame systems, that is not true. We can achieve both simultaneously [4].
Furthermore, if the images are required to be analyzed by Machine Learning (ML) Algorithms [5], the PSNR should be high enough for better accuracy. Refinement of the images for medical applications requires higher PSNR, which was successfully achieved by using MFNR algorithm. In this study, the algorithm was used to improve the imaging results of an ultrasound imaging system. In short, MFNR algorithm is simple in approach, and is versatile, and automatic thus no need to tune the parameters. Computationally, it is light enough, and it is scalable.

II. RELATED WORK
The standard methods currently used in speckle noise removal are single frame methods. MFNR is inspired from [6] wherein it is employed to remove gaussian noise from the astronomical images. The basic theme is to characterise the PDF of the noise to remove it. Hence, we generalized this system to employ it for ultrasound images where we are characterizing the speckle noise PDF in order to remove it.
Median filter is one of the most effective noise removal tools for the Speckle Noise. It has been shown that it performs better than other noise removal tools for the Speckle noise [7]. The median filter is a nonlinear statistical filter that replaces the current pixel value with the median value of pixels in the neighboring region [8]. The median value of the surrounding pixes is taken. We achieve it through sliding window.
Median filter has its own set of problems namely computational resource requirements [9]. Median filter works by finding the median values of the surrounding pixels and replacing the pixel by that median value. It requires sorting which is computationally expensive. There can be optimizers which speedup the process, however it has reached the plateau. One approximation is by using the sliding window such that it reuses the previous median values to save on computation cost. Also, that median filter doesn't completely removes the noise.
Various more popular techniques are available in literature for filtering speckle noise [10]. Spatial Averaging is a linear filtering technique and it has a blurring effect. The Adaptive filtering techniques like local statistics or spectral coefficients preserves the edges and details.
There are also non-linear filters in the literature [11]. They are based on mathematical modelling and are thus good at preserving the over-all shape but they are not good at preserving the details. They use operations such as open, close, top or bottom hat, etc. One such algorithm is called Morphological Image Cleaning which compares the original and smoothed image hence extracting features from original image and reintroducing them into the smoothed image to preserve the details.
We will compare MFNR algorithm with industry standard algorithms. We will show that where other algorithms are limited by single frame and hence can't completely remove the noise unlike our multiple frame approach.

III. PROPOSED METHOD
Practically complete noise removal is achievable when the noise PDF is determined and removed. Herein, multiple frame methods excel over single frame noise removal methods [12]. This is true for all types of noise, be it gaussian as in cameras, or speckle noise as in ultra sound or laser imaging, etc.
MFNR Algorithm can achieve zero noise, depending on the number of frames used. It is simple to implement. All it needs is the type of noise involved, and it removes it. Niche applications of the multiple frame noise removal methods are used [12] especially in astronomical images where it is used to remove gaussian noise to detect far off celestial objects which maybe a single pixel on the images [6]. In this paper, that multiple frame technique for the gaussian noise is generalized, and used for removal of speckle noise from ultrasound images.
Multiple frames of the images are taken at the same position and resolution. images are herein called frames. The same position in all the frames is taken and used to calculate the value of the single pixel for that same position. Thereby, the PDF noise characteristics and features of that single pixel is calculated and removed. Generalized MFNR Algorithm is depicted in Alg. Output the mean calculated through KDE Algorithm, which is the percentile of the mean value. Its the cleaned pixel value of that position; end Store that clean data against the same position in a cleaned output frame end Result: Output clean frame Algorithm 1: Generalized MFNR Algorithm MFNR Algorithm is a General Algorithm for all types of noise. Be it logarithmic, exponential, poisson, etc. Each noise has its unique signature and characteristic which can be calculated from the same position in multiple frames. Herein, we will estimate the noise parameters of the speckle noise from the multiple frames, and hence remove it.
Centered noise has zero mean. Thus when centered noise is added to any data or image, sometimes it adds value and sometimes it subtracts value but on the average it doesn't add any value and has a zero mean. Example is Gaussian Noise. Positive Noise example is salt noise, because it always adds value to the data, never reduces its value. Negative noise can be pepper noise, because it always subtracts value from the original value and is always negative. But the majority of the noise types fall into the category of mixed noise, e.g. Gamma, Rayleigh, Poisson, Exponential Noise. Mixed noise has some value on the positive side, and some on the negative side, hence the percentage needs to be calculated, which is calculated through KDE [14].
We take multiple frames or images, estimate the noise-less value of the pixel by removing the deterministic noise, and hence the image is cleaned. The multiple images are combined to make a 3D stack as shown in Fig. 3. The depth values or "depth vector" are stored into a newer tensor array, fed into MFNR Algorithm to find mean, min, max, or KDE mean value for centered, positive, negative or mixed noise types respectively.

IV. EXPERIMENTAL SETUP
The noise values were simulated on Python using Jupyter and Anaconda. Windows x64 system was used.

V. PERFORMANCE EVALUATION A. Performance Measurements
In image processing, PSNR is the metric used to compare the performance of various image noise removal algorithms or filters. Definition of PSNR is used as in Eqn. 1. PSNR = 10 log 10 (1) Another parameter that was used to compare the performance is the Optical Flow Error. Optical Flow between two frames tries to calculate how much the features in the image moved between successive frames. Thus, we are considering a sequence of images (I(t) {t=1···T } ) taken from the motion between frames. Pixels get translated between the frames. We are trying to find out the path of each pixel x 0 (t) = (x 0 (t), y 0 (t)) from its start position x 0 . It can be mathematically formulated as in Eqn. 2 Its derivative begets Eqn. 3 where u = (u x , u y ) = dx0 dt = dx 0 is the motion of the pixel. Let it be noted that Eqn. 3 is insufficient to find u (our target).
This becomes an optimization problem and we need to find Eqn. 4 (Here g x0 is a Gaussian filter centered at x 0 ). The minimum is achieved when the derivative of E vanishes : where, I x , I y and I t represent respectively the derivative of the image sequence relatively to x, y and t. This linear system can be expressed as follows: where,

B. Performance Analysis
Speckle noise is a nuisance in almost all applications, related to RADAR and ultrasound. The latter is used in about half of all medical diagnosis [1] and thus noise removal is an important step towards automatic disease diagnosis. MFNR requires multiple frames or images. More the frames, better the results. Hence, practically complete noise removal can be achieved depending on the number of frames used. In addition, other noise removal algorithms or filters require difficult conditions like values of unknown parameters which is at best a guesstimate. For example, whether to use 3x3 or 9x9 window in Median Filter, or the level of noise must be known before hand to estimate the clean image in numerous other speckle noise removal filters [15]. MFNR Algorithm is able to outperform the standard noise removal tools due to its inherent multiple frame technology.
Multiple images can be created through taking multiple images or video, given the time instant between the multiple frames is practically zero [16]. MFNR's computational and storage requirements are minimal compared to other competing algorithms. Real-time applications can be made out of it. More accuracy can be achieved using more frames, and hence there is a trade-off between the accuracy and computational processing required.
Generalized MFNR Algorithm can remove most kinds of noise including gaussian noise in cameras, speckle noise in laser and ultrasound images, gamma noise in radar applications, or any other application. Thus, this Algorithm can be successfully applied in the fields of medicine, astronomy, radar with better performance than existing image noise removal tools [17]. For example, in cameras, the Gaussian Noise become especially a nuisance in low light condition, where even standard MFNR Algorithms fail.
In the case of ultrasound images, wherein the images are fed into machine or deep learning algorithms to predict the disease outcome, there is a need for complete noise removal for better precision, recall, and hence accuracy.
In the ultrasound application, MFNR (Speckled) algorithm performed better than even the best algorithms used to remove this noise in current practice. This is very important result because speckled noise is one of the most difficult noises to remove especially in medical applications [18]. MFNR is a versatile algorithm and can be extended to include removing periodic noise also because the pattern has a "deterministic characteristic" and hence can be removed.
The PDF for a mixed noise type is estimated through Kernal Density Estimation (KDE). It is a method based on instanciation. The idea is to compute the average of the density of given points under the training data. The estimate is given as in Eqn. 10f herein h is bandwidth parameter. The kernel is a Gaussian parameter given as in Eqn. 11, KDE maybe written as a linear system as in Eqn. 12, where Thus, w can be made to become any of the convex combination. Also, any changes to K can also have interesting results.
An estimate of log-likelihood of any given sample is depicted in Eqn. Eqn. 14 And its gradiant is given as in Eqn. 15, thus, it moves x i pointing to the center y j accoring to weight proportional to how x i already is to y j .
The differentiate of the log-likelihood wrt to the centers y j is as shown in Eqn. 16 hence it moves y j to the points x i proportional to how close x i already is to y j .
The differentiable of the resubstitution entropy estimate is as shown in Eqn. 17 where x; and ; x are the first and second arguments of L, respectively. The differentiable of the estimate of the KL divergence is given as in Eqn. 18, The histogram density with bin width h is as shown in Eqn. 19 f (x) = 1 nh (number of x i in same bin as x).
An origin x 0 also has to be chosen for the bins [x 0 +mh, x 0 + (m + 1)h). KDE maybe taken as an inner product where the mean of the training data is kernal space. Similarly, the kernal trick shows that the KDE can be interpreted as a dot product with the mean of the training data in kernel space. Indeed, the kernal trick shown as in Eqn. 20 Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 9 June 2020 doi:10.20944/preprints202006.0117.v1 Herein, we applied KDE on the speckle noise distribution [6].

C. Performance Results
Typical speckle noise is modelled as J = I + n*I, where J is the noisy image, I is the original image, and n is a uniform random variable with 0 mean, and some variance. Fig. 4 (a) shows the performance of MFNR Algorithm against other popular algorithms used to remove speckle noise, where speckle noise was simulated with variance 0.01. We can see that MNFR Algorithm out-performs the standard conventional algorithms used to remove Speckle Noise. After more than 3 frames, MFNR performance is more than either Median or Lee Filters, and it get better as more frames are used. Similarly, 4 (b) shows Optical Flow Error percentage also reduces after just two frames. 4 (c) shows the simulated speckle noise. Fig. 5 shows the results on an actual ultrasound image after successive number of frames.

VI. CONCLUSION & FUTURE WORKS
It was shown that our proposed MFNR algorithm performed much better at noise removal than standard conventional noise removal tools. Their performance was compared at the removal of speckle noise. The algorithm can also be applied to remove the Gaussian, Gamma, periodic noise or other type of noises with defined PDF. In the ultrasound application, it was shown that MFNR algorithm had a significantly better performance than conventional standard algorithms, given that this noise is one of the most difficult noises to remove.
We are using KDE to estimate the noise to remove it hence we need to specify the noise type and also that we require multiple frames. It is proposed that using deep learning, we can train the neural networks to detect that automatically and do away with these requirements largely.