MFNR: Multi-Frame Method for Complete Noise Removal of all PDF Types in Multi-Dimensional Data using KDE

In research applications across several areas, noise removal is indispensable for accuracy of final results. Noise is caused due to physical principals, such as background electronic noise, quantum effect, and wave rebound effect to name a few. Noise removal can help improve results in medical, astronomy, defense, and numerous other fields. Addressing this limitation would result in potentially low cost, automatic, and reliable systems. In this paper, a generalized new approach i.e. Multi-Frame Noise Removal (MFNR) is proposed for noise removal. Given any type of data, the probability density function (PDF) of the noise can be determined. Herein, we extracted the noise PDF parameters using KDE (Kernel Density Estimation). Because the data is corrupted by “deterministic” noise, hence can be cleaned. This could be used as a general purpose noise removal tool. The data point with same position in multiple frames helps us determine the noise PDF characteristics and hence making it possible to remove noise. The conventional wisdom which states that noise removal and detail preservation are contrary to each other is not true for MFNR. Experimental results validate our proposed method which showed practically complete noise reduction based on number of frames used, as compared to existing benchmark methods.


I. INTRODUCTION
I N any data processing application, the accuracy of the final results depend on the noise-less data. Thus, normally data is pre-treated with a host of filters to reduce the noise before further processing. But, conventionally a limit is reached where a compromise has to be made between noise removal and compromising the data details [1]. Data corruption by noise is depicted in Fig. 1.
In this study, performance of industry standards for noise removal like Gaussian, NLM (Non-local mean), Median, Wiener, and proposed MFNR algorithm were compared. Simple filers like Gaussian or Median filters should be used if they fit the bill [6]. Other complex filters like NLM, or Weiner perform worse than these or only marginally better, notwithstanding their complexity and resource consumption. MFNR algorithm significantly performed better than conventional standard.
Our results showed significant noise reduction using MFNR algorithm compared to standard Gaussian and similar filters.  The MFNR algorithm was simple in approach, much more powerful, with no need to tune the parameters, while light enough on resources, and was scalable to any desired error level. Though MFNR algorithm was compared with standard image noise removal in three dimensional RGB images, but it can be extended to multi-dimensional data of all PDF types or even periodic type of data. For example, it can be used for noise removal in audio data.

II. RELATED WORK
Presently, single frame noise removal method is standard and industry. Multi-frame Noise Removal is only a niche product [7]. The idea for MFNR is inspired from [7] which uses it to remove Gaussian Noise. Herein in this paper, the idea is generalized to include non-gaussian noise of all types, and of all dimensions. In this paper, we will be comparing several standard algorithms against MFNR including but not limited to NLM (Non-local Means), Median, Convolution Neural Networks.
NLM is one of the most effective noise removal tools for the Gaussian Noise. It has been shown that it performs better than other noise removal tools for the Gaussian noise [12]. Unlike other local-mean filters, which only take the surrounding pixels of the targeted pixel, NLM takes into account all the pixels thus having greater information at hand. This results in much more preservation of details [12].
But NLM still leaves "method noise" in the process, which is more like "white noise" [13]. Our proposed MFNR Algorithm doesn't leave such noise because it is only estimating the same pixel and doesn't take into account the surrounding pixels. MFNR algorithm can be extended to other image and data manipulation techniques wherever NLM is used to enhance data cleaning [14] and various data processes e.g. interpolation [14] and depth maps regularization [15].
MFNR is compared against Median Filter [16], another popular de-noising filter. Median Filter is especially helpful at preserving edges, which in turn is helpful at edge-detection. Such pre-processing helps improve the results of the systems. Median filters are also very useful in signal processing.
Median filter works by taking the median value of the neighbouring data or pixels. Thus, we are essentially taking the "window" which we slide in each step to ensure all of the pixels are taken care of. It works on multi-dimensional data. For 1-D, it simply works by taking into consideration the surrounding two values. Similar goes for more dimensional data.
Finally, MFNR is compared against Convolutional Autoencoders for Image Noise removal [17] Autoencoders have widely used for image noise reduction. Convolutional Networks are the best known way to deal with images, frequently called CNN -Convolutional Neural Networks -or ConvNet. MFNR was found to be better in terms of Peak Signal to Noise Ratio (PSNR) than CNN.

III. PROPOSED METHOD
When an image is taken from a camera or any imaging sensor, noise is added to it, but because it is corrupted by "deterministic" noise, hence can be cleaned. It can be done only if the PDF (probability density function) of the noise is available, thus the information can be extracted against zero noise [6]. MFNR algorithm needs to know the type of noise involved to estimate and subsequently remove it. Its generalized form is depicted in Algorithm 1 Data: Collect multiple frames of the same data while not all points of the data are processed do Load the similar position of data in each frame into an input array; Apply MFNR Algorithm according to noise type, with that input array to predict the output; Store that output for the same position data end Result: Output the clean data Algorithm 1: MFNR Generalized Algorithm Fig. 2 shows the operational flow of MFNR algorithm as a specialized case for 3D RGB images. It takes multiple images, estimates the true value of the pixel given the noise type, and outputs the clean image. The algorithm is provided with the noise PDF type, and the number of images taken. The images are combined to make a 3D stack. X and Y axes denotes the length and breadth of images, while Z denotes the number of images or depth of the image stack. Pixels are traversed in all possible X,Y pairs, and for each pixel "depth vector" is defined containing the depth values. The "depth vector" estimates the noise, removes it for each depth vector, and corrects the corresponding pixel values. This provides a noisefree corrected image. There are 4 PDF types on the large as shown in Fig. 3, i.e. Centered, Positive, Negative and Mixed noise PDF. Centered noise PDF has zero mean, e.g. Gaussian noise, salt & pepper noise and speckle noise. Positive noise PDF has all its values as positive, e.g. Gamma, Rayleigh, Poisson, exponential noise PDF's. Negative noise PDF has all its values as negative, e.g. a uniform distribution with only negative values. Mixed noise PDF has some of the noise values lie on negative, and some on the positive side. Thus, its percentile needs to be obtained. We obtained it using the KDE algorithm. KDE can be applied in all four cases, but it is only really required in the mixed noise type. Types other than mixed noise can be cleaned simply by taking mean, max, min values. In cases where computational power is limited, mixed noise can be approximated with mean and we can still get pretty good results within required accuracy. For example, centered noise PDF is the special case of mixed PDF with 50% percentile. When mixed noise is added to the image, its minimum and maximum sample values need to be found. The percentile is the value corresponding to zero noise relative to the minimum and the maximum value. This value is found in a monte-carlo fashion.
How the noise is removed is shown in Fig. 4. Multiple frames of same image are taken at same resolution and in same position. They are aligned together, and given the noise PDF (Probability Density Function), its features are extracted for each pixel position through machine learning techniques for PDF detection. Multiple-images noise removal techniques have been used widely [6]. and its most common use is in astronomy, where low lightning condition is a limitation [7]. These techniques are mostly for Gaussian noise. The proposed algorithm is a generalized image noise removal tool catering for different noise types. It needs to know the type of noise involved, to estimate and remove it. For example, if the noise is Gaussian distributed, the MFNR algorithm is used to estimate the noise parameters from multiple images, and outputs the Gaussian mean as the corrected value for corresponding pixel values. However, if the noise is logarithmic or exponential, then MFNR algorithm needs to be adapted, accordingly. MFNR algorithm is simple to implement and the PSNR (Peak-Signal-to-Noise Ratio) improves as the number of frames are increased, and thus zero noise can be achieved. PSNR is defined as in Eqn. 1 PSNR = 10 log 10

IV. EXPERIMENTAL SETUP
The results were generated using Jupyter running Python on Windows x64 system. All the noise was simulated using standard libraries or otherwise explicitly mentioned.

A. Performance Measurements
As already discussed, MFNR algorithm is simple to implement and the PSNR (Peak-Signal-to-Noise Ratio) improves as the number of frames are increased, and thus zero noise can be achieved. PSNR is defined as in Eqn. 2 PSNR = 10 log 10 PSNR is the metric we will be using to assess the efficacy of different algorithms. It also helps improve the accuracy of the real world applications that depends on the clean data for better performance.

B. Performance Analysis
Accuracy depends on the ability to acquire low noise images. Thus, captured images are filtered to reduce noise before further processing. Conventionally a limit is reached where a trade-off has to be made in favor of noise removal and thus compromising the fine details of the image. In general, a Gaussian filter is normally regarded as an "optimal" compromise and a default choice. There are other algorithms that may do better than Gaussian filters, but they are usually either too complex or require detailed knowledge of the noise form or structure that is not normally available, and can also require excessive computational power.
Multiple images are required for processing MFNR algorithm [8], and more the images better the noise removal. In short, number of images depends on the accuracy that needs to be achieved. The algorithm has numerous features making it preferable over other noise removal tools. Problem with other image noise removal algorithms is that, their parameters are often unknown most of the time, and frequently need to be estimated. For example, for applying NLM filter, patch size, window size, and noise level has to be given. It performs far better than even "optimal" filters like Gaussian, or Wiener Filter, and other complex filters.
When an image is taken from a camera or any imaging sensor, noise is added to it. The MFNR algorithm is simple to implement and the Peak Signal-to-Noise Ratio (PSNR) of the image improves as the number of frames are increased. [9] Multiple frames of same image are taken at same resolution and position. They are aligned, and given the noise Probability Density Function (PDF), its features are extracted for each pixel position.
Such multiple-images noise removal techniques have been used widely and its most common use is in astronomy, where low lightning condition is a limitation [7]. These techniques are mostly for gaussian noise. The proposed algorithm is a generalized image noise removal tool catering for different noise types.
In particular, it needs to know the type of noise involved, to estimate and then remove it. For example, if the noise is Gaussian distributed, the MFNR algorithm is used to estimate the noise parameters from multiple images, and outputs the gaussian mean as the corrected value for corresponding pixel values. However, if the noise is logarithmic or exponential, then MFNR algorithm uses KDE to find mean, accordingly [2].
There are a few basic PDF types considered here, specifically, Centered, Positive, Negative and Mixed noise PDF [4]. Centered noise PDF has zero mean, e.g. gaussian noise. Positive noise e.g. salt noise, and negative noise e.g. & pepper noise. Negative noise PDF has all its values as negative. Mixed noise examples include speckle noise, Gamma, Rayleigh, Poisson, exponential noise PDF's. Mixed noise PDF has some of the noise values lie on negative, and some on the positive side. Thus, its percentile needs to be obtained using KDE. For example, centered noise PDF is the special case of mixed PDF with 50% percentile. When mixed noise is added to an image, its minimum and maximum sample values need to be found.
The percentile is the value corresponding to zero noise relative to the minimum and the maximum value.
The simplest way to take multiple images is using a quality video camera, and convert the video frames into an image [10]. It is considered that computation and storage requirements are much less than similar noise removal algorithms. It can be done in real time in most modest applications. Generally, the requirements of MFNR algorithm are similar to those of Gaussian or Median filter, but actual time requirement depends on the implementation and optimization. Because it can remove most kinds of noise given the noise PDF, its application list is vast including cameras, ultrasound, PET, and MRI scan, or any other related application [11]. This algorithm can be successfully applied in the fields like medicine, RADAR, astronomy, with performance better than existing noise removal tools. In low light conditions, Gaussian noise is more profound, and MFNR algorithm can be the only solution. For example, cinema experience would be more enjoyable, if the video frames would be using MFNR algorithm to make crisp and sharp images especially during darker conditions. In addition, custom noise can also be removed, while most other filters are limited to fixed noise type.
This algorithm also has its share of limitations, such as being not able to remove hot pixels, and the inherent condition that multiple images are required. Also that, noise type must be known prior to processing.
The algorithm is versatile and the authors opine that this can be applied for removing the frequency domain noise also including the periodic noise where same signal which is periodic can make a PDF of its own and hence the deterministic signal can be extracted from the noise.

C. Performance Results
Performance against the Centered, Positive, Negative, and Mixed Noise removal were compared. Case studies were carried out comparing the performance of various filters including Gaussian, NLM, Median, Wiener, Neural, and MFNR for removing gaussian noise, using python, and the results compared. Similarly, salt and pepper noise and Poisson noise removal were also shown. Results are summarized in Fig.  5 showing the image noise removal comparison between different filters using simulated noise.

1) Centered Noise Removal [Gaussian Noise]:
A sample image was taken and Gaussian noise was added to it. Then the noisy images were pre-processed with a number of different standard filters to remove the Gaussian noise [9]. Previously, Gaussian filter gave the maximum performance, but MFNR algorithm showed far greater PSNR performance. It was estimated that as the number of images were increased, the noise was removed in an exponential manner. The accuracy can be scaled according to requirements. Gaussian Filter is normally regarded as an "optimal" compromise and a default choice . There are other algorithms which may do better than Gaussian Filter, but they are usually either too complex or require lots of computational power.
Gaussian Noise is depicted as in Eqn. 3 with Mean µ, Variance σ 2 .
2) Positive Noise Removal [Salt Noise]: Positive Noise such as a Salt noise [5] was removed and performance of different filters was compared. This result was more dramatic than previous MFNR (Gaussian) case with respect to the number of images used.
MFNR algorithm implies taking the minimum of the 10 samples, to reduce the noise. In this example, the actual value of the pixel was 0, but due to addition of salt noise, it was added in positive direction. After the application of MFNR algorithm, the actual value is restored to original value, so the noise was successfully removed. Eqn 4 depicts Salt Noise.
3) Negative Noise Removal [Pepper Noise]: [5] Similar to Salt noise. Pepper noise is removed by taking the largest value among the multiple frames for each pixel. Pepper noise is depicted in Eqn. 5

4) Mixed Noise Removal [Poisson Noise]:
The fundamental problem of density estimation is that of inferring the density function f , given an i.i.d sample x 1 , x 2 , ..., x n from the corresponding probability distribution. The kernel density estimate (KDE) [2] is a particularly simple instance-based method. The idea is to compute the density of a test point as the average of the density of test point under each point in the training data. The estimate is given as in Eqn. 6 where h is a bandwidth parameter, and The kernel is a Gaussian parameter given as in Eqn. 7, KDE maybe written as a linear system as in Eqn. 8, where Thus, we can see that w makes the convex combination. Any alterations to K will have corresponding results.
Herein, the log-likelihood estimate of any given sample is depicted as in Eqn. 10 And its differential is given as in Eqn. 11, It can be shown that x i is moving towards the center y j proportional to its weightage whereby x i is to y j .
The gradiant of the log-likelihood w.r.t. centers y j is as shown in Eqn. 12 where y j moves to the points x i in step to how close x i already is to y j .
The gradiant of the estimate of the resubstitution entropy is as shown in Eqn. 13 herein x; and ; x are the first and second arguments of L, respectively.
The gradiant of the KL divergence estimate is given as in Eqn. 14, The histogram density with bin width h is as shown in Eqn. 15 An origin x 0 also has to be chosen for the bins [x 0 + mh, x 0 + (m + 1)h).
KDE is the dot product where the mean of the training data is kernal space. The kernal parameter indicates that the KDE is a dot product with the mean of the training data in kernel space. The kernal trick shown as in Eqn. 16 So the implicit training in a KDE is simply taking an average.
Herein, we applied KDE on the Poisson distribution. [3] The shorthand X ∼ Poisson(µ) is the indicator of random variable X and that it has the distribution of the Poisson wherein the positive parameter is µ. The random variable X having scale parameter µ has the probability mass function as showin in Eqn. 17 The number of events can be modelled by the Poisson in an interval associated over the space or time with random variables. For example, the number of potholes over a road, or the number of typographical errors in a magazine, or the frequency of the descent of customers per hour, or the number of earthquakes in a decade. It can also approximate the binomial distribution wherein n is very large and p is very small.

VI. CONCLUSION & FUTURE WORKS
It was shown that our proposed MFNR algorithm performed much better at noise removal than conventional noise removal tools. Their performance was compared at the removal of Gaussian, Salt and Pepper, and Poisson noise as special case of MFNR Algorithm applied to 3D data of colored images. General MFNR Algorithm can be extended to multidimensional data of all types of noises. The PSNR was used as a performance metric. Due to its logarithmic nature, even a single point is worth it, but we were able to achieve significantly beyond that. In fact, MFNR Algorithm can practically remove the noise completely. More frames require more availability of frames and computational power, hence the algorithm can be tailored to individual needs. The algorithm can also be applied to remove the periodic noises. It is proposed that this newer approach should be used to improve upon existing medical, defence, astronomy, and numerous fields where performance needs to be enhanced which is currently limited by noisy data which is difficult to remove.
Currently, we need to specify the type of noise PDF and need multiple frames. It is encouraged to use deep learning or similar techniques to do away with these requirements automatically.