Heart Diseases Diagnose via Artificial Intelligence-Powered Mobile Application

One of the oldest and common methods of diagnosing heart abnormalities is auscultation. Even for experienced medical doctors, it is not an easy task to detect abnormal patterns in the heart sounds. Most of the digital stethoscopes are now capable of recording and transferring the heart sounds. Moreover, it is proven that auscultation records can be classified as healthy or unhealthy via artificial intelligence techniques. In this work, an artificial intelligence-powered mobile application that works in a connectionless fashion is presented. According to the clinical experiments, the mobile application can detect heart abnormalities with approximately 92% accuracy which is comparable if not better than humans since only a small number of well-trained cardiologists can analyze auscultation records better than artificial intelligence. Using the diagnostic ability of artificial intelligence in a mobile application would change the classical way of auscultation for heart disease diagnosis.


Introduction
Heart disease is one of the most common causes of death worldwide [1]. Even in developed countries, healthcare services are expensive, and having a check-up in a clinic is not only a time-consuming and costly task, but also it is relatively a dangerous endeavor under the current conditions of a global pandemic. So, telemedicine services and applications have become popular and gained much importance in recent years [2,3]. There is a consensus that telemonitoring capabilities which include the home assessment of various health parameters such as blood pressure, heart condition, weight, etc. should be enhanced and put at disposal of patients [4]. Automatic diagnose of heart diseases have been studied for a long time and the proposed algorithms have acceptable success on the largest available dataset, the PhysioNet/CinC Challenge 2016 dataset [5,6]. Currently, few products can both record heart sound and analyze it in a cloud-based environment [7][8][9][10].
In this work, a mobile application that is powered by a machine learning algorithm is presented. The algorithm was trained and validated with PhysioNet/CinC Challenge 2016 database. The real-life success of the algorithm has been tested at the clinic using the mobile application. The algorithm can detect abnormal heart sounds in the validation set with 93% accuracy, and the mobile software that is powered by the same algorithm can detect abnormal samples with nearly the same sensitivity rate. The algorithm uses an ensemble nearest neighbors classifier, and features extracted from both Mel Frequency Cepstral Coefficients (MFFCs) and statistical properties of the fundamental heart sounds.
The algorithm is used inside mobile software that can be run on mobile devices. Unlike available commercial products [7][8][9], the software works in an embedded fashion and does not need an internet connection to upload the heart sound to the cloud for classification. Every five-seconds, the software analyses the heart sound and prompts the predicted diagnose to the screen. The software presented in this work is intended to be used both in-clinic settings by medical doctors and by ordinary people at home. So, the software needs to be verified in real-life circumstances. For this, the software was installed on an Android device and the mobile application was tested at Ankara City Hospital, Cardiology Department by expert medical doctors. It is approved by the medical doctors that artificial intelligence-powered mobile application can distinguish healthy and unhealthy individuals with 93% accuracy.

Data Set
For training and validation of the algorithm, the PhysioNet 2016 database which consists of 2430 labeled heart sounds is used. This dataset is a combination of nine heart sound databases collected by independent research groups at different times and with different devices. So, the quality of the sounds, recording position, and patient profile is different. All of the recordings are labeled as normal and abnormal. Since this database consists of nearly all of the available databases, it is widely used and accepted as the reference set for heart sound classification tasks [11][12][13][14]. During the verification tests at Ankara City Hospital, the mobile application was used for 162 individuals. This verification data set consists of 88 normal and 74 abnormal heart sound samples. All participants were between 15-70 years old and ThinkLabs One digital stethoscope was used by focusing multi recording positions on the chest near the heart.

Software Description
Mobile software takes the stream data from a digital stethoscope. For every five seconds, the software extracts some features, then predicts and displays the result concurrently. Since the mobile application is designed to work in a connectionless fashion, all computations are made on-device. The workflow of the software is presented in Figure 1 Figure 1. Workflow of the software; first, data is transferred from digital stethoscope to mobile device then, classification algorithm predicts the diagnose, and the result is displayed simultaneously on the screen.
Software is developed in MATLAB by using the Simulink support package for mobile devices [15][16][17][18]. The support package has all the necessary blocks for using mobile devices' sensors. Moreover, it is possible to use MATLAB functions and the trained algorithm within the projects' block diagram. For using the trained algorithm, it should be converted to a C file beforehand, then it could be called back in the project. Generating C/C++ code from the trained algorithm can be done both by using the MATLAB Coder or by handcrafting the code [19]. Regarding the Simulink model presented in Figure 2; First, an audio capture block is used for getting the audio signal as input. This block enables to get a stereo audio stream from the microphone. In the Bubble_1, there is a button for recording the stereo signal. If this button is switched on, recording time is displayed on the screen and the file write block writes the audio data to a .wav type file on the mobile device. For further steps, stereo data is converted to a mono signal by a MATLAB function file. The converted signal is displayed as a graph in the Bubble_2 bubble.
Secondly, a buffer is used for accumulating the stream data. This buffer collects 220.500 samples from the stream source, the digital stethoscope in our case that has a sampling frequency of 44.100 samples per second. Then, the output of the buffer, a five seconds-long signal is transferred to the feature extraction block. Most of the features are acquired from Mel Frequency Cepstral Coefficients which is a commonly used technique in speech recognition tasks [20,21]. This technique mimics the human ears' ability to scale the sounds by using a logarithmic scale function and a filter. The rest of the features are extracted from both statistical properties and power spectrum of the signal. Those features are respectively; standard deviation, kurtosis, spectral entropy, and maximum frequency. All extracted features are displayed on the screen (Bubble_3).
Third, extracted features are transferred to the classification bubble as input data for the classification algorithm. The classification algorithm predicts the label of five seconds long heart sound data whether it is a healthy or unhealthy sample. To determine the best classification algorithm, well-known methods were evaluated by using the PhysioNet database. During these experiments, for training and validation tasks PhysioNet data was divided into two parts. And, recordings used for training were not used for validation. Accuracy performance of the experimented algorithms is presented in Table 1 and detailed classification metrics of the most successful algorithm is presented in Figure 3. Among several algorithms, the most successful one is an ensemble model of which its learner type is nearest neighbors, number of learners is 30, and the subspace dimension is eighth. This model is saved as the classification algorithm and it is called inside the predict_Class function block. The output of this block is a binary integer where the zero output value stands for healthy samples and the one output value stands for unhealthy samples.  The function of the output bubble in Figure 2. is respectively; to show the predicted diagnose result on the screen as an integer value and to show the corresponding image of the result on the screen. The screen design of the mobile application is seen in Figure 4. On top of the right corner of the screen, the predicted diagnose is shown by a binary value, and a corresponding diagnose image is prompted on the left half of the screen for describing the algorithms' prediction by an image. When the predicted diagnose image is slid to left, this part starts to show real-time phonocardiogram graphs. Below the predicted diagnosis display on the top right corner of the screen, recording time in seconds is displayed. Recording time is displayed according to the outputs of a counter function that is triggered by the boolean value of the start recording button located below the time display on the screen. When this button is switch on, the heart sound is also saved on the device's memory. On the right bottom of the screen, the extracted features are displayed every five seconds.

Results
As stated in the previous part, the classification algorithm has been trained and validated with the publicly available PhysioNet 2016 database. For verification purposes, we have used the mobile application in real-life circumstances at Ankara City Hospital. The cardiac experts tested the algorithm and mobile application over 162 individuals who visited the clinic for regular check-ups. All participants are labeled both by the cardiologists and by the mobile application. According to cardiologists, 88 people were detected as healthy and 74 people were detected as unhealthy. Prospective unhealthy individuals were forwarded to further diagnostic tests and it was confirmed that all unhealthy diagnosed individuals have heart abnormalities. So, the auscultation diagnoses of cardiologists are accepted as the ground truth data. Detailed information about the Ankara City Hospital recordings is presented in Table 2. The recordings used in the verification phase are collected by the developed mobile application. Those recordings are of vital importance since they represent the data quality expected in clinical practice. Verification results that are presented in Table 3. show that the mobile application can detect normal heart sounds above 92% specificity and 81% sensitivity which is very close to validation results in Table 1. This result shows the robustness of the developed classification algorithm since it can work properly on a mobile device. In verification tests, an Android tablet was used and the prediction result of the application is determined by the memory boosted diagnosis display ( Figure 5). The memory display is prompted by a function that accumulates the last five diagnoses in a buffer and tries to catch the four or more same results. Within 25 seconds, the application generates five diagnostic outputs, if four or five consecutive diagnoses are received, the memorized decision of the algorithm is determined and this result is accepted as the final decision of the application.

Discussion
Medical doctors without special auscultation training cannot perform well in detecting murmur. The accuracy of the pediatricians in detecting heart murmurs is 24% [22]. Another study states that general practitioners can diagnose significant valvular heart diseases with a sensitivity of 43% and specificity of 69% but cardiologists can predict up to a sensitivity of 81% [23]. Even though some trained cardiologists can perform well, clinicians have variable auscultation performance [24,25]. So, both cardiologists and non-cardiologist need a preliminary diagnostic tool to boost their auscultation accuracy. Similarly, non-trained clinicians have a low accuracy rate with electrocardiogram data and only cardiologists can diagnose successfully [26,27]. Moreover, electrocardiogram based artificial intelligence models such as KardiaMobile and Eko AI software can perform as well as cardiologists [28]. Eko AI software requires users to upload their data to the cloud for computation. Unlike these commercial products, the developed application can run solely on the mobile device's hardware thanks to its informative features and robust classification algorithm. Regardless of having an internet connection, one can evaluate his or her heart health condition instantly with a sound recorder preferably with a regular digital stethoscope. As a result, users do not need to upload the recordings to a server or a cloud for evaluation. This feature provides data privacy and flexibility for users. In the near future, with the proliferation of AI-powered applications for telemedicine purposes, data privacy issues will be more important.
Another key feature of the application is it accepts the data from the start and it perpetually outputs the predicted diagnose result in five-second periods. For future works, this property can be adopted to intensive-care units to trigger an alarm when an abnormality is detected. Cardiac event monitors control electrocardiogram signals to decide whether patients' heart continues its function or not. Those patient monitoring systems do not provide any information about instant heart abnormalities. If this process can be converted to listening to the heart and continuously analyzing its condition, the effectiveness of patient monitoring systems will increase.
Given that we have a global pandemic, the medical system is under pressure, and clinicians are overwhelmed by the increasing number of patients caused by COVID19. Telemedicine tools like the one presented in this study can reduce the number of healthy individuals visiting hospitals and as a result, take the burden off the medical system. Since the mobile application is designed to be used not only for experts but also by users of no medical education, verification data should consist of sounds that are recorded by ordinary users. To mimic the behavior of users with no medical background, during the verification tests, medical staff pointed digital stethoscope at random locations on the chest near the heart. The verification results are successful but this is partly due to the digital stethoscope's filter property in device level that can eliminate unwanted sounds produced from other organs such as lungs etc. To compensate for the noise effect, an additional preprocessing block could be added in future works.

Conclusions
Without proper training, most clinicians fail to distinguish abnormal heart sounds from normal ones. But, auscultation is one of the oldest, easiest, and main methods used for detecting heart diseases. To not miss a diagnose, heart sounds should be listened to by artificial intelligence too. For this reason, we present a mobile application design that is developed in MATLAB. This application and its classification algorithm can work solely inside the device hardware and produces quick responses. In the verification phase, it is confirmed that the developed mobile application can work properly on mobile devices. As a result, the presented mobile application is proved to be a valuable diagnostic tool not only for clinicians but also for ordinary users.
Supplementary Materials: Android package files, codes, feature set matrices are available online at https://github.com/mesuttguven/CardiacAI.git, and validation data set is available online at https://archive.physionet.org/pn3/challenge/2016/. Android apk is available at the repository, everybody is encouraged to download, install, and test the mobile application to check the results presented here. Informed Consent Statement: Patient consent was waived due to routine auscultation was conducted on all patients, no invasive methods were used and heart recordings of the participants were not saved for not to infringe on their privacy. Verbal consent of all participants had taken beforehand for using the digital stethoscope with the AI-powered mobile application to compare diagnosis results of clinicians and algorithm.