Potential of Machine Learning for Predicting Sleep Disorders: A Comprehensive Analysis of Regression and Classification Models

1. Introduction

Sleep is an important natural activity for humans and plays a very important role in everybody’s health [1]. Our body supports healthy brain functionality and maintains the necessary physical health while sleeping [2]. Moreover, sleeping is very important for body development and growth, especially for children and teens. Sleeping really impacts the way of thinking, working, learning, reacting, and many other aspects of daily life activities. It also affects the circulation, immunity, and respiratory systems of our bodies [3].

On the other side, lack of sleep (sleep disorder) causes several problems and difficulties in daily life [4]. To name a few, sleep disorders increase the levels of hormones that control hunger, increase consumption of sweet, salty, and fatty foods, decrease the levels of physical activity, and increase the risk of obesity, stroke, and heart disease [5]. It may also cause stress, fatigue, and functional weaknesses [6,7]. Also, sleep disorder is one of the main reasons for sleep apnea. According to recent statistics from U. S census data, more than 140 millions (70 millions men, 50 millions women, and 20 millions children) snore because of sleep apena mostly. Globally, around 936 adults suffer from mild to severe sleep apena all over the world. Moreover, According to several global research works, around 10% upto 30% of the world’s poulation suffer from sleep disorder, and in some countries the percentage may reach 60%. Also, sleep disorder is nearly 7% higher among women than among men. Finally, sleep disorder represents a global epedimic that threatens the quality of life and health for around 45% of the world’s populations.

Based on the recent literature of sleep disorder, it can be noted that the following research works dominate this field. Firstly, the relationship between Covid-19 and sleep disorder. Secondly, searching for new tests other than Obstructive Sleep Apnea (OSA) that is less costly and more comfortable to possible patients is an urgent need. Finally, the utilization of machine learning and wearable devices with fewer sensors in sleep disorder diagnosis at home without the need to sleep in specific sleep centers.

Consequently, this research aims to provide added knowledge and contribute to the solution of the sleep disorder problem by utilizing machine learning capabilities in the prediction task of sleep disorders [8]. In specific, this research is more interested in three main objectives:

• To identify the best regression model that highly suits disorder datasets among twenty-three different regression models
• To identify the best classification model that highly suits disorder datasets among twenty-nine different classification models
• To identify the best learning strategy that highly suits disorder datasets among six different well-known strategies

Therefore, this research considers two main machine learning tasks: regression and classification. Both tasks are used to predict unknown values [9]. The difference is that regression is used to predict numeric values, while classification is used to predict non-numeric values [10].

Regarding the classification task, it has been defined as the ability to predict the class label for unseen cases or examples accurately [11,12]. Classification is of two types: Single Label Classification (SLC) and Multi Label Classification (MLC). The former type associates every instance or case with only one class label, while the latter may associate an instance or example with more than one class label [13,14,15].

SLC is also divided into two subtypes: binary classification and multi-class classification [16]. For binary classification, the total number of class labels in the dataset is only two [17,18]. For multi-class classification, the number of class labels in the dataset is more than two. The dataset in this research belongs to the multi-class classification [19,32,33].

Regarding the regression task, it is defined as the task of understanding the relationship between the objective variable (the dependent variable) and the considered variables and features in the dataset (independent variables) [20]. The objective variable in regression must be continuous; hence, it is a main supervised task in machine learning that aims to predict the value of a continuous variable based on a set of known variables [21]. Regression has many real-life applications, such as forecasting house prices [22,34], predicting users' trends [23], and predicting interest rates [24,25], among several other applications.

To achieve the first objective, this research considers twenty-three regression models that belong to four learning strategies. These regression models have been evaluated and compared using two datasets with respect to five well-known evaluation metrics. To achieve the second objective, twenty-nine classification models are being evaluated and compared with respect to five popular metrics in the domain of classification.

The rest of the paper is organized as follows: Section 2 reviews the most recent work related to sleep disorder detection using machine learning techniques. Section 3 describes the research methodology and the considered datasets, and it provides the empirical results, followed by the main findings. Section 4 concludes and suggests a future direction.

2. Related Work

Everyone requires sleep. It is a crucial component of how our bodies work. You may require more or less sleep than others, but doctors advise people to get seven to nine hours per night. Most people face a problem with sleeping called a sleep disorder. Sleep disorders are situations in which the usual sleep pattern or sleep behaviors are disrupted, and the main sleep disorders include insomnia, hypersomnia, obstructive sleep apnea, and parasomnias.

In addition to contributing to other medical concerns, several of these disorders may also be signs of underlying mental health problems, which led researchers to do a lot of research. In [26], authors present a thorough study of the relationship between vitamin D and sleep problems in children and adolescents who suffer from sleep disorders such as insomnia, Obstructive Sleep Apnea (OSA), Restless Leg Syndrome (RLS), and other sleep disorders. The research synthesized information regarding the role and mechanism of action of vitamin D. A review of the use of melatonin and potential processes in the sleep disturbances of Parkinson's disease patients can be found in [27].

In [28], researchers conducted a systematic study and meta-analysis to identify the key elements contributing to sleep and anxiety problems during the COVID-19 pandemic lockdown. Additionally, the study aims to forecast potential correlations and determinants in conjunction with results connected to COVID-19 pandemic-induced stress and difficulties and analyze the various symptoms and complaints that people experience with regard to their sleep patterns. The Pittsburgh Sleep Quality Index (PSQI), machine learning algorithms, and the general assessment of anxiety disorders were used to analyze the outcomes. The study looked at a significant correlation between symptoms such as poor sleep, anxiety, depressive symptoms, and insomnia, as well as the COVID-19 pandemic lockdown.

In [29], a cross-validated model is proposed for classifying sleep quality based on the goal of the act graph data. The final classification model demonstrated acceptable performance metrics and accuracy when it was assessed using two machine learning techniques: Support Vector Machines (SVM) and K-Nearest Neighbors (KNN). The findings of this research can be utilized to cure sleep disorders, create and construct new methods to gauge and monitor the quality of one's sleep, and enhance current technological devices and sensors.

In [30], they proposed a general-purpose sleep monitoring system that may be used to monitor bed exits, assess the danger of developing pressure ulcers, and monitor the impact of medicines on sleep disorders. Additionally, they contrasted a number of supervised learning algorithms to find which was best appropriate in this situation. The experimental findings from comparing the chosen supervised algorithms demonstrated that they can properly infer sleep duration, sleep postures, and routines with a fully unobtrusive method.

In [31], they proposed a reliable approach for classifying different stages of sleep using a sleep standard called AASM based on a single channel of electroencephalogram (EEG) data. The use of statistical features to analyze the sleep characteristics and the three distinct feature combinations utilized to categorize the two-state sleep phases are the main contributions of this work. Both patients with sleep disorders and healthy control subjects participated in three separate trials with three distinct sets of characteristics. As a result, many machine learning classifiers were developed to categorize the various stages of sleep.

3. Materials and Methods

This section represents the core of this research. At first, the datasets are described along with the required preprocessing steps. Then, the evaluation results for the twenty-three considered regression models are provided and discussed. After that, a comparative analysis among twenty-nine classification models (classifiers) is conducted and analyzed. Finally, a discussion regarding the most interesting findings is carried out.

Regarding the experimental design, all classification and regression models have been used with their default settings and parameters except for the IBK algorithm, where the KNN parameter has been changed from 1 to 3. Moreover, the considered models have been implemented using the Python programming language. Also, experiments have been conducted on the Intel i3 core. Finally, to handle the problem of missing values, all missing values have been estimated to be the average of the values within the same class.The Main Phases of Research Methodology are shown in Figure 1.

Datasets and Preprocessing Step

Two datasets are considered in this research. The first one (Dataset 1) consists of 62 cases and 11 features. This dataset is an extended version of the second dataset (Dataset2), where three features have been added and considered. Both datasets suffer from missing values. The main goal of collecting the datasets was to study sleeping patterns in mammals. Another main goal behind collecting this data was to identify the main factors affecting the quality of sleep and to diagnose the main risks regarding sleep disorders. The main features (attributes) in both datasets are: body weight, brain weight, predation index, sleep exposure index, gestation time, and danger index. All these features are numerical. Also, both datasets consist of five class labels. Both datasets are graciously shared on Kaggle and freely available at the following URL: (https://www.kaggle.com/datasets/volkandl/sleep-in-mammals). Table 1 summarizes the main characteristics of the considered datasets, while Table 2 provides more information regarding the features in both datasets.

Originally, both datasets were of type regression. Nevertheless, a mapping has been carried out to convert the objective feature from being a number to a class variable (string). For example, instead of having ‘1’ as a value for the “overall danger index” feature, it has been converted to ‘A’, and instead of having ‘5’ as a value for the “overall danger index” feature, it has been converted to ‘E’.

Figure 2 and Figure 3 depict the correlation matrices for Dataset1 that consists of 10 features (excluding the class feature), and Dataset2 that consists of 7 features (excluding the class feature) respectively.

4. Results

4.1. Identifying the Best Regression Model

Identifying the best regression model is the main objective of this research. To meet this objective, twenty-three regression models have been considered and evaluated. These models belong to five well-known strategies.

The function learning strategies have been represented through four models: Gaussian processes, linear regression, multilayer perception, and SMOreg. Also, three models are used to represent the lazy learning strategy: IBK, KStar, and LWL. For meta-learning strategy, the following eight regression models have been considered: AdditiveRegression, Bagging, RandomCommittee,RandomizableFilteredClassifier, RandomSubSpace, RegressionByDiscretization, Stacking, and vote. The rules learning strategy has been represented using the following models: DecisionTable, M5Rules, and ZeroR. Finally, five models have been used to represent tree learning strategies (DecisionStump, M5P, RandomForest, RandomTree, and REPTree).

It is worth mentioning that all these models have been used with their default settings and parameters, except for the IBK algorithm, where the KNN parameter has been changed from 1 to 3.

The evaluation phase of the considered regression models has been carried out on both datasets (Dataset 1 and Dataset 2) with respect to five different and well-known evaluation metrics such as Correlation Coefficient (CC), Mean Absolute Error (MAR), Root Mean Squared Error (RMSE), Relative Absolute Error (RAE), and Root Relative Squared Error (RRSE). These metrics are computed using the following equations:

$$CC=\frac{\sum (x_i-\overline{x})(y_i-\overline{y})}{\sqrt{\sum {(x_i-\overline{x})}^2}\sum {(y_i-\overline{y})}^2}$$

(1)

$$MAE=\frac{\sum _{i=1}^n\left|y_i-x_i\right|}{n}$$

(2)

$$RMSE=\sqrt{\frac{\sum _{i=1}^n{‖y\left(i\right)-\widehat{y}\left(i\right)‖}^2}{N}}$$

(3)

RAE = mean of the absolute value of the actual forecast errors / mean of the absolute values of the naive model’s forecast errors

(4)

$$\mathit{R}\mathit{P}\mathit{S}\mathit{E}=\sqrt{\frac{\sum _{\mathit{i}=1}^{\mathit{n}}{({\mathit{y}}_{\mathit{i}}-{\widehat{\mathit{y}}}_{\mathit{i}})}^2}{\sum _{\mathit{i}=1}^{\mathit{n}}{({\mathit{y}}_{\mathit{i}}-{\overline{\mathit{y}}}_{\mathit{i}})}^2}}$$

(5)

Table 3 depicts the evaluation results for CC metrics on both datasets using twenty-three regression models.

According to Table 1 and considering Dataset 1, several models achieved strong results, such as GaussianProcesses, MultilayerPerceptron, SMOreg, IBK, RegressionByDiscretization, and RandomForest. The best regression model, according to the table, is the MultilayerPerceptron regression model, which belongs to the function learning strategy. Moreover, the second-best model belongs to the function strategy, which is SMOreg.For Dataset 2, both MultilayerPerceptron and SMOreg achieved the best results among the twenty-three considered regression models.

Table 4 represents the MAE results for the twenty-three regression models on both datasets. According to Table 4 and considering Dataset 1, RegressionByDiscretization which belongs to the meta-learning strategy, achieved the best (lowest) results compared with the other twenty-two regression models. Also, MultilayerPerceptron achieved the second-best value. It is worth mentioning that MAE itself is not sufficient to assess the regression models. Therefore, this research considers other evaluation metrics. For dataset 2, SMOreg achieved the best results, followed by the KStar algorithm. Both models belong to lazy learning strategy.

Table 5 shows the results for the RMSE metric on both datasets using the same twenty-three regression models. For the RMSE metric, the lowest the value, the better the performance. From Table 5, and considering Dataset 1, MultilayerPerceptron and SMOreg from the function learning strategy achieved the best two results, respectively. Moreover, RegressionByDiscretization, GaussianProcesses, and IBK achieved acceptable results compared with the other regression models considered in this research. For Dataset 2, the IBK and KStar models achieved the best two results, respectively.

Table 6 depicts the empirical results for the RAE metric, considering twenty-three regression models and two datasets. For the RAE metric, the lowest the value, the best the predictive performance. According to Table 6, and considering Dataset 1, MultilayerPerceptron and SMOreg achieved the best two results, respectively. Both regression models belong to the function learning strategy. The third regression model is IBK, which belongs to the lazy learning strategy. For Dataset 2, SMOreg achieved the best RAE result, followed by the KStar model.

Table 7 represents the RRSE evaluation results for the twenty-three considered regression models on both datasets. For this metric, the lower the value, the better the predictive performance. Considering Dataset 1, and according to Table 7, MultilayerPerceptron and SMOreg are the best two regression models, respectively. RegressionByDiscretization regression model from the meta-learning strategy achieved the best third results on dataset 1.

Considering Dataset 2, KStar from the lazy learning strategy achieved the best RRSE result, followed by SMOreg from the function learning strategy.

Table 8 summarizes the previous tables in order to identify the best regression model among the twenty-three considered models. For Table 8, MLP is short for MultilayerPerceptron and RBD is short for RegressionByDiscretization.

According to Table 8, the MLP model achieved the best results on Dataset 1, while SMOreg achieved the second-best results on the same dataset. For dataset 2, SMOreg achieved the best results, followed by the KStar model. Hence, it can be concluded that ensemble learning is the best way to handle the prediction task for sleeping disorder datasets with respect to utilizing the following models: MLP, SMOreg, and KStar.

4.2. Identifying the Best Classification Model

This section aims to identify the best classification algorithm to use with the problem of sleep disorders. The evaluation phase in this section considers twenty-nine classification models that belong to six learning strategies.

These classification models are BayesNet, NaiveBayes, NaiveBayesUpdateable from Bayes learning strategy, Logistic, MultilayerPerceptron, SimpleLogistic, and SMO from Functions learning strategy. IBK, KStar, and LWL from the lazy learning strategy Bagging, ClassificationViaRegression, FilteredClassifier, LogitBoost, MultiClassClassifier, RandomCommittee, RandomizableFilteredClassifier, RandomSubSpace, and Vote from Meta Learning Strategy DecisionTable, JRip, OneR, PART, and ZeroR are from the rules learning strategy. J48, LMT, RandomTree, RandomForest, and REPTree from the Trees learning strategy

Moreover, the evaluation phase for this section considers five different and well-known metrics. These metrics are accuracy, precision, recall, F1-measure, and Matthew’s Correlation Coefficient (MCC). The considered evaluation metrics are computed using the following equations:

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

(6)

$$Precision=\frac{TP}{TP+FP}$$

(7)

$$Recall=\frac{TP}{TP+FN}$$

(8)

F1=Measure = (2*Precision*Recall)/ (Precision + Recall)

(9)

MCC = (TP*TN – FP*FN) / √(TP+FP) (TP+FN)(TN+FP)(TN+FN)

(10)

For all the previously mentioned metrics, the higher the value, the better the performance of the classification model.

Table 9 shows the accuracy and precision results for the twenty-nine considered classification models on the two considered datasets. According to Table 9, IBK and RandomForest classifiers achieved the highest accuracy and precision results on dataset 1. For Dataset 2, IBK showed the best results among the twenty-nine considered classifiers with respect to accuracy and precision metrics. Moreover, RandomizableFilteredClassifier showed the best accuracy result on Dataset 2 and the second-best precision result on the same dataset.

Figure 4 depicts the constructed tree for Dataset1 when using RandomTree as a classification model.

Table 10 depicts the evaluation results for the twenty-nine considered classifiers on both datasets, considering recall and F1-measure metrics. According to Table 10, the IBK classifier achieved the best recall results on both datasets and the best F1-measure result on Dataset 1. RandomForest classifier achieved the best F1-measure result on Dataset2 in addition to the best recall result on Dataset1 along with the IBK classifier.

Table 11 depicts the MCC results for the considered classifiers on both datasets. Based on Table 11, the IBK classifier that belongs to the lazy learning strategy achieved the best MCC results on both considered datasets. Moreover, the RandomForest classifier, which belongs to the Trees learning strategy, achieved the best MCC result on Dataset 2.

Table 12 summarizes the best results obtained in Table 9 and Table 11 with respect to the five evaluation metrics considered on both datasets. For Table 12, RF stands for Random Forest classifier, and RFC stands for RandomizableFilteredClassifier.

According to Table 12, IBK and RandomForest classifiers are the best classification models to handle dataset 1, respectively, while IBK, RandomizableFilteredClassifier, and RandomForest are the best classification models to handle dataset 2.

5. Discussion

In this section, a comparative analysis regarding the best regression and classification models that could handle the task of predicting the problem of sleep disorders has been introduced. The analysis considers two datasets with respect to several evaluation metrics.

Regarding the best regression model to use, it has been clearly noted that no single regression model showed a general high performance considering all the metrics on both datasets. Therefore, it is highly recommended to utilize ensemble methods for this task with consideration for the best regression models, as shown in Section 3.2 (Multilayer Perceptron, SMOreg, and KStar).

For the best classification model to use, the IBK classification model showed superior performance compared with the other models. Nevertheless, other classification models showed excellent performance, like RandomForest and RandomizableFilteredClassifier. Hence, it is highly recommended to utilize these three classification models (IBK, RandomForest, and RandomizableFilteredClassifier) in ensemble learning for handling the problem of classifying disordered sleep.

Moreover, regarding the best learning strategy to use with the problem of sleep disorder, and generally speaking, the following strategies showed excellent performance: Lazy, Functions, Trees and Meta. In depth, Table 13 depicts the average results for the considered models with respect to the learning strategies they belong to. The shaded rows represent Dataset 2, while the unshaded rows represent Dataset 1.

According to Table 13, the Lazy learning strategy is the best learning strategy to use with the regression task for disorder datasets, considering the five metrics. Functions are the second-best learning strategy.

Considering the classification task, it is clearly seen from Table 13 that the best choice is to consider dataset 1, while the Tree strategy is the best choice when considering dataset 2, and for all five evaluated metrics. The conclusion that could be drawn is that the function strategy is more suitable for datasets that have a large number of features, while the Trees learning strategy is more preferred to use with datasets that have a smaller number of features.

Once again, based on Table 13, the function strategy showed superior performance considering the two considered tasks (regression and classification). Therefore, it is the most appropriate strategy to use with the prediction task of disorder datasets.

Finally, it is highly recommended to conduct more integrated research, considering experts from the machine learning domain and the sleeping disorder domain. Also, considering other new features than the features considered in the utilized datasets is highly recommended.

6. Conclusion and Future Work

Sleep disorders involves problems with the amount, timing, and quality of sleep, which results in several daytime problems such as fatigue, stress, and impairment in functioning. This research aimed to add knowledge to this domain by investigating the applicability of machine learning techniques in the domain of sleep disorders. Mainly, three objectives are considered in this research. These objectives are to identify the best regression model, the best classification model, and the best learning strategy to handle the sleep disorders dataset. The results showed that MultilayerPerceptron, SMOreg, and KStar are the best regression models, and IBK, RandomForest, and RandomizableFilteredClassifier are the best classification models. Finally, the function learning strategy showed superior performance compared with the other strategies, considering both regression and classification tasks on both datasets, with strong competition from the Lazy and Trees strategies. For future work, an ensemble learning model that consists of the best regression and classification models is highly recommended.