Sleep Stage Classification based on Two-Cycle Sleep Model Recognition using Continuous Heart Rate Variability

* Correspondence: nihb@nwpu.edu.cn; Tel.:+86-29-88431531 Abstract:Sleep stage on the whole night is not steady. Sleepers generally pass through three to five cycles. In each cycle, there are occur four typical sleep stages, such as wake stage (WS), light stage (LS), deep sleep (DS), rapid eye movement sleep stage (REM). According to the natural routine, in this paper, we investigate the stage transition and analyze the feature of stage transition using the local cluster Algorithm (LCA). Two-cycle sleep model (TCSM) is proposed to automatically classify sleep stages using over-night continuous heart rate variability (HRV) data. The generated model is based on the characteristics of the nested cycle's sleep stage distribution and the transition probabilities of sleep stages. Experiments were conducted using a public data set including 400 healthy subjects (female 239, male 161) and the model’s classification accuracy was evaluated for four sleep stages: WS, LS, DS, REM. The experimental results showed that based on the TCSM model, the segmentation classification of pure sleep is 5.2% higher than that of the traditional method, and the accuracy of segmentation classification is 11.2% higher than the traditional sleep staging accuracy. The experimental performance is promising in terms of the accuracy, sensitivity, and specificity rates compared with the ones of the state-of-the-art methods. The study contributes to improve the quality of sleep monitoring in daily life using easy-to-wear HRV sensors.

REM. Meanwhile, stage dynamic transition information reveals significant characteristics of the degree of sleep continuity [9]. The gold standard in this domain is overnight polysomnography (PSG), which include the electrocardiogram (ECG), electroencephalogram (EEG), electromyogram (EMG), ectrooculogram (EOG), and respiration signals [10]. Sleepers cyclically passed through five different stages of sleep, i.e., stage 1, stage 2, stage 3, stage 4, and stage 5 or REM sleep. Sleep in stage 1is light. The eyes move slowly and muscle activity is slow.
It is in stage 2 that eyes stop moving and the brain waves become slower. Deep sleep occurs in stages 3 and 4 when no eye movement and muscle activity exist [11]. But the recording of the necessary electrophysiological signals is extensive and complex and the environment of the sleep laboratory, which is unfamiliar to the patient, might lead to distorted results. Meanwhile, manual stage scoring on PSG by domain experts is time-consuming and labor-intensive.
At present, many studies focus on analyzing the HRV features, and some of them are focusing on one stage distribution and transition [12]. Besides, deep learning is also used to classify sleep stage scoring. In this paper, we divided sleep stages into four categories: WS, LS, DS, and REM stage, during which were also called pure sleep stage. Pure sleep stage means in one segment; no other sleep stages are shown. The sleep stage detection algorithm is proposed that used only the heart rate signal, derived from ECG, as a discriminator. This would make it possible for sleep analysis to be performed at home, saving a lot of effort and money. ECG data along with a hypnogram scored by professionals was used from the SHHS database, make it easy to compare the results. For sleep stage classification, we analyzed the sleep stage structure and proposed a TCSM concept and on this foundation, a new framework of sleep stage classification and prediction are presented based on machine learning and pattern recognition technique.
The rest of this paper is organized as follows. Firstly, we review the related work in section 2. Afterward, we present the datasets, ECG signals preprocessing and sleep stage smoothing algorithm in section 3. The details of our solution are also described in section 3. In section 4, Local Cluster Algorithm (LCA) for smoothing sleep stage results and TCSM model results are presented. Finally, we conclude the paper and discuss future work in section 6.

Related Works
Many systems using ECG signals have been proposed in the literature for sleep stage classification. In this section, we briefly review the related work which can be grouped into three categories, different biological signals were adopted to classify sleep stage, sleep stage classification based on stage distribution and transition, feature analysis, and extraction based sleep stage classification.

Signals adopted to classify Sleep stage
Such methods are divided into two categories, i.e. multi-channel and single-channel processing. In the former approach, the combination of various biological signals such as multichannel EEG [16] [17] signals, electromyogram (EMG) [18] and ectrooculogram (EOG), radio measurements without any attached sensors on subjects [19] capturing the temporal progression of sleep, are utilized to extract informative features. In [29] study, actigraphy and heart rate signals are used to classify the sleep stage. HRV and R-R intervals from EEG signals are used to classify the sleep stage [20]. According to the available evidence [22], EEG signals are almost sufficient for reliable scoring, but the challenge is getting the EEG signals from the subjects. Conversely, HRV data are easy to access from non-contact sensors.

Stage distribution and transition
The probabilistic properties of sleep stage sequences and transitions are used to improve the performance of sleep stage detection using cardiorespiratory features [12]. The classifier, based on conditional random fields, achieved an average accuracy of 87.38%. [13] leverages both signal and stage transition features of human sleep for automatic identification of sleep stages. Random forest classifier is trained using thirty EEG signal features. The correction rules with a Markov model were constructed based on stage transition feature, importing the continuity property of sleep and characteristic of a sleep stage transition. In [14], the authors applied a smoothing process on classified results to improve their classifier's performance, which considers the continuity of a subject's sleep. It considered the temporal contextual information and improved the continuity of a subject's sleep stage scoring results. Some correction rules were constructed according to the relationship among the current epoch, priorepoch, and posterior-epoch. Similarly, correction rules are used in paper [14]. However, their correction rules only contained a few manually defined rules and need more prior knowledge of the sleep study and characteristics of the sleep dataset. [23] previously showed that human sleep-wake stage distributions exhibit multi-exponential dynamics, which are fragmented by obstructive sleep apnea (OSA), suggesting that Markov models may be a useful method to quantify architecture in health and disease. Some works used a hidden Markov model [24] to classify the sleep stage.

Feature analysis and extraction
In [28], the author proposed a sleep stage classification system based on noise-reduced fractal property of heart rate variability, sensitivity is 77% that distinguish wake stage from the other sleep stages and 72% deep sleep stage from light sleep; In [29] study, actigraphy and heart rate signals were used to classify sleep stage. Body movement indices derived from actigraphy data and autonomic functional indices from heart rate variability is used for discriminating between non-REM sleep and waking/REM sleep at 76.9% sensitivity and 74.5% specificity and between REM sleep and waking at 77.2% sensitivity and 72.3%specificity. In the [30] study, REM sleep was determined with an adaptive threshold applied to the acquired feature. Only the heart rate signal feature: LF/HF ratio, the relative peak frequency power in the HF band, the variability within the HF band, derived from electrocardiogram (ECG), to perform classification [31]. The authors tried to detect sleep and wakefulness using a combination of ECG, actigraphy, and respiratory signals [32]. With an automated Bayesian classifier, they achieved an accuracy of 86.8%. In the study of [33], the author grouped the 6 consecutive windows into one epoch (length, 3 min), whose stage is defined as that of the majority among 6 windows, i.e., if the stage of ≥4 windows were the same, it was chosen as the stage of an epoch. If there was no majority stage among 6windows, the epoch was defined as a transitional stage. Moreover, the existing work only focused on analyzing signal features without considering the dynamic stage transition information.
The main contributions of this paper are listed as follows.
(1) Sleep structure characterization and modeling. Sleep is a gradual evolutionary process.
Sleeping overnight usually takes three to five sleep cycles, and when transitioning from one sleep stage to another, there must be a transition, usually called sleep stage transition; Accordingly, if there is no stage transition, called pure sleep in this paper, and the local cluster algorithm (LCA) method is used to smooth the sleep stage to obtain pure sleep and stage transition.
(2) HRV feature extraction and selection. This paper extracts ECG data from PSG data, and then uses the heartbeat interval detection method to detect the R wave to calculate the RR interval and perform false and missed detection. Based on the HRV data and sleep structure, for the sleep staging requirements and the TCSM model, the time domain, frequency domain and nonlinear domain characteristics of the HRV are utilized respectively, and the feature selection is based on the max-relevance min-redundancy criteria.

Material and Methods
In this section, we introduce the dataset, ECG and sleep stage preprocessing, the system flowchart and classification. The method consists of three phases: processing, feature selection, and classification. The flowchart of the whole architecture is shown in Fig.1.

Data Collection
There are two standards commonly used to define sleep stages: The Rechtschaffen and Kales (R&K) [34], and that developed by the American Academy of Sleep Medicine (AASM) [35]. The AASM standard adopted in this paper classifies sleep into 4 different stages. To develop and validate the proposed method, a set of PSG recordings from real subjects are used.
The recordings are taken from the Sleep Heart Health Study (SHHS). This database emerged from a multi-center cohort study to determine cardiovascular and other consequences of sleepdisordered breathing. Details about the design of the SHHS study can be found in [36]. Each recording comes with the annotations of physicians' off-line scorings following the R&K procedure [37]. The Sleep Heart Health Study (SHHS) is a multi-center cohort study [36], initiated by the American National Heart Lung and Blood Institute to determine whether sleepdisordered breathing is associated with a higher risk of various cardiovascular diseases. The study includes two rounds of PSG recordings. We use only the first round (SHHS-1) because all records have the same sampling rate (125 Hz), contrary to the second round where records can be sampled at 125 or 128 Hz. Dataset SHHS-1 contains 5793 PSG records.
Here we adopted 400 subjects (male 161 subjects, female 239 subjects) and all of them were healthy. And the age is between 40 and 59, the mean age of all the subjects is 53.44. In the PSG records, for most subjects, along the 'wake' period before the subject goes to sleep and another after he or she wakes up is observed [38]. To preserve all the sleep data and HRV signals, the WS is not trimmed at the beginning of sleep. The hypnogram, which reports the different sleep stages across the sleep time, was obtained in TABLE I. We use the ECG deal package [39] to deal with the ECG signal and get the R wave index.
The R wave detection results are as the Fig.2. According to the definition of HRV, the RR interval is calculated by the subtraction of later R wave and its adjacent front one. The sequences of the RR interval were HRV. Before the extraction of the frequency measures, a preprocessing was carried out on the RR interval based on the fact that the spectral analysis cannot be affected only on regularly sampled signals [40]. This preprocessing is presented by resampling the data at a frequency of 4 Hz using an interpolation method developed by Berger et al [41].

Sleep Stage Structure Characterization
Pure Sleep Stage Analysis:        unlikely that the stage suddenly changed [13]. Consequently, we smoothed the data. The sleep stage smoothing rules are based on the local cluster Algorithm (LCA).
The stage transition was ambiguous and the time of stage transition was not very clear.
Consequently, we defined the stage transition time domain as the T minutes which mean that the former T/2 is one sleep stage, and the later T/2 is another sleep stage. For example, if T is 5, the former 2.5 minutes is in LS, the latter 2.5minutes in another maybe in WS, DS or REM. The next will explain how to pre-smooth the sleep stage based on the LCA algorithm and T is set 5.
Firstly, the LCA obtains every stage and corresponding count. Secondly, LCA sets domain 2, if the corresponding count less than the domain, the stage will be changed to the adjacent stage whose count is larger. After that, the LCA sets domain 3, the similar processing as the above until the domain equal to T. Finally, the minimum stage's corresponding count is T.
As seen in Fig.3, the initial sleep stage is 00000011212222332220022211112, which corresponding count all are 1. After merging the same value, the stage is 01212320212, and the corresponding count is 62114232341 which domain is 1. Next, the domain is set 2, which means that only the countless than domain, the corresponding stage will be smooth. The smoothing rules are as follows: if the former stage's count larger than the latter, the current stage will be changed to the former; If the latter stage's count larger than the former, the current stage will be changed to the latter. The same operations of the domain in 3,4,5. Figure.9 The LCA method for sleep stage smooth.
The LCA pseudocode is as the Algorithm 1. After LCA algorithm, the results are as TABLE IV.      1. Use the sliding window for segmentation, the purpose was to build the sample, use the classifier to classified which was pure sleep, which was stage transitions, the time scale was 1-2. Further subdivide the detected pure sleep into pieces at different times (1-10 minutes) 3. Directly classified the detected sections, this section did not equal, some were 10 minutes, some were half an hour, some were a few minutes (Fig.12); large did not divided Pure sleep is called a slice, which is called a piece that is cut into fixed lengths (Fig.13); 4. Classified the stage transitions. Based on the first step, the stage transition was detected, and the stage transition was further subdivided into 6 classifications or 12 classifications (Fig.14). To set the system parameters and to evaluate the system performance, we used a k-fold cross-validation process. To do this, the dataset divided randomly into k equal-sized subsets.
At each fold, the (k-1) subsets were used as the training and validating data and 1 subset was used for testing. This process was repeated k times (the number of folds), with each subset used exactly once for testing. We also used a k-fold subject cross-validation process, where k was the number of subjects in the dataset. At each fold, the data of one subject were used for testing, while the data of other subjects were used as the training and validating data. This process was repeated k times. The results of the k folds were averaged and reported as the system performance.

Results
In the experiment, we firstly classified the stage transition and pure stage and then classified the four pure sleep stages. We choose the Random Forest (RF) classifier to classify the different classes. RF classifier, proposed by Breiman [43], was a classification and prediction models, belongs to a kind of combination classifier. It overcomes the problem of easy to converge to the local optimal solution and the problem of excessive fitting which may occur in the single decision tree model and can be a very good tolerance data set of noise and outliers, suitable for processing the unbalanced data sets and high dimension data. In the first step, according to the stage transition definition, we constructed the train set. If the restored stage, we choose the last 2/T stages and the first 2/T stages as the stage transition, and calculated the feature of the corresponding HRV.
Due to the ambiguity of stage transition during sleep in different people, it is difficult to directly set the stage transition to a fixed length [43]. Therefore, in the experiment, this paper classifies the stage transition and pure sleep at different time scales, and uses the detection rate of the stage transition (that is, the degree of effect, which evaluates the proportion of all negative classes detected) is appropriate in the case of evaluating how long the time scale is set.
As shown in Fig.15, in 1-10 minutes, as the time scale increases, the detection rate of the stage transition increases; from 5-10 minutes, the detection rate decreases. Too long or too short is a difficult classification during sleep. It can be seen from the experimental results that the stage transition is set to 4 minutes.  classified. It can be seen from the figure that pure sleep was sliced for 5 minutes, then classified, using three classifiers. The highest accuracy rate is 86.618% with HMM. Figure From the classification results, the classification effect using HMM classifier was the best, 81.6%. Figure.18 Pure sleep stage classification in slices using different classifier in a different time scale minutes, and the time difference between the traditional sleep stage and 5 minutes was not much different. Therefore, in the experiment, this paper used the 6-minute At the same time, by setting different time scales, the paper analyzed the HRV segmentation, the scale was set to 6 minutes, and the classification effect was better. This article compared the traditional method with TCSM and used 34 people's data to illustrate. From the data, it can be seen that there are 30 people whose classification effect is higher than the traditional classification method, more than 7%. The model presented in this paper is valid and feasible, as shown in Fig.22.  The study has some limitations: 1) Only signals from a single ECG channel were used in classification, 2) The pure sleep stage and stage transition classification were not satisfactory, 3) We have not considered the effects of diseases, such as narcolepsy, insomnia and sleep disorder, on their influences on classification.
In future work, we will focus on sleep stage scoring using multiple PSG signals, such as EOG and EMG, to improve the performance of our approach. Moreover, we will design more elegant algorithms for analyzing and classifying sleep data. Additionally, we will apply our approach to more datasets, especially to the datasets of different sleep disease studies, which can be obtained from the NSRR website [44]. In future work, the inner cycle will be classified directly rather than slicing into 5 minutes.