Clustering and Machine Learning Models of Skeletal Class I and II Parameters of Arab Orthodontic Patients

1. Introduction

Angle’s classification is one of the most widely used in malocclusion classifications [1]. Angle class I molar classification is determined by mesiobuccal cusp of the maxillary first molar occluding with the buccal groove of the mandibular first molar [2]. Class I malocclusion was also categorized into five types by Dewey's modification. In Dewey type 1, anterior teeth are crowded, and in Dewey type 2, maxillary incisors are proclined. In type 3, there is an anterior crossbite, whereas in type 4, there is a posterior crossbite. Type 5 is characterized by permanent molars’ mesial drifts [2,3]. SCIIMO’s etiology is heterogeneous, because it can be caused by a mandibular retrusion, or maxillary protrusion, or both of them [1]. Also, in SCIIMO, patients are categorized by the interarch relation. In a Class II molar relationship, the mesiobuccally cusp of the maxillary first permanent molar occludes mesial to the buccal groove of the mandibular first molar with correct inclination of the front teeth. Class II division 1 occurs when the maxillary incisors are protruded (upper incisors are proclined), often resulting in an excessive overjet and deep overbite. Class II division 2 occurs when the maxillary central incisors are palatally inclined and eventually overlapped by the maxillary lateral incisors. A deep overbite and a broad maxillary arch define a class II division 2 [2].

Many approaches have been applied to diagnose craniofacial anatomy, commonly based on bidimensional imaging. In recent years, three-dimensional (3D) technologies have been developed, and more tools have been enabled in orthodontics [3].

3D can define the occurrence of malocclusion: sagittal, transverse, and vertical [4]. In the sagittal plane, the intermaxillary angle (SNA – SNB = ANB), which was recommended by Steiner 5 to determine an individual’s skeletal class, indicates SCIO if the ANB angle has values ranging between 0° and 4°, and SCIIMO if the ANB angle presents values >4° [5,6]. According to Jacobson [7], the ANB angle does not consider the relative relationship of the denture bases to cranial reference planes. Due to this limitation, the “Wits” appraisal parameter was suggested, which overcomes this shortcoming and enables the measurement of the severity of the degree of anteroposterior jaw disharmony from lateral cephalograms [7,8]. In the following years, many studies presented equations that consider the individual properties of the ANB angle and Wits appraisal [9,10,11,12,13] .

In summary, all traditional skeletal classifications depend on manually calculating linear and angular variables using the craniomaxillary and mandibular landmarks. However, due to variations in the mandible’s position that can be a result of occlusion and temporomandibular joint, skeletal classification can be difficult [14]. To overcome these difficulties, classification algorithms like the support vector machine (SVM) can generate skeletal classifications based on the automatically extracted craniomaxillary variables [14,15]. Another study that examined automated skeletal classification found that convolutional neural networks (CNN), using the patient’s sex and a cephalogram, exhibited >90% sensitivity, specificity, and accuracy for vertical and sagittal skeletal diagnosis [16]. Taraji et al. [17], used cephalometric parameters, along with covariates such as gender, age, and race, to appraise machine-learning algorithms for adult Class III malocclusion treatment planning. Taraji et al. [17] study demonstrated that artificial neural network algorithms predicted treatment approach with 91% accuracy. The model highlighted Wit's appraisal, anterior overjet, and Mx/Md ratio as key predictors.

In a scoping review that evaluated the use of artificial intelligence in orthodontics, and included 62 studies, demonstrated that the majority of the studies originated from the USA, South Korea, Japan and China. In addition, the review revealed that diagnosis and treatment planning field, was one of the most major domains that were investigated [18].

Therefore, the main aim of this study was to derive a novel classification machine learning (ML) model to predict whether it’s SCIO or SCIIMO malocclusion among Palestinian Arab residents of Israel patients, using lateral cephalogram parameters, in addition to gender and age as covariates. This population can be considered a permanent population of this area, with family histories dating back for generations and high levels of consanguinity.

To our knowledge, this research will be the first to apply machine-learning models to this ethnic group. As a secondary outcome, a clustering analysis was intended to represent the data in clusters and to examine the differences between these clusters.

2. Material and Methods

2.1. Ethical Statement

This research was conducted according to current guidelines of the ethics and regulations of the Ethics Committee of the University of XXXXX (approval number 19-1596-101 (dated 13.11.2019)). This study consisted of 394 coded records of Palestinian Arab citizens of Israel who were diagnosed as SCIO or SCIIMO. All information collected as part of the standard of care by the orthodontists’ team at the XXXX.

The inclusion criteria were:

• Patients diagnosed with SCIO (-1≤Calculated_ANB≤1) or SCIIMO (Calculated_ANB >1).

Calculated_ANB is defined as equal to ANB measured – ANB_ind. The ANB_ind was defined by Panagiotidis and Witt [9]: ANB_ind = -35.16 + (0.4 x SNA) + (0.2 x ML-NSL).

In some cases, patients were included as SCIO or SCIIMO, even when the Calculated_ANB was not in the accepted range, according to clinical diagnosis and other important parameters, like the ANB angle and Wits appraisal. The fact that the Calculated_ANB is not suitable for all patients was described by Panagiotidis and Witt [9].

2.

Available pre-treatment cephalometric data.

In this study, we performed a clustering analysis. In addition, we applied various ML algorithms, which differed in the number of input variables, enabling us to precisely classify the patients as skeletal class I or II via automatic machine learning techniques.

2.2. Cephalometric Variables

The following cephalometric parameters were the essential variables in this study analysis, and the complete information and location of all the parameters are presented in Supplementary Table 1 and Supplementary Figure 1.

2.3. Clustering Analysis

Data was analyzed using the R software platform, Clustering Analysis. Before starting the process of clustering, we conducted scaling process to reach a common scale.

The clustering algorithm was performed for 3 clusters, including SCIIMO or SCIO patients (separately). A scatter plot and dendrogram were produced using the R statistical program to implement the visualization of the cluster analysis results. We got the optimal cluster number by inspecting the dendrogram of different clusters.

To implement a hierarchical clustering algorithm, one has to choose a linkage function that defines the distance between any two sub-sets [19]. In all our clustering calculations, we mainly used the Ward error sum of squares hierarchical clustering method that has been commonly used since its first description by Ward in 1963 [20].

The performance of the machine learning models was evaluated by determining the accuracy and kappa scores.

2.4. Machine Learning Methods

Machine-learning analysis was applied using the R package Caret [21]. Before starting this analysis, we preprocessed the data with centering and scaling functions to reach a standard scale and to improve the performance of the models.

2.5. Classification Models

Classification models - Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), K Nearest Neighbor (KNN), Random Forest (RF), and Classification and Regression Tree (CART). They were all applied using the K-fold cross-validation (K=10).

2.6. Linear Discriminant Analysis (LDA)

LDA was proposed by R. Fischer in 1936. It consists of finding the projection hyperplane that minimizes the interclass variance and maximizes the distance between the projected means of the classes [22].

2.7. Support Vector Machine (SVM)

The machine conceptually implements the idea that input vectors are non-linearly mapped to a high-dimension feature space. In this feature space, a linear decision surface is constructed [23]. This model can be relatively simple and flexible for addressing various classification problems. SVMs afford balanced predictive performance distinctively, even in studies with limited sample sizes [24].

2.8. K-Nearest Neighbors (KNN)

The nearest neighbor decision rule assigns an unclassified sample point to the classification of the nearest of a set of previously classified points. The k defines how many nearest neighbors need to be examined to classify the class of a sample point [25,26,27]. This study used accuracy to select the optimal model using the largest k-value. The final value used for the general model (all parameters), model 1 (ANB angle), and model 2 (ANB angle and Wits appraisal), was 9 (k=9), while the model without the ANB parameters was k=5 (5 neighbors).

2.9. Random Forest (RF)

It is a classification method that uses many decision trees. This algorithm is a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest [27,28].

2.10. Classification and Regression Tree (CART).

CART analysis is a form of binary recursive partitioning. In this method, each group of patients, represented by a “node” in a decision tree, can only be split into two groups. Each parent node can be split into two child nodes. The term “partitioning” refers to the fact that the dataset is split into sections or partitioned. It’s important to know that CART can handle numerical data that are highly skewed or multi-modal, as well as categorical [29].

2.11. Model Validation

We divided our data into 70% for training and 30% of the data for validation (unseen data). We validated our models using the k-fold cross-validation approach. Cross-validation provides a simple and effective method for both model selection and performance evaluation; under k-fold cross-validation, the data are randomly partitioned to form k-disjoint subsets of approximately equal size [30,31]. K (10)-fold cross-validation was employed in this research. For conducting the ML models and calculating the Accuracy, Kappa, Receiver-operating characteristic curve (ROC), Sensitivity, and specificity scores, we used the R package Caret [21].

3. Results

This study included 394 patients, with 157 (39.8%) presenting SCIO and 237 (60.2%) SCIIMO. The total study collective consisted of patients aged between 7 and 55 years, with a mean age of 19 ± 7.1 years in class I and 17 ± 6.5 years in class II cases. Concerning the gender distribution, among SCIO patients, 66% were female (n = 104), and in class II 67% were female (n = 158). The characteristics of the study collective, including demographic and cephalometric data, are presented in Supplementary Tables 2A-B for SCIO and SCIIMO patients, respectively.

3.1. Clustering

In this section, we performed hierarchical clustering analysis, and we inspected the number of clusters according to the dendrogram result. It was acceptable to present the current results with k=2,3 and 4 clusters. However, we decided to describe the k=3 clusters and show the distinct variations between the clusters. The same analysis was performed for skeletal class I and II separately.

3.2. Skeletal Class I Occlusion (SCIO) Clustering

The hierarchical clustering results showed that the majority of patients were assigned to cluster 1 consisted of 75 (n=75) patients, and cluster 2 (n=54), while 27 patients (n=27) were assigned to the third cluster (Figure 1A & Table 1A).

In addition, the three clusters interestingly variated significantly (P<0.05) in all cephalometric parameters except in the parameters (Calculated_ANB and Wits appraisal, P>0.05). Besides, from the results, age differences were not significant between the three clusters. The results showed that in the second cluster, retrognathism of the mandible was less severe, as represented by a a lower ANB angle. In addition, the third cluster had a lower NL-ML angle, gonial angle, SN-Ba angle, and lower ML-NSL angle compared to clusters 1 and 2, which demonstrate the distinct features from the other two clusters. Detailed information can be found on Table 1B.

We repeated the same clustering analysis with males and females separately and found that there are some similar patterns, like the fact that in both males and females, the second cluster was characterized by less severe retrognathism of the mandible, as represented by a lower ANB angle. However, the results also showed variations between males and females. For instance, most dental parameter variations were not significant between the three clusters among males, and significant among females clustering. The results showed that age was insignificant between clusters for both males and females. Overall, females showed more significant differences among clusters among the cephalometric parameters (Table 1C).

3.3. Skeletal Class II Malocclusion (SCIIMO) Clustering

In skeletal class II hierarchical clustering, the results showed that the majority of patients were assigned to cluster 1, consisting of 125 (n=125) patients. In contrast, 111 patients (cluster 2, n=62, cluster 3, n=49) were assigned the second and third clusters (Figure 1B & Table 1A).

In addition, the three clusters interestingly variated significantly (P<0.05). Also, here, among skeletal class II patients, age differences in the different clusters were not significant. Interestingly, the results showed a significant among most of the sagittal parameters, especially the ANB angle and the Calculated_ANB. The results showed that the second cluster has less severe retrognathism of the mandible, which is represented by a lower ANB angle and Calculated_ANB and a higher SNB angle (P<0.05). Detailed information can be found in Table 1D.

Finally, we repeated the same clustering analysis with males and females separately and found that age differences were significant between clusters for males and females (P<0.05). Cluster 1 here was characterized by a younger age average M=12.90 (M=12.90, SD=2.20) among males, and M=15.88 (M=15.88, SD=5.11). For both males and females, the results showed that the first cluster has less severe retrognathism of the mandible, which is represented by a lower ANB angle and Calculated_ANB (significant among males only, P<0.05), compared to the other two clusters. In addition, and on the contrary to skeletal class I, here Wits appraisal variations were significant between the clusters for both males and females (P<0.05), and was lower in the first cluster, compared to the other two clusters. Overall, also among skeletal class II patients, females showed more significant differences among clusters among the cephalometric parameters (Table 1E).

3.4. Machine Learning Models

Considering the knowledge about cephalometric measurements in SCIO and SCIIMO obtained from cluster analysis and comparisons of cephalometric parameters between subgroups, various machine learning models were tested to predict the classification of an individual based on machine learning (ML) models that will not be based on the Calculated_ANB (model 3).

The general ML model, which included all cephalometric and demographic parameters, could predict a SCIO or SCIIMO with an accuracy of 0.87 (0.87%) (RF, Accuracy = 0.87, Kappa = 0.74, ROC = 0.92, Sensitivity = 0.92, and Specificity = 0.83). As expected, calculated_ANB was the most critical parameter in the model, followed by Wits appraisal, ANB, -1/NB, and gonial angle, as shown in Figure 2.

The first model included only the ANB angle, the second important parameter in the general model. In this model, we received an accuracy of 0.75 (LDA, Accuracy = 0.75, Kappa = 0.47, ROC = 0.79, Sensitivity = 0.59, and Specificity = 0.86). Results are presented in Figure 3A,B.

The second model included the ANB angle, and the Wits appraisal, which improved the accuracy to 0.78 in the KNN model (KNN, Accuracy = 0.78, Kappa = 0.56, ROC = 0.82, Sensitivity = 0.80, and Specificity = 0.76) (Figure 4A,B). A decision tree visualization for these two parameters is presented in Figure 4C. Finally, adding the third model (model 3) included all parameters except for ANB, ANB_ind, and Calculated_ANB and achieved 0.82 accuracy in the LDA model (LDA, Accuracy = 0.82, Kappa = 0.63, ROC = 0.88, Sensitivity = 0.75, and Specificity = 0.87). In addition, we examined the classification ability without ANB, ANB_ind, and Calculated_ANB parameters via decision tree, which showed that the tree starts with Wits appraisal, followed by SNB, NL-NSL, +1/NA (mm), and age in the first three branches (Figure 5A–D). A summary of all the ML models is available in Table 2.

4. Discussion

Our study aimed to reveal novel information about the Palestinian Arab ethnic minority who are citizens of Israel. Hierarchical clustering analysis was performed separately for both skeletal class I and II patients, and based on the dendrogram we decided to apply 3 clusters for every analysis. Among skeletal class I patients, three distinct patterns were revealed. The second cluster was characterized with less severe retrognathism. For skeletal class II patients, we also applied hierarchical clustering for three clusters, and here also the results showed a significant among most of the sagittal parameters, especially, the ANB angle, and the Calculated_ANB. The results showed that the second cluster has less severe retrognathism of the mandible which is represented by a lower ANB angle and Calculated_ANB, and higher SNB angle. Interestingly, age differences were significant between clusters, among males and females.

In a study that was done by Moreno Uribe et al.[32] about phenotypic diversity in white adults with Class II malocclusion, it was found that models with 2, 3, or 4 clusters were statistically acceptable. Still, in the end, they identified five distinct Class II phenotypes [32].

A Cluster analysis study in Class I occlusion revealed that the grouping pattern in Class I occlusion is present at younger age levels and disappears with age. Also, they found that the clustering pattern is very similar in males and females with Class I occlusion [33].

Finally, the general ML model that included all parameters could predict an individual as SCIO or SCIIMO with 0.87 accuracy (RF, Accuracy = 0.87, Kappa = 0.74, ROC = 0.92, Sensitivity = 0.92, and Specificity = 0.83). As expected, Calculated_ANB was the most critical parameter in the model, followed by ANB angle, Wits appraisal, and ANB_ind. Our Machine Learning results demonstrated that with the ANB angle and the Wits appraisal, two cephalometric parameters can predict skeletal class I/ II with 0.78 accuracy (KNN, Accuracy = 0.78, Kappa = 0.56, ROC = 0.82, Sensitivity = 0.80, and Specificity = 0.76). In addition, our third model (model 3) included all parameters except for ANB, ANB_ind, and Calculated_ANB and achieved 0.82 accuracy in the LDA model (LDA, Accuracy = 0.82, Kappa = 0.63, ROC = 0.88, Sensitivity = 0.75, and Specificity = 0.87).

In recent research that was conducted by Midlej et al. [34] and aimed to accurately classify individual Arab patients, being citizens of Israel, as skeletal class II or III, found that Wits appraisal and SNB angle were able to predict the classification of patients to either skeletal class II or III with an accuracy of 0.95.

Jayathilake et al. [35] examined the prediction of malocclusion patterns using a classification model. Their study considered SNA, SNB, and ANB as cephalometric variables. The patients were classified into malocclusion patterns according to the ANB angle (pattern I, II, and III). The authors found that the accuracy of the multinomial logistic regression model, K-NN algorithm, random forest, and Naïve Bayes classification of malocclusion patterns were 88.89%, 83.33%, 88.89%, and 55.56%, respectively.

In another study that aimed to accurately diagnose skeletal class-III malocclusion applied through mobile images, and used three models: a deep learning algorithm, a machine learning algorithm, and a rule-based algorithm, found that the best model was able to correctly classify skeletal class-III malocclusion, with an accuracy of 76% [36].

4.1. Traditional vs. New Machine Learning Methods

When comparing the performance of ML models compared to traditional diagnostic approaches, like the Calculated_ANB formula that we used in this article to classify the patients. We need to consider that this regression formula (Calculated_ANB) and others, don’t fit all cases. In many cases, patients are diagnosed clinically, and according to other cephalometric parameters, like ANB angle and Wits appraisal. Panagiotidis and Witt [9] investigated this limitation by the correlation coefficient of the ANB_ind equation, as r = 0.808. In addition, the available individualized were established based on specific populations, that consider the individual properties of the ANB angle and Wits appraisal [9,10,11,12,13]. On the other hand, this research and other studies suggest to apply uniform machine learning algorithms, that can be trained on one ethnic group, like this study, or future studies that will aim to combine different ethnicities models.

4.2. Limitations

This study applied clustering analysis only on three clusters. Although the results showed significant patterns within each skeletal class, further investigations for different cluster numbers could be done. This study used a machine learning model based on a moderate sample size, and future studies should consider increasing it. Furthermore, our results were drawn from specific populations, and the results might vary among other populations. Finally, future research should aim to include all skeletal malocclusion classifications, and not only skeletal class I and II.

4.3. Conclusion and Future Research

This research revealed new information regarding the distinct characteristics of each cluster within each patient group (SCIO/SCIIMO) based on various cephalometric parameters.

Among skeletal class I patients, three distinct patterns were revealed. The second cluster was characterized by less severe retrognathism. Regarding skeletal class II patients, the three clusters showed significant differences among most of the sagittal parameters. The results showed that the second cluster has less severe retrognathism of the mandible which is represented by a lower ANB angle and Calculated_ANB, and higher SNB angle. Interestingly, age differences were significant between clusters, among males and females on in SCIIMO patients. These results can have implications both on the diagnosis and treatment plan

The ML models showed a high ability to predict a SCIO or SCIIMO with an accuracy of 0.87 (RF, Accuracy = 0.87, Kappa = 0.74, ROC = 0.92, Sensitivity = 0.92, and Specificity = 0.83) in the general model and a 0.78 accuracy (KNN, Accuracy = 0.78, Kappa = 0.56, ROC = 0.82, Sensitivity = 0.80, and Specificity = 0.76) using Wits appraisal and ANB angle. The study presents a machine learning model as a promising universal approach for precise and fast skeletal class I/ II diagnosis, advancing personalized orthodontic diagnostics and treatment. Finally, further research is recommended to be done, on applying artificial intelligence and machine-learning methods on the treatment choices and outcomes.