Bone Age Measurement using a Hybrid HCNN-KNN Model: A Case Study on Dental Panoramic Images

Bone age measurement is the process of evaluating the level of skeletal maturity to estimate actual age of bone. This evaluation is usually done by comparing a radiograph of a bone with an existing standard chart that includes a set of identifiable images at each stage of development. Manual methods are based on the analysis of specific areas of bone images or dental structures. Both methods are highly dependent to human experience and are time-consuming. An automated model therefore is needed to estimate the age accurately. In this study, we propose a hybrid convolutional neural network (CNN) combining K nearest neighbours (KNN) and PCA to estimate the age of bone automatically and accurately. We applied our model, HCNN-KNN, on a dataset collected by dental teaching institutes and private dental clinics in Malaysia. A total of 1,922 panoramic dental radiographs of dental patients aged between 15 to 25 years old were obtained from the various centres. These radiographs were separated by age, classified as those in the range of 12-months, six-months, three-months, and one-month gaps. This novel investigation, implemented for the first time with precision to the range of the age for ± twelve months, ± six months, ± three months, and ± one month, and these age ranges determine the age of minors which could help the model to find better features and train the model more accurately. Replacing SoftMax with KNN generally improves traditional CNN performance to reduce the noises in images. Therefore, the optimal number of image similarities in a larger dataset is more significant, and the proposed method can benefit from large amounts of annotated data. Since the similarities of radiographic images are very similar, there may be several similar possibilities in the SoftMax classification method. These similar probabilities increase the risk of misdiagnosis of bone age measurements. Therefore, replacing KNN with SoftMax is the best choice for age group differentiation in classifiers. Finally, the accuracy rate is evaluated with the accuracy criterion according to the equation in confusion metrics and comparing existing models. The accuracy results on the dataset by ± 12 months, ± 6-months, ± 3-months, and ± 1-month are 99.98, 99.96, 99.87, and 98.78, respectively.


13
Determining a person's actual age is legally essential, and the accurate identification of an 14 unidentified deceased is presently still a primary forensic concern. A solid reason to identify 15 the exact age of unknown individuals with a precise algorithm to solve the issue is at present. 16 Many illegal immigrants have died from natural disasters or the Coronavirus (COVID-19), 17 which is killing thousands of people around the world (Amon, 2020; Clark et al., 2020; 18 Satterstrom et al., 2020). Assessing the age of unknown human corpses is significant in 19 determining criminal investigations or a collective catastrophe because the age at death, date 20 of birth, year of death, as well as gender, can guide researchers toward the correct identity 21 among many matching cases (Sironi et al., 2018;Kapadia, Stevens, & Silver, 2020). 22 Most previous approaches for age estimation are in the primary stage in terms of accuracy. 23 Skeletal age can be measured using several different human bones (Wik et al., 2020;McGill, 24 2021). The high cost, long-term monitoring, and risk of radiation exposure suggest that this is 25 not feasible or practical for maximum human body segments (Shi et al., 2020). Researchers 26 have looked at the entire progress of the frame and examined the numerous approaches used in 27 different regions of the body (Lee et al., 2017;Spampinato, 2017; Shobha Rani, 2021). 28 Extensive medical age measurement methods are usually extracted based on centers' body parts 29 (Mutasa, 2018). The regions used to measure bone age from the body comprise the foot, 30 shoulder, ankle, hip, elbow, cervix, ankle, and teeth (Dahlberg, 2019). Therefore, maximum 31 age estimation used hand wrist and face characteristics (McGill, 2021;Shen, 2021). However, 32 the accuracy of these sections in estimating age may be affected by age, race, genes, work 33 environment, lifestyle, smoking, health status, and even other factors such as mortality and 34 disaster (Gurpinar, 2016). 35 To achieve the best and most accurate age detection from radiographic images, the use of dental 36 panoramic images is more effective than comparing to other parts of the body images as teeth 37 are immortal, long-lasting, and will even remain in high temperature and a dead body (Ezhil, As stated above, the skeletal maturity index of the extraction centre radiographs is bone age 1 (Cameriere, 2019). Despite the great content of scientific research on dental age measurement, 2 there is no consensus on the validity of bone age measurement (BAM) methods adapted for 3 use in clinical ambience (Bagattoni, 2019). In childhood, dental age can be correctly assessed 4 because most of the teeth are evolving concurrently (Ginnis, 2019). Still, among minors and 5 the adolescent population, age measurement cannot be precise (Asif, 2019) unless a predictive 6 model with optimal layers is provided for accurate results, which has not yet been developed 7 (Čular, 2017). 8 Traditional dental age measurement is a manual method that may include several steps such as 9 segmentation, feature extraction, image pre-processing, classification, or regression. The form 10 and variations in the above stages are seen in dry bone and radiographer images. Age 11 nomination is undoubtedly based on the timing of the emergence of extraction centres, 12 identifying the region of epiphysis and fusion, depending on whether the bone is dry, or using 13 an imaging technique such as radiography. Age segmentation assessment is based on a 14 corresponding procedure that involves comparing a radiographer's image of the subject to an 15 existing reference that includes a known gender and age sample. Estimating age is mainly a 16 measurement of biological maturation, which is changed into a time by comparing a picture 17 with a specified reference (De Tobel, 2020). A clear disadvantage of this traditional bone age 18 method is the accuracy in determining the age, especially in determining young age (minors 19 age) because this method uses to identify the age of the person by comparing standard charts 20 of the region of bone growth which usually cannot be an accurate as the bone growth changes 21 in a one-person depend on the health individual's care and lifestyle. An inversion in these 22 strategies, considering the differences in anatomical structure, the range of accuracy is very 23 wide, and the results came with the problem of low accuracy that leads this research to use 24 machine learning techniques instead of conventional methods. 25 Research has shown that most researchers have used machine learning algorithms like NN, 26 SVM, KNN, Decision Tree in bone age measurement (Tao et al., 2018;Farhadian, 2019;27 Hemalatha., 2021). Machine learning algorithms cannot process data in raw form, as they are 28 subject to errors that lead to inaccurate class recognition. Therefore, carefully engineered 29 attribute extractors need to convert raw data into attribute vectors for proper classification.

30
However, extracting characteristics that can accurately represent behaviour in different 31 environmental conditions is a challenging task that has left a wide range of research open to 32 researchers in this field (Houssein, 2020;Spampinato, 2017 6 Shamshirband, 2020). Moreover, deep learning approaches are effectively used to resolve 7 several issues in many analysis areas and computer vision. 8 Deep learning-based methods, also known as end-to-end learning-based methodologies, are 9 those in which deep neural networks, such as convolutional neural networks, operate directly 10 on the input images and generate the desired output without intermediate steps such as 11 segmentation and extraction (Haque, 2020). However, developing and training deep neural 12 networks can be a challenging and time-consuming process. It is possible to use a pre-trained 13 deep network to perform the desired tasks rather than creating and teaching deep neural 14 networks from scratch. Transfer learning is a term used to describe this process (Qummar,15 2019; Li, 2020). 16 Today, machine learning techniques has been used widely to identify patterns in complex data 17 including medical data. i.e., genomics, bioimaging, and phenotypic data [Alinejad-Rokny, the input images as the first step. In the second step, the model will be trained by a 30 convolutional neural network (CNN). In the third step, the fully connected layer from CNN 31 will be used as an input for PCA to perform data transform and dimensionality reduction. In 32 the last step, this data is classified using the KNN algorithm to predict classes in the dataset, 33 explained in detail in Section Three, the proposed model. This paper is divided into five sections: The strategies for determining dental age are detailed 2 in Section 2. The different phases of the proposed automated dental age calculation are depicted 3 in Section 3. The deployment elements and performance review are covered in Section 4. As a 4 result, Section 5 is divided into two sections: conclusion and future work.  7 Several approaches have been applied to identify the age using dental images. A deep neural 8 network algorithm estimating the age using dental X-ray images was used as a foundational 9 forensic science role. The features are derived using two deep neural networks, AlexNet and 10 ResNet, in the recommended process. A dataset with 1,429 dental X-ray images was used. The 11 proposed approach is tested using several acceptable output parameters (Houssein, 2020). 12 Avuçlu et al., 2020 used a morphological measurement on dental X-ray images 2020 to assess 13 age and gender. Details of age and gender were typically identified by function and dental X-14 ray images of the teeth. With 1315 dental images and 162 different dental classes, the photos 15 went beyond the borders. This dental knowledge is also contained in separate XML directories. 16 The highest estimated age and gender estimates are 95% for accuracy (Avuçlu, Başçiftçi, Fatih,  network that relied on dental data to alter age assessment. The age group of between 14 and 60 20 was used, and the overall data sample size for the research was three hundred images. For 21 statistical analysis, SPSS21 statistical software and R were utilized. The neural network 1 methodology, with an MAE of 4.12 years, showed reasonably good results (Farhadian, 2019). The tests utilize CNN to fix the inconvenience of forensic dentistry's automatic age 6 calculation without any modifications. The transfer learning principle is also used to train the 7 common CNN architectures for age estimation, such as AlexNet, VGGNet, and ResNet. Age 8 estimation efficiency is measured by evaluating its recall, accuracy, F1-score, accuracy, and 9 average accuracy for all the tested architectures. It has been claimed that this is the first paper 10 that seeks to predict age estimation from dental images using Capsule-Net with the best of 11 knowledge. However, the proposed architecture indicates that Capsule Network has increased 12 36% over CNNs and transfer learning to reach a cumulative accuracy of 76% (+_ 1 Year). experimentally confirmed that this latest feature set makes the dental age estimation more 16 reliable (+_ Year). 17 An age evaluation approach was tested on Malaysian adolescents between the ages of 1 to 17.

18
The first to the third molar teeth were segregated first. Next, based on an intensity projection 19 method, the invariant deformation characteristics were collected. Finally, the designed DCNN  24 Kim et al. (2019) applied deep learning algorithms to estimate age using dental panoramic 25 images and stated that this assessment is a fundamental task in forensic science. The dataset 26 used incorporates dental X-ray pictures of 9435 people (4963 male, 4472 female) sorted out in 27 three-age gatherings. The result of deep learning algorithms based on CNN neural networks 28 shows that the proposed approach functions evaluated based on a database of panoramic dental 29 radiographs and worked well for accuracy. with the agreement of three observers. In dealing with a restricted dataset, transfer learning 34 principles, using pre-trained CNNs, and data augmentation was used to minimize the problems. 35 On average, it took 2.72s for the entire automatic workflow to compute, which is considerably 36 quicker than conventional staging starting from the OPG. Given the small size of the dataset, 37 this pilot study indicates that the completely automated solution suggested demonstrates 38 encouraging outcomes relative to manual staging.
A semi-supervised fuzzy clustering algorithm with spatial limitations for dental segmentation 1 from X-ray images was introduced by Tuan et al. (2017). The experimental findings show that 2 the recommended work has superior accuracy than the initial semi-supervised fuzzy clustering 3 and several related approaches. 4 De Tobel J. (2017) developed machine learning algorithms to evaluate the growth of bones of 5 the hands, wrists, and dental on radiography and magnetic resonance imaging. Twenty 6 panoramic radiographs at every step for each sex were selected for the World Dental Federation 7 notation element (FDI) 38. The two monitors were unanimously determined on the steps. If 8 needed, a third viewer functioned as the referee to collect the third molar reference point. The 9 radiographic collection was used as training data. Initially, image optimization was the contrast 10 setting for evaluating the third molar, and a standard rectangular box was positioned all around 11 it is using Adobe Photoshop CC 2017. This narrow box shows the area for the next step. Then, 12 the machine learning algorithms are utilized to find the automatic phase. The classification 13 presentation was then measured in a 5-fold reliability situation using several credibility criteria 14 (accuracy, degree-N detection speed, mean absolute difference, kappa linear coefficient). The 15 mean accuracy was 51.0, the average unconditional disagreement was 0.6, and the kappa 16 average was 0.82 in (+_ 1 Year). 17 Age of majority is the threshold of adulthood that is recognized or declared in the law. Most younger. The word majority in this context refers to older years and having a full age versus a 20 minority, at is, being a minor. When minors are considered as an adult, the legal control and 21 responsibilities of their parents or guardians over them will end (Sharifonnasabi, 2020). Minors 22 may be legally exempted from certain privileges such as voting, buying, and drinking alcohol, 23 buying tobacco or cannabis products, gambling, marriage, buying or owning a firearm, owning 24 property, or getting married, migration, adult sports, or get full driving privileges (Kruessmann,25 2021; Sharifonnasabi, 2020). 26 Considering the multiple related issues in the age of majority, which is the part of essential 27 matter in legal and forensic and sports organizations, estimating age in forensic medicine and 28 clinical dentistry is particularly important. Age estimation based on tooth improvement is a 29 consistent method as tooth bones are immortals, and 18 to 20 years, the only skeletal 30 components visible in the macroscopic of these facial bones are teeth through post-mortem 31 decomposition (Blatt, 2020). As a result, the tooth and jawbone are selected to develop the 32 hybrid method because it is a helpful bone for age measurement amongst parts of the forensic 33 association. It becomes essential that a predictive model to measure bone age is identified. 34 Modelling an accurate method for measuring age in the medical field can help accelerate 35 personal identification and thus reduce costs. Although the number of bone age measurement 36 (BAM) models has increased, most remain in the testing phase because precise results have not 37 been obtained, especially in predicting the minor's age. Furthermore, there are no robust 38 computerized methods for BAM within the healthcare setting, with a precision to the range of 39 +_ six months for dental age estimation techniques due to image analysis and image processing 40 techniques limitations (Sharifonnasabi, 2020).  Step 1: Image Pre-Processing and Data Augmentation 8 For the better resolution of OPG x-ray images, a series of pre-processing and normalization 9 operations are used. Initially, the images were removed from the extra margins and cropped. 10 The image is then adjusted to increase the contrast of the images and highlight the images' edges. It is also done to normalize the image with zero base and variance 1. A lot of data is also 1 needed from OPG x-ray images to train CNN better. If numerous images are not available, data 2 augmentation is used. In this study, data mirroring and data duplication have been used to 3 increase the number of samples.

4
Step 2: Split Dataset to Train, Valid and Test Dataset as Cross-Validation Step 5 To perform the classification, the data in the dataset are divided into two categories: train data 6 and validation data. This classification has been done in the current research, including cross- various test data will indicate the correct performance of the proposed model.

13
Step 3: Hybrid HCNN-KNN 14 The architecture of the proposed HCNN-KNN model is considered in this research according  Similarly, to the conv1 layer, the output of conv2 will be 56 × 56 × 32. Then fully connected 31 layer is used to connect all the neurons in the layers. Therefore, Table 1 shows the parameters  is assumed to be 500.
11 Figure 3 shows the HCNN-KNN model algorithm pseudo-code proposed in the bone age 12 measurement problem based on dental images. K are 1, 2, 3 in KNN, and the distance metric 13 is Euclidean distance. The model is trained using an SGD momentum optimizer with a 0.0001 14 learning rate. The model is then trained for ten epochs with a mini-batch size of 16 images.

15
Finally, after learning the network and classification operations, the error loss rate is obtained 16 to evaluate the proposed network model in subsequent epochs of network learning.  Step 5: Validating the Proposed Model on Second Dataset (Test Dataset) 28 As mentioned in step 3, the test data is new (not the part of training dataset) and is collected 29 separately from the dataset with different properties. This data combines X-ray OPG images of

38
In the following, first, the database used in this article is introduced, then the tools and the 39 framework are introduced. Finally, we evaluated our proposed model on the dataset.

A. Datasets Description
Dental OPG is a panoramic radiograph that scans the upper and lower jaws with a two-1 dimensional view that shows a semicircle from ear to ear. The method presented in this article 2 is based on dental OPG data collected from dental teaching institutes and private dental clinics 3 in Malaysia. These collected data were with the different image sizes are resized before 4 importing into the proposed model. The original image size for all data has been changed to 5 600 × 1024 pixels. Subjects from this study were randomly selected from 1922 patients   The collected dataset shows in Figure 4 has a prominent feature as it contains the addition of 14 the categorization of data in the form of age range by year, age range by month, by season (3 15 months), and six months (Dataset on Year, on ± six months, on ± three months, and ± one 16 month). This classification allows us to evaluate our proposed method on a test database to 17 examine the performance of the proposed method. Figure 5 shows the frequency of data in the 18 database based on each of the Dataset on Year, on ± six months, on ± three months, and ± one 19 month. As previously mentioned, the model introduced for bone age measurement in this study is based 3 on CNN. Since deep and CNN models need huge data for training, data augmentation methods 4 such as mirroring, and duplication are used to increase the data. The results of data 5 augmentation and data non-augmentation are shown in the experimental section. This study used a machine with NVIDIA GeForce GTX 1080 8GB, 8GB memory, Intel Core 8 i7, and 3.40GHz. We also used libraries NumPy, Matplotlib, Sklearn. Metrics, PyTorch, 9 Torchvision to perform the analyses.  In the first step, the accuracy value for training and valid data is evaluated from the original 3 dataset without any pre-processing and data augmentation. Subsequently, the accuracy value 4 for training and validation dataset from the database is evaluated by pre-processing operations 5 on the original dataset and data augmentation operations.  The consistency value on data conducted with pre-processing and data augmentation operations 10 has given more suitable performance as shown in Table 1. Figure 6 displays the loss reduction  Figure 7 shows the accuracy of the proposed model on train and validation data in 10 epochs. 4 As can be seen, the accuracy of train data is generally slightly higher than validation data.

10
Therefore, the upward trend inaccuracy at ten epochs indicates an increase in CNN learning. As mentioned in the proposed model section, the combined methodology of convolutional 16 neural networks and the nearest neighbour (classification) is used in this study. In this proposed hybrid model (HCNN-KNN), the input data passes through the three main layers of CNN, 1 PCA, and KNN methods, respectively, and is finally categorized by KNN.

2
The results show that the KNN classification with 500 features selected from 'FullyPCA' (PCA 3 data transform after fully connected in CNN) has an acceptable outcome. As shown in Table   4 3, the accuracy on the dataset on Year, dataset on ± 6-months, dataset on ± 3-months, and 5 dataset on ± 1-month are equivalent to 99.98, 99.96, 99.87, and 98.78, respectively and, the 6 accuracy for train data is equal to 100. This means that the network is fully trained on train 7 data. It is noteworthy that the results obtained for k = 1 in the proposed HCNN-KNN hybrid 8 algorithm. 9 In this experiment, the HCNN-KNN model was evaluated for bone age measurement. 10 11 Figure 8 shows the accuracy and error loss in the combined HCNN-KNN model for k = 1 and 14 10 epochs on the train and valid data.  To evaluate the KNN method in the proposed HCNN-KNN hybrid model, k equivalent 2 and 1 3 have been used in evaluating the proposed model. Table 4 shows the accuracy results of k = 2 1, 2, 3 on the validation data.  In analysing the value of k in the nearest neighbourhood and avoiding overfitting for small ks 7 such as k = 1 and underfitting for large k, use new dental images separate from the data in the  Table 5 show 15 the evaluation of accuracy on different cross-validation on the dataset. 16 17 The key feature of this experiment is the validation of the proposed model. with other conditions than ordinary dentistry (orthodontic and malignant images). This 7 statistical population includes 130 new dental X-Ray OPG collected to solve the dental age 8 accuracy problem. Figure 9 shows a sample of data outside the first dataset and effectively 9 improves the age estimation accuracy even in malignant dental images of the proposed model. 12 13 As  According to Table 6, The results of this experiment have been evaluated on a test dataset with 3 the form of different races. HCNN-KNN effects have also been tested for k 1, 2, and 3. The   Finally, an attempt was made in the last experiment to compare the results obtained in this 2 study with the currently existing models. 3 4  The collected data along with the proposed model have been tested with other models such as 9 ResNet, CNN, GoogLeNet Inception. As Table 7 shows, the hybrid HCNN-KNN model 10 (HCNN-KNN) has obtained much better accuracy compared to other classification algorithms.

11
No reported research has been done on ± six months, on ± three months, and ± one month so Perceptron model with 1315 numbers of dental OPG images, and the highest accuracy was 15 90% just for accuracy of the years. Also, in 2020, Hussein used the SVM algorithm for a 16 population of 1429 dental radiographs, and they only achieved 92% accuracy for years. 17 Therefore, based on Table 7, the results of the comparative studies for accuracy by years are 18 still below the accuracy achieved in this study which is 99.98. also showed that the HCNN-KNN model is the best model for bone age measurement. 14 15 Conflict of interests 16 The authors declare no competing financial and non-financial interests.

18
Acknowledgment 19 We are thankful to the school of computer science and engineering, SCE Taylor's University, 20 for providing the scholarship to complete the studies. HAR is supported by the UNSW Scientia