Using Machine Learning to Identify the Risk Factors of Pancreatic Cancer from the NIH PLCO Dataset

1. Introduction

Pancreatic cancer (PC) is a disease with poor prognosis and survival rate. About 95% of people who contract PC would not make it to the five-year survival period [1]. Pancreas is an inner organ of the human body, surrounded by the duodenum and the small intestine; hence early symptoms are hard to detect [2]. Malicious cells in the pancreas are typically detected at a very advanced stage when it is impossible to save the patient. There is a pertinent need for a prediction model that can lead to early detection of this disease.

Biomarkers for early diagnosis of PC have been investigated (see for example, [3,4,5,6,7,8]). However, evidence for identified biomarkers has not been very conclusive. Image analysis and machine learning algorithms have been used for distinguishing between benign and malignant tissues in endoscopic ultrasound and computed tomography images (see for example, [9,10,11,12]). However, these models can detect PC only at an advanced stage and hence are not very useful.

The purpose of this study is to identify a set of factors as predictors of PC. We use a cancer dataset collected from 154,897 participants, each responding to 65 questions (or factors) about demographics, cancer and health history, medication usage, and smoking habits. There are two challenges to selecting the subset of 65 factors that predict PC with highest probability: the problem is computationally intractable, and the dataset is highly imbalanced. Our approach consists of balancing and preprocessing the dataset, and rank the risk factors based on their ability to predict PC.

Our study found that risk factors belonging to cancer and health history are the most prominent ones for PC. In particular, previously diagnosed with PC is chosen as the most prominent risk factor by majority of methods. We also discuss findings in the medical sciences literature that corroborate our findings. Some of our findings shed light on novel factors that may not have received their due attention by the research community.

2. Models and Methods

2.1. Problem Statement

Our problem is to predict whether a subject is diagnosed with PC or not, given information about his demographic characteristics, health history, medication usage, smoking habits, and his and his family’s cancer diagnosis history. This information is encoded as a vector of predictor variables where each predictor represents a risk factor (a.k.a. feature). The predictors are discrete and finite random variables.

Formally, given a set of data points $X=[x_1,...,x_N]\in {\mathbb{N}}^{d\times N}$, $\mathbb{N}=\{0,1,2,...\}$, and a set of labels $\{True,False\}$, the task is to map each data point $x_i\in {\mathbb{N}}^d$ into one of the labels, where d is the dimension of each data point, and N is the number of data points in the dataset. This is a binary classification problem. Our goal is to select a subset of predictors such that classification using the subset is at least as accurate as that using the entire set. It has been shown that the accuracy does not always improve with increase in number of variables [13], hence choosing the optimal subset of predictors is imperative for accurate prediction of PC.

2.2. Materials

The Prostate, Lung, Colorectal and Ovarian (PLCO) cancer dataset [14] is collected by the National Cancer Institute from 154,897 participants. Among them, 76,678 or 49.5% were males, and 132,572 or 85.6% were non-Hispanic White. The participants, randomly selected based on a set of criteria from different parts of United States, were between 55-74 years of age with no history of prostate, lung, colorectal or ovarian cancer. Each participant filled out three questionnaires, thereby responding to 65 questions about demographics, cancer and health history, medication usage, and smoking habits. Therefore, $N=154,897$ and $d=65$ for our problem. The dataset is highly imbalanced; only 749 or 0.48% of the participants were diagnosed with pancreatic cancer. Visualizations of the PLCO dataset in two-dimensional (2D) space are shown in Figure 1. Table 1 lists the 65 risk factors.

2.3. Dataset Balancing

A balanced dataset contains equal number of data points in all classes. Usually, an imbalanced dataset is balanced using methods such as fixed-rate downsampling or clustering that downsample the majority subset, or using methods such as the SMOTE algorithm [17] that upsample the minority subset. Both approaches inherit drawbacks unless the true distribution generating the data is known. The true distribution is unknown for the current problem.

We use a balancing method, similar to that proposed in [18], whereby the majority subset is iteratively and randomly subsampled such that in each iteration, the sampled subset is balanced. This method refrains from eliminating any data point from or introducing any new data point into the given dataset. A feature selection method is applied independently on each subset. The final result is obtained by computing the mean over all the subsets.

2.4. Data Preprocessing

The PLCO dataset has a number of missing values. We employ two steps iteratively to obtain a less incomplete dataset. First, we eliminate factors that are either missing responses from more than 10% of the participants, or responses from all participants are same. Next, we eliminate participants who did not respond to more than 10% of the remaining factors. The two steps are again applied to the resulting dataset. Application of the two steps continues until there is no change in the dataset between two consecutive iterations.

Each feature is standardized by subtracting its mean and dividing by its standard deviation. The missing values in the resulting dataset are filled in. The $j^{\mathrm{th}}$ element of the $i^{\mathrm{th}}$ data point, if missing, is filled by:

$$\begin{array}{cccc}\hfill & {\widehat{x}}_{ij}=\sum _{\begin{array}{c}k=1\\ k\ne i\end{array}}^N{x}_{kj}dist(x_i,x_k)/\sum _{\begin{array}{c}k=1\\ k\ne i\end{array}}^{N}dist(x_i,x_k)\hfill & & \\ \hfill \mathrm{where}& dist(x_i,x_k)=\frac{|x_i·x_k|}{\parallel x_i\parallel \parallel x_k\parallel },x_i·x_k=\sum _{\begin{array}{c}m=1\\ m\mathrm{not}\mathrm{missing}\end{array}}^d{x}_{im}{x}_{km},\parallel x_i\parallel =\sqrt{{\sum }_{\begin{array}{c}m=1\\ m\mathrm{not}\mathrm{missing}\end{array}}^d{x}_{im}^2},\hfill & & \hfill \end{array}$$

(1)

$|.|$ denotes the absolute value, “$m\mathrm{not}\mathrm{missing}$” refers to the $m^{\mathrm{th}}$ element of a data point that is not missing, and $dist$ is the absolute of the cosine similarity (or normalized dot product) of two data points. Therefore, $0\le dist\le 1$; as two data points get closer, their $dist$ increases. In Eq. 1, a missing element of a given data point is computed as the weighted mean of that element from all data points in which values of all elements are present, and the weights are proportional to the absolute cosine similarity. After filling in all missing values, each feature is standardized again.

2.5. Variable or Feature Selection

Our problem of selecting the optimal subset of features is intractable as a total of ${\sum }_{n=1}^d\left(\genfrac{}{}{0pt}{}{d}{n}\right)\in \mathcal{O}\left(2^d\right)$ subsets are possible. Computing $\mathcal{O}\left(2^d\right)$ subsets to determine the optimal one is impractical for the PLCO dataset with $d=65$. Hence we resort to variable or feature selection methods [19,20].

We used several feature selection algorithms suitable for categorical and continuous features and classification task [21,22,23,24,25,26,27,28,29,30,31], implemented in MATLAB, to rank the features, such as rank features using chi-square tests (`fscchi2’ in MATLAB), rank features for classification using minimum redundancy maximum relevance (MRMR) algorithm (`fscmrmr’ in MATLAB), estimate predictor importance for classification using ensemble of decision trees (`fitcensemble’ in MATLAB), estimate predictor importance for classification using a binary decision tree (`fitctree’ in MATLAB), estimate predictor importance for classification with an ensemble of bagged decision trees (e.g., random forest) which assigns positive and negative scores to the predictors (`fitcensemble’ with method `bag’, and `oobPermutedPredictorImportance’ in MATLAB), rank key features by class separability criteria (`rankfeatures’ with criteria `ttest’, `entropy’, `bhattacharyya’, `roc’, and `wilcoxon’ in MATLAB), and Pearson correlation between each feature/predictor variable and response variable (`corrcoef’ in MATLAB) with correlation set to zero if not significant (i.e. $p>0.01$). Figure 2, Figure 2 show the ranking of the features by each of these algorithms for males and females respectively. A brief description of each of these algorithms is presented in Appendix.

3. Experimental Results

Our analysis is done on the entire dataset as well as separately on the male and female participants. After preprocessing (ref. Section 2.4), the PLCO dataset containing 65 features and 154897 points (76682 male, 749 $True$, 430 male $True$) reduces to 32 features and 148315 points (73162 male, 706 $True$, 405 male $True$). For balancing (ref. Section 2.3), we randomly sample $\lfloor 148315/706\rfloor =210$ non-overlapping subsets for PC=$False$, each containing 706 or 707 data points. Thus after balancing, each subset contains a total of 1412 or 1413 points. Similarly, for male only analysis, we obtain $\lfloor 73162/405\rfloor =180$ balanced subsets, each containing 810 or 811 points. For female only analysis, we obtain $\lfloor 75153/301\rfloor =249$ balanced subsets, each containing 602 or 603 points.

3.1. Classification

Twenty four machine learning algorithms, briefly described in Section 6.4 were used and their statistical parameters are reported.

Using classification ensemble. In this ensemble algorithm, the weights or costs can be modified to correctly train the algorithm to predict PC. The weights are normalized to add unity, depicting the prior probabilities. Suppose ${\in }_{ij}(i,j\in \left\{1...c\right\},{\in }_{ii}=0)$ is the cost of misclassification of the example of the $i^{th}$ class to the $j^{th}$ class, where c is the number of classes. Then, the weight assigned to the $i^{th}$ class after rescaling is given as [32]:

$$w_i=\frac{nx{\in }_i}{{\sum }_{k=1}^c\left(n_{k}x{\in }_k\right)}$$

(2)

where n is the number of training samples, ${\in }_i={\sum }_{j=1}^c{\in }_{ij}$.

It uses the algorithms as described in [32,33,34]. For example, we can say the weight of predicting no PC for subjects with PC(False positive) is 1000 times more serious than predicting PC for subjects with no PC (False negative). Accordingly, we can change the weights to get a confusion matrix as per our need.

3.2. Feature selection.

We used several feature selection algorithms [21,22,23,24,25,26,27,28,29,30,31], implemented in MATLAB, to rank the features. Figure 2, Figure 2 show the ranking of the features by each of these algorithms for males and females respectively.

3.3. Finding probability feature combination using a Bayesian Network

Russell and Norvig in their book, Artificial Intelligence: A Modern Approach [35] have illustrated about Bayes Theorem and joint probability. Consider that the symptoms $E_1,E_2$ are conditionally independent. Then their co-occurrence is as follows:

$$P(E_1,E_2|C)=P\left(E_1\right|C)P\left(E_2\right|C)$$

(3)

Using the above equation,

$$P\left(C\right|E_1,E_2)=\frac{P\left(C\right)P(E_1,E_2|C)}{P\left(E_1\Delta E_2\right)}$$

(4)

As any individual will either have PC or not have PC with the given symptoms, considering a universal set , $P\left(E1\Delta E2\right)$ can be resolved using normalization as follows:

$$P\left(C\right|E_1,E_2)+P\left(\overline{C}\right|E_1,E_2)=1$$

(5)

$P\left(\overline{C}\right)$ is the probability of non-occurrence of PC. Hence,

$$P\left(E_1\Delta E_2\right)=P\left(C\right)P(E_1,E_2|C)+P\left(\overline{C}\right)P(E_1,E_2|\overline{C})$$

(6)

Substituting equation 3 in equation 6

$$P\left(E_1\Delta E_2\right)=P\left(C\right)P\left(E_1\right|C)P\left(E_2\right|C)+P\left(\overline{C}\right)P\left(E_1\right|\overline{C})P\left(E_2\right|\overline{C})$$

(7)

Substituting equation 4 in equation 7,

$$P\left(C\right|E_1,E_2)=\frac{P\left(C\right)P(E_1,E_2|C)}{P\left(C\right)P(E_1,E_2|C)+P\left(\overline{C}\right)P(E_1,E_2|\overline{C})}$$

(8)

Substituting equation 3 in equation 8,

$$P\left(C\right|E_1,E_2)=\frac{P\left(C\right)P\left(E_1\right|C)P\left(E_2\right|C)}{P\left(C\right)P\left(E_1\right|C)P\left(E_2\right|C)+P\left(\overline{C}\right)P\left(E_1\right|\overline{C})P\left(E_2\right|\overline{C})}$$

(9)

4. Discussions

A number of risk factors of PC have been identified [36,37,38], such as, smoking, obesity, exposure to certain chemicals (e.g., pesticides, benzene, certain dyes, petrochemicals), age (older than 55 years), gender (male), race/ethnicity (Blacks, Ashkenazi Jewish heritage), family history (two or more first-degree relatives with PC), inherited genetic syndromes, diabetes, pancreatic cysts and chronic pancreatitis. Several different genes are associated with increased risk of PC. However, genetic risk factors are beyond the scope of this work as our dataset does not contain genetic information. Table 3 shows the ratio in which each symptom was distributed in the HPT(high probability table) chosen by selecting top percentage values of probability for that feature combination and in the LPT(low probability table) chosen by selecting bottom percentage values of probability for that feature combination.

Factors with unclear effect on risk include nature of diet, lack of physical activity, coffee and alcohol consumption, and certain infections (see for example, [38]) .

Smoking. Several studies have shown that smoking has a significant relationship with PC (see for example [39,40,41,42,43,44]). Yadav et al. [43] found that smoking cessation can significantly reduce risk of PC. Raimondi et al. [44] argue that smoking is the most common risk factor and accounts for 20-25% of all pancreatic tumors.

Diabetes. Diabetes also has a positive correlation with PC [45]. Huxley et al. [3] shows that individuals who have had type-II diabetes for less than four years were at a 50% higher risk of contracting PC than individuals who have had type-II diabetes for more than four years. Everhart et al. [4] have concluded that subjects with long standing diabetes have a higher relative risk of PC. Ben et al. [5] have also found similar relationship between diabetes and PC. Liao et al [46] shows that subjects in Taiwan who have had diabetes for less than 2 years are at elevated risk of PC. Long standing diabetes did not pose a strong risk. Also concurrent occurrence of diabetes and chronic pancreatitis puts subjects at a higher risk.

Reproductive history in women. Lo et al. [47] have shown that women with 7 or more live births had a lower risk of PC. Lactation period also had a significant effect on the possibility of PC. This study shows that women who lactated for 144 months or more had a one-fifth the risk of PC than women who lactated only for 89 months or less. Kreiger et al. [48] have shown that PC is an estrogen-dependent disease and aspects of reproductive history and hormone replacement are associated with a greater risk of this disease. Reduced risks were observed with 3 or more pregnancies and with the use of oral contraceptives.

Marriage. Baine et al. [31] shows marriages improves the survival rate and longevity of patients with PC. This paper also shows using Kaplan-Meier analysis, that patients who were married had a median survival rate of 4 months in comparison to unmarried patients who had a survival rate of 3 months. Aizer et al. [49] have shown that marriage has a beneficial effect on any cancer with regards to detection, treatment and survival. This improvement was observed more in males than females, highlighting the socio-economic elevation that a married person could have. The paper concluded that ”married people were less likely to present metastatic disease, more likely to receive definitive therapy, and less likely to die as a result of their cancer after adjusting for demographics, stage, and treatment than unmarried patients.” Multivariate logistic and Cox regression were used to analyze the patients.

Occupation. Logan et al. [50] shows how specific types of occupation pose higher risk to exposure to carcinogenic substances.In 1961 and 1971, for men, occupation categories of clothing, food, drink and tobacco and armed forces had higher standardised mortality rates(SMR) and relative standardised mortality rates(RSMR) whereas people in the clerical and leather industry saw low SMR and RSMR. For men in the occupation categories of mining, labourers and service, sport and recreation saw elevated but reduced RSMR. For men in administrative and managerial, and professional and technical disciplines, the trend was reduced SMR and elevated RSMR.In case of married women, if husbands worked in engineering, leather, wood, sales, clothing, construction work, both SMR and RSMR were high. For wives of husbands working in farm, gas, coke and chemicals industry, glass and ceramics and warehouse, both SMR and RSMR were low. In 1961, wives of husband in food, drink and tobacco had high SMR and RSMR and values were low for husbands in painting and decoration industry, and the trend was reversed in 1971.

Family composition. Gharidian et al. [51] have found an interesting relationship wherein there is the occurrence of this disease in two brothers and one sister in all the seventh decade of their life. This study was based in Montreal and there was no pancreatitis history between the patients or their relatives.

Use of certain medications. Tan et al. [52] have shown that aspirin use decreases risk of procuring PC. Aspirin use for 1day/month or greater was associated with a lower risk of PC than subjects who had aspirin for less than 1day/month. According to this study, there is no relationships between non-aspirin non-steroidal anti-inflammatory drugs (NSAID) and PC. Larsson et al. [53] have provided a doubtful evidence that regular use of aspirin over longer duration increases risk of PC. No relationship was found between use of frequent aspirin (7 tablets or more/week) or prolonged use of aspirin (more than 20 years) and the increase/decrease in PC. Harris et al. [54] have found a relationship between aspirin, ibuprofen, and other non-steroidal anti-inflammatory drugs (NSAID) and cancer prevention. However, results varied for different types of cancer.

Surgical history. Rosenberg et al. [55] have shown a positive correlation in increase in risk of PC by 1.8% because of vasectomy.

Inherited genetic syndrome. Certain rare genetic conditions causes almost 10 percent of all PCs. In our investigation, it can be found under family history of PC , that have been chosen by two of the feature-selection algorithms, viz, Relieff and Lasso in Table 3. Also, no of relatives with PC has been chosen by 4 of the feature selection algorithms, viz, ECFS, UDFS, LLCFS and CFS. From the graphs in Figure 3, it can be seen that if subject has family history of PC or any form of cancer, there is increase in probability of PC. Further the trend of increase is almost exponential as no. of relatives with PC increases, which strongly suggests that genetics play an important role in determination of possibility of PC. Such rare genetic conditions include [36]:

• Hereditary breast and ovarian cancer syndrome, caused by mutations in the BRCA1 or BRCA2 genes,
• Hereditary breast cancer, caused by mutations in the PALB2 gene,
• Familial atypical multiple mole melanoma (FAMMM) syndrome, caused by mutations in the p16/CDKN2A gene and associated with skin and eye melanomas,
• Familial pancreatitis, usually caused by mutations in the PRSS1 gene,
• Lynch syndrome, also known as hereditary non-polyposis colorectal cancer (HNPCC), most often caused by a defect in the MLH1 or MSH2 genes,
• Peutz-Jeghers syndrome, caused by defects in the STK11 gene. This syndrome is also linked with polyps in the digestive tract and several other cancers.

Race. Race has been a predominant factor in the determination of the risk of PC [36,37,38]. According to literature, blacks or African American people have a higher risk of contracting PC. This could be attributed to their dietary habits or smoking history. Race has been chosen as one of the features by 3 of our feature-selection algorithms, viz, Laplacian, FSASL and LLCFS in Table 2 and also Black race has been chosen as one of the highest probability of PC causing feature in Table 2.

Gender. Literature has shown that men are more likely to contract PC than women [36,37,38]. This could be because men are more likely to smoke than women and smoking has a significant effect on PC. Gender has been chosen as one of the features by 3 of our feature-selection algorithms, viz, Laplacian, CFS and ECFS in Table 2 and also gender is male in the highest probability of PC in Table 2.

Female hormones. Experimental findings from this article by [56] on use of affect of female hormones suggest that female hormones have a protective role towards incidence of PC.

Bronchitis. Although there is no direct evidence between bronchitis and PC risk, [57] is a study conducted on male smokers in Finland that suggests that bronchial asthma predict the subsequent risk of developing PC in male smokers and that greater physical activity may decrease the risk. Also bronchial asthma can increase chances of developing bronchitis.

Heart attack.Many references suggest the increased association between heart attack and stroke with any type of cancer (not necessarily PC). [58] shows the increased risk of heart attack and stroke in the months leading up to cancer diagnosis. In another article [59], it shows that recent epidemiological analyses suggest that cancer incidence is more common among subjects with a history of heart failure versus subjects with no history of heart failure.

Hypertension. Some references, for example [60] suggest that hypertension at baseline was associated with an increased risk of PC incidence.

Although the above factors-inherited genetic syndrome, race, diabetes history and gender have a strong relationship with PC, yet they were not one of the highly selected features by our algorithms, probably because other features have a stronger dependence when considered in unison.

Most of the remaining features as seen in Table 2 do not have a strong evidence yet to their dependency with PC, however they can act as a guide to biologists and researchers to delve into possible correlation between these symptoms.

Table 4. Table containing 2 features combinations from PLCO dataset that produces highest risk of PC for female.

Symptom 1	Symptom 2	Probability
No of cigarettes smoked daily= 61-80	No of relatives with PC=2+	0.156
No of tubal/ectopic pregnancies=2+	No of relatives with PC=2+	0.137
Usually filtered or not filtered?=Both	No of relatives with PC=2+	0.115
No of cigarettes smoked daily= 61-80	No of relatives with PC=2+	0.095
No of tubal/ectopic pregnancies=1	No of relatives with PC=2+	0.084
Heart attack history?=yes	No of relatives with PC=2+	0.08
No of cigarettes smoked daily=21-30	No of relatives with PC=2+	0.077
No of relatives with PC=2+	Race=Asian	0.076
No of relatives with PC=2+	No of still births=1	0.074
No of relatives with PC=2+	Diabetes history?=Yes	0.0737
No of relatives with PC=2+	Race=American Indian	0.0737
No of relatives with PC=2+	Emphysema history?=Yes	0.0737
No of relatives with PC=2+	No of cigarettes smoked daily=31-40	0.0708
No of relatives with PC=2+	Colorectal Polyps history?=Yes	0.0708
No of relatives with PC=2+	Stroke history?=Yes	0.0704
No of relatives with PC=2+	Age at hysterectomy=40-44	0.0686
No of relatives with PC=2+	No. of brothers=7+	0.0645
No of relatives with PC=2+	Bronchitis history?=2+	0.064
No of relatives with PC=2+	Liver comorbidities history?=Yes	0.063
No of relatives with PC=2+	No of cigarettes smoked daily=11-20	0.063

Table 4. Table containing 2 features combinations from PLCO dataset that produces highest risk of PC for female.

Symptom 1	Symptom 2	Probability
No of cigarettes smoked daily= 61-80	No of relatives with PC=2+	0.156
No of tubal/ectopic pregnancies=2+	No of relatives with PC=2+	0.137
Usually filtered or not filtered?=Both	No of relatives with PC=2+	0.115
No of cigarettes smoked daily= 61-80	No of relatives with PC=2+	0.095
No of tubal/ectopic pregnancies=1	No of relatives with PC=2+	0.084
Heart attack history?=yes	No of relatives with PC=2+	0.08
No of cigarettes smoked daily=21-30	No of relatives with PC=2+	0.077
No of relatives with PC=2+	Race=Asian	0.076
No of relatives with PC=2+	No of still births=1	0.074
No of relatives with PC=2+	Diabetes history?=Yes	0.0737
No of relatives with PC=2+	Race=American Indian	0.0737
No of relatives with PC=2+	Emphysema history?=Yes	0.0737
No of relatives with PC=2+	No of cigarettes smoked daily=31-40	0.0708
No of relatives with PC=2+	Colorectal Polyps history?=Yes	0.0708
No of relatives with PC=2+	Stroke history?=Yes	0.0704
No of relatives with PC=2+	Age at hysterectomy=40-44	0.0686
No of relatives with PC=2+	No. of brothers=7+	0.0645
No of relatives with PC=2+	Bronchitis history?=2+	0.064
No of relatives with PC=2+	Liver comorbidities history?=Yes	0.063
No of relatives with PC=2+	No of cigarettes smoked daily=11-20	0.063

Table 5. Table containing 3 features combinations from PLCO dataset that produces highest risk of PC for male and female.

Male
Symptom 1 conditional probability	Symptom 2 conditional probability	Symptom 3 conditional probability	Total probability
No of cigarettes smoked daily is 61-80=0.005	Age when told had enlarged prostate is 70+ =0.0175	prior history of cancer is yes=0.05	0.00521
No of cigarettes smoked daily is 61-80=0.005	Age when told had enlarged prostate is 70+ =0.0175	family history of PC=yes=0.533	0.002368
No of cigarettes smoked daily is 61-80=0.005	Age when told had enlarged prostate is 70+ =0.0175	no. of relatives with PC is 1=0.04	0.004593
prior history of cancer is yes=0.05	Age when told had enlarged prostate is 70+ =0.0175	family history of PC is yes=0.533	0.002153
prior history of cancer is yes=0.05	Age when told had enlarged prostate is 70+ =0.0175	no. of relatives with PC is 1=0.04	0.004177
Age when told had enlarged prostate is 70+ =0.0175	family history of PC is yes=0.533	no. of relatives with PC is 1=0.04	0.00189
No of cigarettes smoked daily is 61-80=0.005	prior history of cancer is yes=0.05	family history of PC=yes=0.533	0.02742
No of cigarettes smoked daily is 61-80=0.005	prior history of cancer is yes=0.05	no. of relatives with PC is 1=0.04	0.05197055
No of cigarettes smoked daily is 61-80=0.005	family history of PC is yes=0.533	no. of relatives with PC is 1=0.04	0.024218
prior history of cancer is yes=0.05	family history of PC is yes=0.533	no. of relatives with PC is 1=0.04	0.02206
Female
No of tubal/ectopic pregnancies is 1=0.003	No. of relatives with PC is 2+=0.011	no. of cigarettes smoked is 61-80=0.007	0.3578

Table 5. Table containing 3 features combinations from PLCO dataset that produces highest risk of PC for male and female.

Male
Symptom 1 conditional probability	Symptom 2 conditional probability	Symptom 3 conditional probability	Total probability
No of cigarettes smoked daily is 61-80=0.005	Age when told had enlarged prostate is 70+ =0.0175	prior history of cancer is yes=0.05	0.00521
No of cigarettes smoked daily is 61-80=0.005	Age when told had enlarged prostate is 70+ =0.0175	family history of PC=yes=0.533	0.002368
No of cigarettes smoked daily is 61-80=0.005	Age when told had enlarged prostate is 70+ =0.0175	no. of relatives with PC is 1=0.04	0.004593
prior history of cancer is yes=0.05	Age when told had enlarged prostate is 70+ =0.0175	family history of PC is yes=0.533	0.002153
prior history of cancer is yes=0.05	Age when told had enlarged prostate is 70+ =0.0175	no. of relatives with PC is 1=0.04	0.004177
Age when told had enlarged prostate is 70+ =0.0175	family history of PC is yes=0.533	no. of relatives with PC is 1=0.04	0.00189
No of cigarettes smoked daily is 61-80=0.005	prior history of cancer is yes=0.05	family history of PC=yes=0.533	0.02742
No of cigarettes smoked daily is 61-80=0.005	prior history of cancer is yes=0.05	no. of relatives with PC is 1=0.04	0.05197055
No of cigarettes smoked daily is 61-80=0.005	family history of PC is yes=0.533	no. of relatives with PC is 1=0.04	0.024218
prior history of cancer is yes=0.05	family history of PC is yes=0.533	no. of relatives with PC is 1=0.04	0.02206
Female
No of tubal/ectopic pregnancies is 1=0.003	No. of relatives with PC is 2+=0.011	no. of cigarettes smoked is 61-80=0.007	0.3578

5. Conclusion

We have used widely used algorithms for our prediction for PC. Since the exact relationship between features and the cause of PC cannot be ascertained for sure, for example, some factors like education, marital status and several others could have an indirect causal relationship with this disease, hence these factors were not excluded from our prediction study. After running all the above algorithms, it is observed that k-means clustering and SMOTE method of oversampling are some of the superior algorithms for PC prediction. The artificial intelligence based Bayesian network prediction model can signify which individuals are at an elevated risk for PC.

Until now, very limited work has been done in PC prediction, so the accuracy obtained by our research is significant. Lack of online available datasets for PC have limited the work that can be done in this field. Still the PLCO dataset by NIH has been a very valuable resource. Future improvements can be made based on taking into account other features that would have been found as a possible precursor to PC, based on further research and availability of more datasets.

6.1. Data Visualization Methods

t-distributed stochastic neighbor embedding (t-SNE) algorithm. [61]The t-SNE method is a non-linear dimensionality reduction method, particularly well-suited for projecting high dimensional data onto low dimensional space for analysis and visualization purpose. Distinguished from other dimensionality reduction methods, the t-SNE method was designed to project high-dimensional data onto low-dimensional space with minimum structural information loss. So that the points close to each other on the low-dimensional surface represent states that are similar in the high-dimensional space.

As Van der Maaten and Hinton explained [16] “The similarity of datapoint $x_j$ to datapoint $x_i$ is the conditional probability $p_{j|i}$ , that $x_i$ would pick as its neighbor $x_j$ if neighbors were picked in proportion to their probability density under a Gaussian centered at $x_i$.”

This method [61] starts with converting the high-dimensional Euclidean distance between data points (the Cartesian coordinates of each frame in the simulation) into the conditional probability $p_j|i$. Given $x_i$ and $x_j$ as two data points representing two structures in Cartesian coordinate, the probability density distribution of its neighboring data points for $x_i$ is assumed as a Gaussian function centered at $x_i$ with variance ${\sigma }_i$. The probability of $x_j$ to be selected as the neighbor of $x_i$ is a conditional probability calculated as

$$p_{j|i}=\frac{exp(-\frac{{∥x_i-x_j∥}^2}{2{\sigma }^2})}{{\sum }_{k\ne i}exp(-\frac{{∥x_i-x_k∥}^2}{2{\sigma }^2})}$$

(10)

Adaptive Synthetic (ADASYN) algorithm. ADASYN is a method of generating synthetic examples for minority classes using a weighted distribution as shown in Figure 1a. The algorithm flowchart is described in detail in [15].

6.2. Data Balancing Methods

K-means clustering.k-means clustering performed on the majority class of the dataset yielded 743 cluster centers. The value of k is based on idea of equalizing minority class with majority class and generating 743 clusters for majority class. These points were mixed were the datapoints of the minority class to remove bias, and generate a total of 1486 datapoints. The 24 prediction algorithms were run on this new dataset and the results were validated using 5-fold cross validation.

Suppose we are given a dataset $X=x_1,...,x_N,x_n\in R^d$ as stated in [62]. The M-clustering problem aims at partitioning this data set into M disjoint subsets (clusters) $C_1,...,C_M$, such that a clustering criterion is optimized. The most widely used clustering criterion is the sum of the squared Euclidean distances between each data point $x_i$ and the centroid $m_k$ (cluster center) of the subset $C_k$ which contains $x_i$. This criterion is called clustering error and depends on the cluster centers $m_1,...,m_M$:

$$E(m_1,...,m_M)=\sum _{i=1}^N\sum _{k=1}^{M}I(x_i\in C_k){|x_i-m_k|}^2$$

(11)

where I(X)=1 if X is true and 0 otherwise

SMOTE method of oversampling.[17] The SMOTE (Synthetic Minority Over-Sampling Technique) function takes the feature vectors with dimension (r,n) and the target class with dimension (r,1) as the input and returns final features vectors with dimension (r’,n) and the target class with dimension (r’,1) as the output. The minority class was oversampled and the new dataset was run through the algorithms. The highest accuracy was given by Fine Decision Tree of 95.4%.

Downsampling method. [63] The majority class dataset can be downsampled by an integer sampling factor, n. It samples the dataset by keeping the first sample and then every nth sample after that. In case of several columns in the dataset, each column will be treated as a separate sequence. After feeding the downsampled dataset into 24 algorithms, the highest accuracy was reported by Quadratic SVM of 56.4% only.

y = downsample(x,n) [63] decreases the sample rate of x by keeping the first sample and then every nth sample after the first. If x is a matrix, the function treats each column as a separate sequence.

6.3. Feature Selection Methods

Chi-square tests(fscchi2). [63] fscchi2(Tbl,ResponseVarName) ranks features (predictors) using chi-square tests. The table Tbl contains predictor variables and a response variable , and ResponseVarName is the name of the response variable in Tbl. The function returns idx, which contains the indices of predictors ordered by predictor importance, meaning idx(1) is the index of the most important predictor. You can use idx to select important predictors for classification problems

As stated in [64], the χ2 test is applied to test the independence of two events, where two events A and B are defined to be independent if $P\left(AB\right)=P\left(A\right)P\left(B\right)$ or, equivalently, $P\left(A\right|B)=P(A)$ and $P\left(B\right|A)=P(B)$. In feature selection, the two events are occurrence of the term and occurrence of the class. We then rank terms with respect to the following quantity:

$${\chi }^2(D,t,c)=\sum _{e_t\in (0,1)}\sum _{e_c\in (0,1)}\frac{{(N_{e_t{e}_c}-E_{e_t{e}_c})}^2}{E_{e_t{e}_c}}$$

(12)

where,

N =observed frequency in D,

E=expected frequency,

$e_t=1$(the document contains term t) and $e_t=0$(the document does not contain term t),

C is a random variable that takes values $e_c=1$(the document is in class C) and $e_c=0$(the document is not in class c)

Minimum redundancy maximum relevance (MRMR)(fscmrmr) [65] This algorithm tends to select a subset of features having the most correlation with the class (output) and the least correlation between themselves. It ranks features according to the minimal-redundancy-maximal-relevance criterion which is based on mutual information. Minimal redundancy as stated in [66] will make the feature set a better representation of the entire dataset. Let S denote the subset of features we are seeking. The minimum redundancy condition is:

$$min{W}_I,W_I=\frac{1}{{\left|S\right|}^2}\sum _{i,j\in S}I(i,j)$$

(13)

where, $\left|S\right|$ is the number of features in S

Fitcensemble [63] fitcensemble(Tbl,ResponseVarName) returns the trained classification ensemble model object (Mdl) that contains the results of boosting 100 classification trees and the predictor and response data in the table Tbl. ResponseVarName is the name of the response variable in Tbl. By default, fitcensemble uses LogitBoost for binary classification and AdaBoostM2 for multiclass classification.

As stated in [67], let each (y, x) case in $\mathcal{L}$ be independently drawn from the probability distribution P. Suppose Y is numerical and $\varphi (x,\mathcal{L})$ the predictor. Then the aggregated predictor is the average over L of $\varphi (x,\mathcal{L})$, i.e.

$${\varphi }_A\left(x\right)=E_{\mathcal{L}}\varphi (x.\mathcal{L})$$

(14)

Fitctree [63,68] Estimate predictor importance for classification using a binary decision tree. fitctree(Tbl,VarName) returns a fitted binary classification decision tree based on the input variables (also known as predictors, features, or attributes) contained in the table Tbl and output (response or labels) contained in Tbl.VarName. The returned binary tree splits branching nodes based on the values of a column of Tbl.

We consider the problem of finding the best test for a nominal attribute with n values in a k-class L-ary decision tree as stated in [69]. We are particularly concerned with cases in which n or k, or both, are quite large; this situation arises in many pattern recognition problems and in some large real data mining applications. The problem is to find the optimal partition of n elements into L bins. A partition of the n distinct values of the attribute induces a partition of the examples: each example is put into the branch corresponding to the value that the given attribute takes for that example. The class impurity of the examples in each branch is computed, weighted, summed and assigned to the given partition. An n by k contingency matrix, computed at the start of the procedure, can be used to compute the impurity measure for each partition that is considered. The number of distinct partitions of n elements is exponential in n: for example, if L = 2, a binary tree, there are $2^{n-1}-1$ two-way partitions [69].

Rank key features by class separability criteria (rankfeatures)IDX = rankfeatures(X,GROUP) ranks the features in X using an independent evaluation criterion for binary classification. X is a matrix where every column is an observed vector and the number of rows corresponds to the original number of features. GROUP contains the class labels. IDX is a list of indices to the rows of X with the most significant features

The Bhattacharyya distance [70] is a common method for measuring the separation between two multivariate gaussians. Therefore, we will have to use this method based on the assumption that data is drawn from a Gaussian distribution. Because we have five classes in our data, we first estimate a Gaussian distribution from which the data is drawn, then we calculate all the possible combinations among the classes. Finally we add all this distances to produce a ranking for each feature. The Bhattacharyya distance for two multivariate distributions P1 and P2 can be calculated as follows:

$$BhatDistance(P1,P2)==(1/8){(m1-m2)}^T{P}^{-1}(m1-m2)+0.5ln\frac{detP}{\sqrt{detP1detP2}}$$

(15)

$$P=\frac{P1}{P1+P2}$$

(16)

The Wilcoxon signed rank test as stated in [71]is used to test that a distribution is symmetric about some hypothesized value, which is equivalent to the test for location. We illustrate with a test of a hypothesized median, which is performed as follows:

• Rank the magnitudes (absolute values) of the deviations of the observed values from the hypothesized median, adjusting for ties if they exist.
• Assign to each rank the sign (+ or - ) of the deviation (thus, the name “signed rank”).
• Compute the sum of positive ranks,T(+) , or negative ranks,T(-) , the choice depending on which is easier to calculate. The sum of T(+) and T(-) is n(n+1)/2, so either can be calculated from the other.
• Choose the smaller of T(+) and T(-), and call this T.
• Since the test statistic is the minimum of T(+) and T(-), the critical region consists of the left tail of the distribution, containing a probability of at most $\alpha /2$. If n is large, the sampling distribution of T is approximately normal with

  $$\mu =n(n+1)/4,and{\sigma }^2=n(n+1)(2n+1)/24$$

  (17)

  which can be used to compute a z-statistic for the hypothesis test.

In case ofentropy criterion, atypical [72] ranking process consists of four steps:

• Initialize set F to the whole set of p features. S is an empty set.
• For all features $f\in F$ compute J(f) coefficient.
• Find feature f that maximizes J(f) and move it to $S\leftarrow S\cup \left\{f\right\}$, $F\in F\left\{f\right\}$
• Repeat until the cardinal of S is p

where J(f) is a criterion function (specific for a given algorithm) which gives a measure of dependency between features (f) and classes (C). Feature (variable) selection problems can be formulated into a [73] cardinality optimization (In general, it is NP hard)

$$\beta =arg\underset{\beta \in R^p}{min}Q\left(\beta \right)=\sum _{i=1}^{n}q(X_i^T\beta ,y_i)s.t.{\beta }_0\le s$$

(18)

p is the number of features,

$X_i$, $\beta \in R^p$,

n is the number of training samples,

$X\in R^n\times p$, $y\in R^n$ denotes training data;

s is the given sparsity level,

Q(·) is convex and smooth[74].

PredictorImportance [63] Imp = oobPermutedPredictorImportance(Mdl) returns a vector of out-of-bag, predictor importance estimates by permutation using the random forest of regression trees Mdl.

A classification tree [75] is a rule for predicting the class of an object from the values of its predictor variables. The tree is constructed by recursively partitioning a learning sample of data in which the class label and the values of the predictor variables for each case are known. Each partition is represented by a node in the tree.

If X is an ordered variable, this approach searches over all possible values c for splits of the form $X\le c$. A case is sent to the left subnode if the inequality is satisfied and to the right subnode otherwise. The values of c are usually restricted to mid-points between consecutively ordered data values. If X is a categorical predictor (i.e., a predictor variable that takes values in an unordered set), the search is over all splits of the form $X\in A$ where A is a non-empty subset of the set of values taken by X.

predictorImportance [63] estimates predictor importance of the predictors for each tree learner in the ensemble ens and returns the weighted average imp computed using ens.TrainedWeight. The output imp has one element for each predictor. predictorImportance computes importance measures of the predictors in a tree by summing changes in the node risk due to splits on every predictor, and then dividing the sum by the total number of branch nodes. The change in the node risk is the difference between the risk for the parent node and the total risk for the two children. For example, if a tree splits a parent node (for example, node 1) into two child nodes (for example, nodes 2 and 3), then predictorImportance increases the importance of the split predictor by [63]:

$$(R_1–R_2–R_3)/N_b$$

(19)

where $R_i$ is node risk of node i, and $N_b$ is the total number of branch nodes. A node risk is defined as a node error weighted by the node probability:

$$R_i=P_i{E}_i,$$

(20)

where $P_i$ is the node probability of node i, and $E_i$ is the mean squared error of node i.

Corrcoef [63] corrcoef(A) returns the matrix of correlation coefficients for A, where the columns of A represent random variables and the rows represent observations Correlation as stated in [76]is a measure of a monotonic association between 2 variables. A monotonic relationship between 2 variables is a one in which either (1) as the value of 1 variable increases, so does the value of the other variable; or (2) as the value of 1 variable increases, the other variable value decreases. In correlated data, therefore, the change in the magnitude of 1 variable is associated with a change in the magnitude of another variable, either in the same or in the opposite direction. In other words, higher values of 1 variable tend to be associated with either higher (positive correlation) or lower (negative correlation) values of the other variable, and vice versa. Assumptions of a Pearson correlation are as follows:

• As is actually true for any statistical inference, the data are derived from a random, or at least representative, sample. If the data are not representative of the population of interest, one cannot draw meaningful conclusions about that population.
• Both variables are continuous, jointly normally distributed, random variables. They follow a bivariate normal distribution in the population from which they were sampled. The bivariate normal distribution is beyond the scope of this tutorial but need not be fully understood to use a Pearson coefficient. The equation for correlation coefficent is represented as follows [77]:

$${\rho }_{x}y=\frac{cov(X,Y)}{{\sigma }_x{\sigma }_y}$$

(21)

where,

cov is the covariance,

${\sigma }_x$ is the standard deviation of X,

${\sigma }_y$ is the standard deviation of Y

Infinite Latent Feature Selection (ILFS). [78]: Consider a training set $X=\{\overrightarrow{x_1},...,\overrightarrow{x_n}\}$, such that the distribution of the values assumed by the $i^{th}$ features is given by $m\times 1$ vector $\overrightarrow{x_l}$, taking into account m samples. An undirected graph G is formed so that the features are represented by the nodes and the inter-node relationships are represented by the edges. If $a_{ij}$ is an element of the adjacency matrix, A associated with G, that represents the pairwise relationship between the features $x_i$ and $x_j(1\le i,j\le n$ ) . G can be represented by the binary function [78]:

$$a_{ij}=\varphi (x_i,x_j)$$

(22)

where $\varphi$ is a real valued potential function. The probability of each co-occurrence in $x_i$ and $x_j$ is framed as a mixture of conditionally independent multinomial distribution, where parameters are learned using Expectation Maximization(EM) algorithm.

Feature selection via Eigenvector Centrality (ECFS). The adjacency matrix of the above graph G can be written as [21]:

$$A=\alpha k+(1-\alpha )\sum (i,j)$$

(23)

where $\alpha \in \left[0,1\right]$ is a loading coefficient. In Eigenvector Centrality measure (EC), $v_o$ is calculated as the eigen vector of A associated with the largest eigen value. If e is any vector,

$$\underset{l\to L}{lim}\left[A^{l}e\right]=v_o$$

(24)

Relieff. Relieff is an algorithm developed by Kira and Rendell [79,80] in 1992 that uses filter-method approach for feature selection. If a dataset consists of n instances of p features, belonging to two classes. At each iteration, X is a feature vector belonging to one random instance and the feature vectors of the instances closest to X from each class using Manhattan L1 norm are chosen. ’Near hit’ is the closest instance of the same class and ’Near miss’ is the closest instance of different class. The weight vector is updated as follows [81]:

$$W_i=W_i-{(x_i-nearHi{t}_i)}^2-{(x_i-nearMis{s}_i)}^2$$

(25)

Feature Selection Concave (FSV). If matrices $A\in R^{mxn}$ and $B\in R^{kxn}$ are two point sets, then they can be discriminated by a separating plane, P as in [22]:

$$P=x|x\in R^n,x^{T}w=\gamma$$

(26)

where normal $w\in R^n$ and 1-norm distance to the origin is defined as $\frac{\left|\gamma \right|}{{∥w∥}_{inf}}$.

Laplacian. A parameter used in this algorithm is the Laplacian Score(LS) which means that two points are related to the same topic if they are close to each other. Laplacian score of the $r^{th}$ feature is calculated as follows [23]:

$$L_r=\frac{{\widetilde{f_r}}^{T}L\widetilde{f_r}}{{\widetilde{f_r}}^{T}D\widetilde{f_r}}$$

(27)

Unsupervised Discriminative Feature Selection (UDFS). UDFS aims to select the most discriminative features for data representation, where manifold structure is considered. $X=\left\{x_1,x_2,...,x_n\right\}$ is the training set, $x_i\in R^d(1<i<n)$ is the $i^{th}$ datum and n is the number of data points in the training set. The objective function of this algorithm is: For an arbitrary matrix, $A\in R^{rxp}$, its $l_{2,1}$-norm [24] is:

$${∥A∥}_{2,1}=\sum _{i=1}^R\sqrt{{\sum }_{j=1}^p{A}_{ij}^2}$$

(28)

Local Learning Clustering based Feature Selection (LLCFS). This algorithm constructs the k-nearest neighbor graph in the weighted feature space. It performs joint clustering and feature weight learning [25].

Correlation based Feature Selection(CFS). This algorithm [26] performs feature selection on the basis of the hypothesis,“good feature subsets contain features highly correlated with the classification, yet uncorrelated to each other”. A merit function is a function that measures the agreement between data and the fitting model, for a particular choice of parameters. By definition, the merit function is small when the agreement is good. The merit function of a feature subset S consisting of k features is given as [82]:

$$Meri{t}_{S_k}=\frac{k\overline{r_{cf}}}{\sqrt{k+k(k-1)\overline{r_{ff}})}}$$

(29)

where $\overline{r_{cf}}$ is the mean of all feature-classification correlations, and

$\overline{r_{ff}}$ is the mean of all feature-feature correlations.

6.4. Classification Methods

Bayesian network. A Bayesian network is a directed acyclic graph with some quantitative probability information assigned to each node that corresponds to a random variable. It has many other synonyms, viz, belief networks, probabilistic network, causal network and knowledge map [35]. A conditional probability distribution $P\left(x_i\right|parents\left(X_i\right))$ defines the relationship between each node and its parents. It is defined by the following equation [35]:

$$P(x_1,.....,x_n)=\prod _{i=1}^{n}P\left(x_i\right|parents\left(X_i\right))$$

(30)

where $P(x_1,.....,x_n)$ = probability of joint conjunction of events $x_1$, $x_2$, .....$x_n$.

Decision tree. The goal attribute is true if and only if the input attributes follow the paths towards a leaf with value true. This assertion gives a decision tree and its propositional logic can be written as follows [35],

$$Goal\iff Pat{h}_{1}VPat{h}_{2}V...$$

(31)

In MATLAB definition, fine trees have the highest model flexibility [63] as they have many leaves to make many fine distinctions between classes. They allow a maximum of 100 splits. In case of medium trees, the model flexibility is medium. They allow a maximum of 20 splits. In case of coarse trees, the model flexibility is low and they allow a maximum of 4 splits.

Logistic regression. The logistic function is given by the following equation [35]:

$$Logistic\left(z\right)=\frac{1}{1+e^{-z}}$$

(32)

It gives the probability of belonging to the class labeled 1. The process of fitting the weights of this model to minimize loss on a data set is called logistic regression.

RUS boosted trees. Random under-sampling (RUS) is used to balance an imbalanced class that is a common problem for any datasets having rare occurrences of a particular event, the algorithm of which can be found in [83].

Bagged trees. “Bagging predictors [67] is a method for generating multiple versions of a predictor and using these to get an aggregated predicton”- Breiman. Consider data $\left\{(y_n,x_n),n=1...,N\right\}$ in a learning set, where the y’s are either class labels or a numerical response. If the input is x we predict y by $\varphi (x,L)$, taking repeated bootstrap samples $\left\{L^B\right\}$ from L, and forming $\left\{\varphi (x,L^B)\right\}$ and if y is numerical

$${\varphi }_B\left(x\right)=a{v}_b\varphi (x,L^{\left(B\right)})$$

(33)

If y is a class label, let the $\left\{\varphi (x,L^B)\right\}$ vote to form ${\varphi }_B\left(x\right)$. This is called “bootstrap aggregating” or bagging.

k-means clustering.k-means clustering performed on the majority class of the dataset yielded 743 cluster centers. The value of k is based on idea of equalizing minority class with majority class and generating 743 clusters for majority class. These points were mixed were the datapoints of the minority class to remove bias, and generate a total of 1486 datapoints. The 24 prediction algorithms were run on this new dataset and the results were validated using 5-fold cross validation.

Suppose we are given a dataset $X=x_1,...,x_N,x_n\in R^d$ as stated in [62]. The M-clustering problem aims at partitioning this data set into M disjoint subsets (clusters) $C_1,...,C_M$, such that a clustering criterion is optimized. The most widely used clustering criterion is the sum of the squared Euclidean distances between each data point $x_i$ and the centroid $m_k$ (cluster center) of the subset $C_k$ which contains $x_i$. This criterion is called clustering error and depends on the cluster centers $m_1,...,m_M$:

$$E(m_1,...,m_M)=\sum _{i=1}^N\sum _{k=1}^{M}I(x_i\in C_k){|x_i-m_k|}^2$$

(34)

where I(X)=1 if X is true and 0 otherwise

where ${\mu }_i$ is the centroid of cluster $S_i$ imbalanced dataset by increasing the number of samples of the minority class. The algorithm flowchart is described in detail in [17].

Support Vector Machine. SVM is a type of supervised learning, where data that is not linearly separable can be easily separated by mapping them into higher dimensional space. The optimal SVM separator is found by solving the following [35] :

$$arg\underset{a}{max}\sum _j{\alpha }_j-\frac{1}{2}\sum _{j,k}{\alpha }_j{\alpha }_j{y}_j{y}_k(x_j.x_k))$$

(35)

where ${\alpha }_j\ge 0$ and ${\sum }_j{\alpha }_j{y}_j=0$. Solution of this equation is done using software called as quadratic programming.

K-nearest neighbor KNN algorithm classification is a type of clustering where the nearest k datapoints NN(k,$x_q$) are considered. The distance metric is measured using Minkowski distance as follows [35]:

$$L^p(x_j,x_q)={\left(\sum _i{\left|x_{j,i}-x_{q,i}\right|}^p\right)}^{\frac{1}{p}}$$

(36)

When $p=2$, it is called Euclidean distance and if $p=1$, it is Manhattan distance.

6.5. Evaluation Matrices

The statistical parameters calculated are as follows [84]:

$$Accuracy=\frac{t_p+t_n}{total}$$

(37)

$$Precision=\frac{t_p}{t_p+f_p}$$

(38)

$$Recall=\frac{t_p}{t_p+f_n}$$

(39)

$$F1-Score=\frac{2xPrecisionxRecall}{Precision+Recall}$$

(40)

where $t_p,t_n,f_p,f_n$ are the number of true positives, true negatives, false positives, false negatives respectively.