Development and Validation of the Communities Geriatric Mild Cognitive Impairment Risk Calculator (CGMCI-Risk)

1. Introduction

Mild Cognitive Impairment (MCI) is an intermediate stage between normal aging and dementia, often considered a clinical state that precedes Alzheimer's disease (AD) [1,2]. The global prevalence of MCI among community-dwelling adults aged 50 and older was over 15% [3]. In China, the prevalence of MCI among individuals aged 60 and above is 15.5%, with the number of affected reaching 38.77 million [4]. It is estimated that over 50% of individuals diagnosed with MCI will develop dementia within five years, with only a small percentage will maintain stable cognitive function [5]. In the absence of timely diagnosis, patients may exhibit impairments in two or more cognitive domains, such as memory, language, executive function, perceptual speed and visuospatial abilities [6,7,8]. These impairments can severely affect the patients’ independence in daily life and place a significant burden on caregivers and families, ultimately increasing societal burden [9]. Therefore, it is crucial to identify individuals at high risk for MCI as early as possible.

Prior research has identified several risk factors affecting cognitive function in older adults. Demographic characteristics such as age, sex, educational level, and marital status have all been shown to significantly impact cognitive impairment [10,11]. A significant body of research has established a link between cognitive decline and various physical health conditions. These include basic activities of daily living (BADL) [12], instrumental activities of daily living (IADL) [13], body mass index (BMI) [14], blood pressure [15], vision [16], hearing [17], and chronic diseases such as diabetes and stroke [18]. Additionally, lifestyle factors such as exercise [19], diet [20], smoking, and alcohol use [21], have been shown to have a considerable impact on cognitive function in older adults. At the biological level, evidence indicates that sex hormones [22] and hemoglobin levels [23] are linked to cognitive dysfunction. Ferri et al. [24] established a strong correlation between depressive states and cognitive decline. Additionally, Gui et al. [25] identified a clear association between the APOEε4 allele and MCI. Numerous factors influence cognitive function in the elderly, and risk factors varying across studies. Developing a prediction model for MCI is therefore essential for the early identification, prevention, and treatment [26].

In prior literature, Huang et al. [27] developed an MCI prediction model using data from 478 community-dwelling middle-aged and older adults (≥45 years old). The predictors included age, sex, educational level, place of residence, and reading, with an area under the receiver operating characteristic curve (AUROC) of 0.870. This model lacks external validation, and its predictive effect may be biased. Ma et al. [28] developed an MCI risk prediction model for older adults (≥60 years old) using public datasets. This risk prediction model used different MCI assessment methods during the development and validation phases, which may impact the stability and accuracy of the model. Additionally, MCI prediction models have been developed for other specific patient populations, including those with hypertension [29], diabetes [30], and stroke [31]. While prediction tools for these populations are available, variations in study populations and study designs have resulted in a diversity of model variables. MCI prediction models have also been developed using data from various data sources, including neuro biomarkers like A-β amyloid [32] and tau protein [33], neuroimaging variables such as brain microstructure [34], and genetics such as mitochondria-related genes [35]. The predictive accuracy of these models has significantly improved; however, implementing these biomarker tests in community settings remains challenging. Consequently, the objective of this study was to develop and validate a risk prediction model, known as CGMCI-Risk, for MCI in community-dwelling older adults. This model aims to help community healthcare professionals identify high-risk MCI groups, thereby facilitating the optimization of prevention and intervention strategies.

2. Materials and Methods

2.1. Dataset

The data were obtained from the Chinese Longitudinal Healthy Longevity Survey (CLHLS). The CLHLS is one of the largest national longitudinal studies examining the health status of older adults in China. It includes eight surveys conducted in 23 provinces, municipalities, and autonomous regions between 1998 and 2018. The sampling area covered approximately 85% of China’s total population [36]. The CLHLS dataset includes basic information, health assessments, personality traits, lifestyle factors, personal backgrounds, and indicators of physical health. All eligible participants who consented signed an informed consent form. For older adults unable to sign the form themselves, consent was provided by a family member on their behalf.

2.2. Design and participants

In this study, the CLHLS cohort data from 2008 to 2011 was selected as the derivation set, and the CLHLS cohort data from 2011 to 2014 was used as the temporal validation set for the model. The study population included community-dwelling older adults aged 60 years and older with healthy cognition. Individuals who were institutionalized or who self-reported or were diagnosed with dementia were excluded. Figure 1 presents flowchart of sample selection in the present study. This study was conducted in accordance with the TRIPOD reporting specifications [37].

2.3. Assessment of MCI

Cognitive functioning was assessed using the Chinese Mini-Mental State Examination (CMMSE), which consists of 24 items assessing general cognitive ability, reactivity, attention and calculation, recall, and language comprehension and coordination. Each correct response earns one point, while incorrect responses receive zero points. Question 6, "Count the number of food groups in one minute," has a maximum score of seven, making the total score range from zero to thirty. Based on previous research [38,39], the CMMSE score below 24 was considered as MCI.

2.4. Definition of candidate variables

In accordance with previous research on cognitive dysfunction in older adults [10,40,41], 41 candidate variables were selected from the CLHLS survey data for analysis in this study. The variables were grouped into four categories. 1) Demographic characteristics, these include age, educational level, sex, ethnic group, place of residence, marital status, and cohabitant status. 2) Health status and lifestyle, these included BMI, medical expenses, baseline CMMSE score, sleep duration, BADL, IADL, physical performance test (PPT), chronic diseases, masticatory function, vision, hearing, frequency of intake of fruits, fresh vegetables, animal protein, plant protein and tea, house work, field work, garden work, raise domestic animals or pets, read newspapers or books, TV watching or radio listening, playing cards or mahjong, social activities, smoke, alcohol use, and exercise. 3) Mental health, it includes resilience score, life satisfaction, health satisfaction, sleep satisfaction, and financial satisfaction. 4) Community and family support, this includes child support and community services. See Supplementary Materials Table S1 for information on candidate variables.

2.5. Sample size

The study was designed to achieve an AUROC of at least 0.7 [42]. A total of 5,470 participants and 821 outcome events were calculated using the R package "pmsampsize" (version 1.1.3), in accordance with the methodology outlined by Riley et al. [43].

2.6. Missing value

Values marked as "missing", "unclear" and "unanswerable" were considered as missing data. Individuals with more than 5% of their variables missing were excluded from the analysis. Missing values were imputed using the nearest neighbor (KNN) method, which involves estimating the value of a missing data point based on the mean (for numeric variables) or the most common (for categorical variables) value observed in the K participants who are most similar to the participant with the missing value [44]. In this study, the value of K was set to 5.

2.7. Statistical analysis

The study data were statistically analyzed using R (version 4.3.2). The numeric variables were analyzed according to the distribution characteristics of the variables. Variables with approximately normal distributions were described using means and standard deviations. Comparisons between groups were performed using the independent samples t-test. Numeric variables with severe skewed distributions were described using medians and interquartile ranges (IQR). Comparisons between groups were performed using non-parametric tests. Categorical variables were described as frequencies and percentages. Comparisons between groups were performed using the chi-square test. All tests were performed with two-sided p<0.05 as statistically significant differences. The R packages used in this study include: "pmsampsize" (version 1.1.3), "VIM" (version 6.2.2),"tableone" (version 0.13.2), "MASS" (version 7.3-60), "rms" (version 6.7-1), "pROC" (version 1.18.5), "rmda" (version 1.6), "caret" (version 6.0-94), and "nomogramEx" (version 3.0).

2.8. Model development and validation

A backward stepwise regression analysis was employed to screen the variables in accordance with the Akaike Information Criterion (AIC) minimization principle. The screened predictor variables were then included in the first multivariate logistic regression analysis, and subsequently, significant predictor variables were included in the second logistic regression analysis until all predictor variables in the model were shown to be significant. The CGMCI-Risk model was developed using logistic regression. Scores for each variable were extracted using the R package "nomogramEx" (version 3.0), which contributed to the formation of the CGMCI-Risk model.

The AUROC [45], calibration curve [46], and DCA [47] were employed to evaluate the discriminatory power, calibration, and clinical utility of CGMCI-Risk, respectively. The AUROC ranged from 0 to 1, with a value of 1 indicating 100% correct predictions [45]. The Hosmer-Lemeshow (H-L) goodness-of-fit (GOF) test was employed to construct calibration plots, which were utilized to assess the degree of agreement between the predicted risk and actual status of MCI. In a calibration plot, the x-axis represents the predicted probability, the y-axis represents the observed probability, and the diagonal line represents a perfect prediction [48]. DCA calculates the net benefit through a series of risk-probability thresholds and analyzes the value and consequences of the decision under consideration [49]. The robustness of the model was evaluated through the application of bootstrap resampling and temporal validation. The accuracy, sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) of CGMCI-Risk were subsequently reported.

3. Results

3.1. Participants

Of the 6,058 participants in the derivation set (median [IQR] age was 79 [71, 87] years), 933 (16.3%) developed MCI within 3 years, 2963 (48.9%) were female and 3,095 (51.1%) were male. Of the 4,448 participants in the temporal validation set (median [IQR] age was 79 [73, 87] years), 635 (14.1%) developed MCI within 3 years, 2,205 (49.1%) were female and 2,283 (50.9%) were male. More cohort characteristics of the derivation set and temporal validation set are presented in Supplementary Materials Table S2.

3.2. CGMCI-Risk development and validation

The set of variables with a minimum AIC value of 4,548 was screened by backward stepwise regression analysis. The first multivariate logistic regression analysis was performed using the screened variables, the statistically insignificant variables were removed, and the multivariate logistic regression model was reconstructed using the remaining significant variables. The eventually multivariate logistic regression model included age, educational level, sex, exercise, garden work, TV watching or radio listening, IADL, hearing, and masticatory function. The results are shown in Supplementary Materials Table S3. Based on this model, the CGMCI-Risk tool was developed to calculate disease scores, with personalized results obtained by inputting the appropriate values or options. The corresponding scores for each parameter are presented in Supplementary Materials Table S4. The equation that describes the relationship between the predicted risk value and the sum of the parameters scores is as follows:

Risk = 0.852405178 - 4.17e-07 * Total Points³ + 0.000194207 * Total Points² - 0.022360467 * Total Points.

(1)

We provide this CGMCI-Risk model with a user-friendly web platform that enables the estimation of normative deviation scores from any sample with minimal technical and computing requirements: http://g152335m31.imdo.co/Communities Geriatric Mild Cognitive Impairment Risk Calculator.

The CGMCI-Risk model demonstrated robust discriminatory power in identifying MCI risk in community-dwelling older adults, with an AUROC of 0.781 (Figure 2). The accuracy, sensitivity, specificity, NPV, and PPV of CGMCI-Risk in the derivation set were 0.717, 0.710, 0.719, 0.927 and 0.330, respectively. The study employed the H-L GOF test to assess the consistency between the model's predicted probability and the actual probability (p = 0.146). Calibration plots were also constructed to facilitate this assessment. The results demonstrated a robust correlation between the actual and predicted probabilities, with a mean absolute error (MAE) of 0.011 and a mean square error (MSE) of 0.00023 (Figure 3). The DCA illustrated that CGMCI-Risk offers a greater net benefit compared to full treatment or no treatment when the risk threshold ranges from 4% to 57% (Figure 4).

Internal validation employed bootstrap resampling 1,000 times with AUROC, sensitivity, and specificity of 0.776, 0.977, and 0.125, respectively. Temporal validation set was conducted using data from the CLHLS cohort from 2011 to 2014. The AUROC, sensitivity, and specificity were 0.782, 0.765, and 0.666, respectively (Figure 2). The calibration curve and DCA are similar to the derivation set indicate that the CGMCI-Risk is an effective tool for identifying older adults at risk for MCI in the community (Figure 4 and Figure 5), thereby providing a foundation for early cognitive intervention by community healthcare workers.

4. Discussion

In this study, CGMCI-Risk was developed based on a dataset comprising 6,058 samples, and its external validity was subsequently temporal validated with an additional 4,488 samples. The AUROC values demonstrated consistent performance, with a mean of approximately 0.8, indicating a high degree of discriminative ability. The calibration curve demonstrated excellent consistency, and DCA validated its utility, establishing a robust tool for MCI risk assessment in community-dwelling older adults. The CGMCI-Risk incorporates age, educational level, sex, exercise, garden work, TV watching or radio listening, IADL, hearing, and masticatory function.

Age and sex are significant non-intervention factors in cognitive impairment. The prevalence of MCI increases with age. As a consequence of the aging process, the volume of the cerebral cortex and hippocampus diminishes [50]. This results in a blockage of information delivery, which in turn impairs cognitive function [50]. Yesavage et al. [51] modeling the prevalence and incidence of AD and MCI. Primary found of the model include that the conversion rate from normal cognitive state to MCI increased from 1% per year at age 60 to 11% at age 85. This suggests that age is a significant risk factor for the development of MCI. A meta-analysis of the association between sex and MCI revealed that women are a risk factor for MCI [52]. The role of estrogen in neurogenesis in the hippocampus is significant, and fluctuations in its levels may be associated with an increased risk of MCI in female [53]. Furthermore, women are more prone to the formation of ApoEε4-associated neurogenic fiber tangles, which may contribute to an elevated risk of cognitive impairment [54]. A review of the literature reveals a correlation between educational attainment and a number of factors related to cognitive functioning, including the thickness of the cerebral cortex, gray matter volume, and neural network connectivity [55]. Individuals with higher levels of education tend to demonstrate superior cognitive functioning [56], whereas illiteracy or lower educational attainment represents a substantial risk factor for MCI [57].

There is a strong correlation between IADL and cognitive function. As IADL declines, older adults may also experience a decline in cognitive abilities [13]. This association may be attributed to the fact that sustained stimulation of cerebral function through IADL preserves the activity and plasticity of the brain's neural networks, thereby assisting in the mitigation of cognitive decline [58]. A reduction in IADL may also result in a decline in socialization among older adults, which may further impact their cognitive function [59]. Hearing impairment represents a significant risk factor for the onset of MCI. A study investigating the impact of hearing on cognitive function demonstrated that individuals with normal hearing exhibited superior performance on cognitive assessments [60]. This may be attributed to the fact that hearing impairment can result in alterations to brain structure and function [17]. Examples of these changes include a decline in brain signals, degeneration of the auditory cortex, loss of neurons and neuron branches, and a reduction in overall brain volume [61]. Such alterations may impact the brain's capacity to process and perceive sound, potentially contributing to cognitive decline. Tooth loss can result in difficulty chewing, which may affect nutrient absorption and cognitive function in the brain [62]. Momose et al. [63] and Onozuka et al. [64] have demonstrated increased hemodynamic responses in the prefrontal cortex and hippocampus during chewing, which plays a crucial role in cognitive function. Research has indicated a correlation between tooth loss and chewing difficulties and cognitive decline [65], while effective mastication has been shown to have a beneficial impact on the prevention of MCI [66].

Regular exercise has been demonstrated to exert a beneficial influence on the brain [67]. A research study demonstrated that sustained exercise can delay the onset of cognitive impairment in older adults [68]. An intervention study by Kim and colleagues also confirmed that exercise may improve cognitive function in older adults aged 65 and above [19]. Regular exercise has been demonstrated to facilitate the formation of neural connections between regions of the brain that are essential for optimal cognitive function [69]. Furthermore, it facilitates the release of brain-derived neurotrophic factor (BDNF) in the brain, which is instrumental in promoting neuronal growth, connectivity, and maintenance [70,71]. It is hypothesized that gardening may confer benefits with respect to cognitive function in older adults. Findings from a four-year longitudinal study indicate that gardening may be a significant factor in the reversal of MCI in older adults [72]. In addition to providing enriching stimulation [73], gardening has been shown to result in significantly higher levels of BDNF, which can lead to improvements in both physical and cognitive functioning [74]. Furthermore, the role of passive activities such as watching television or listening to the radio in cognitive impairment has been demonstrated. Lin et al. [75] and Major et al. [76] have shown that these activities can significantly improve cognitive performance in older adults. However, Jung et al. [77] posit that television viewing may be associated with an increased risk of cognitive impairment in later life. This may be attributed to the fact that prolonged television viewing is frequently linked to sedentary behavior, which can result in inadequate physical activity or reduced time spent gardening. Consequently, watching television or listening to the radio may become a risk factor [76].

Currently, more than 55 million individuals worldwide are affected by dementia, with AD representing approximately 60 to 70 percent of dementia cases [78]. MCI progresses to AD at a rate of 10 to 15 percent per year, whereas the rate of transition to AD in normal older adults is only 1 to 2 percent per year [79]. Although current clinical interventions may not be capable of curing these diseases, timely recognition and diagnosis are essential for improving patient prognosis and reducing the burden on caregivers [80]. CGMCI-Risk enhances accessibility and feasibility of assessment and optimizes healthcare worker engagement. It is particularly suited to community settings, providing community healthcare workers with a foundational resource for conducting early cognitive interventions.

The CGMCI-Risk model was developed to identify high-risk groups for MCI in community-dwelling older adults. Although laboratory parameters, imaging features, biomarkers, and genetic indicators have significant potential for MCI prediction, they were excluded in this study due to logistical and operational feasibility in a real-world community setting. In addition, the three-year interval for MCI assessment may introduce bias and fail to capture subtle disease changes. The elevated mortality rate among older adults may also lead to increased data loss and impact assessment precision. The CGMCI-Risk model was developed based on Chinese community-dwelling older adults and while promising for use in community settings, requires further validation for its applicability in different care facilities and cultural backgrounds.

5. Conclusions

The CGMCI-Risk model incorporates factors such as age, educational level, sex, exercise, garden work, TV watching or radio listening, IADL, hearing, and masticatory function. This tool can be used in community settings to help healthcare providers identify older adults in the community at elevated risk for MCI within three-year.