The Chinese Longitudinal Healthy Longevity Survey (CLHLS) is a prospective cohort study of Chinese older adults that contains information on demographics, cognitive function, lifestyle factors, chronic diseases, and more [18,19]. The CLHLS began in 1998, and follow-up surveys have been conducted every 2-3 years since. The data include older adults from 23 of China’s provinces that together make up 85% of the country’s total population.

We used the 2011 and 2014 CLHLS waves in our study. Baseline characteristics were gathered from the 2011 survey and used to predict if an individual became cognitively impaired by 2014. CLHLS participants younger than the age of 60 years or those with cognitive impairment at baseline were excluded. Sample size calculations were conducted following the methodology for multivariable prediction models by Riley et al [20]. This study is presented following the TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines where appropriate [21,22].

The CLHLS received ethics approval from the Duke University Institutional Review Board (Pro00062871) and Peking University’s Biomedical Ethics Committee (IRB00001052–13,074). Written informed consent was given by all participants prior to the survey interviews. This study secondarily analyzed anonymized data from the CLHLS.

Cognition was assessed through the Chinese-language version of the Mini Mental State Examination (MMSE) [23]. MMSE scores range from 0 to 30, with lower scores indicating worse cognitive function. We adopted education-specific cutoffs that have been previously validated in the Chinese older adult population to indicate cognitive impairment [24]. Those with no formal education and MMSE scores less than 18 were labeled as cognitively impaired, as were those with 1-6 years of education with scores less than 21 and those with more than 6 years of education with scores less than 25 [24].

Prediction models were selected based on the following criteria: the model was developed for use in China, was reproducible using the CLHLS, and had an AUC of >0.75 during development. We selected 3 models published in Zhou et al [8], Hu et al [5], and Wang et al [7]. Each model was developed for use in the general Chinese population and showed excellent predictive performance (AUC>0.80) during development. All the models we recreated were based on logistic regression, which returns predictions by summing weighted values of each covariate before the sum is passed through the logistic function to produce predicted probabilities between 0 and 1. The logistic regression model recreated from Zhou et al [8] included age, a functional independence score based on ADL, baseline MMSE score, chewing ability, visual function, history of stroke, whether the participant watches TV or listens to the radio, and whether the participant grows flowers or raises pets. From Hu et al [5], the recreated model included age, marital status, IADL, and baseline MMSE score. Lastly, the model from Wang et al [7] included age; education; sex; ADL; baseline MMSE; and whether the participant gardens, reads newspapers or books, plays mahjong or cards, watches TV, or listens to the radio.

All analyses were performed using the R Statistical Software (version 4.0.5; R Foundation for Statistical Computing), and all code required to reproduce the analyses presented herein can be found on the web [25]. Predictive ability was quantified using AUC, sensitivity, and specificity. We assessed the predictive ability of each risk factor group using logistic regression models evaluated through 20 repeats of 10-fold cross validation, which has been recommended to obtain optimism-corrected performance metrics for prediction models [26]. This resulted in 200 training sets and 200 validation sets. All “I don’t know” or “refused” responses in the CLHLSs were set to missing, ordinal variables were integer encoded, and nonordinal categorical variables were dummy encoded. Missing values were imputed on each training and validation set separately using k-nearest neighbors imputation [27]. During each iteration of cross-validation, the data were split into training and validation sets before 9 logistic regression models, each containing all covariates in 1 particular risk factor group, were fit to the training data. Thereafter, each model was used to make predictions on the validation set for the general population and 6 subpopulations: male, female, rural-dwelling, urban-dwelling, educated, and not formally educated older adults. The same procedure was also followed for evaluating the prediction models from Zhou et al [8], Hu et al [5], and Wang et al [7]. Average AUCs and accompanying 95% CIs were calculated across the 200 validation-set AUCs for each model in this study. Sensitivity and specificity curves, 1 from each validation set, were also plotted for the risk factor group models.

In this study, we quantified the ability of 9 risk factor groups and 3 prediction models to predict future cognitive impairment in the general Chinese population and 6 population subsets: male, female, rural-dwelling, urban-dwelling, educated, and not formally educated older adults. In the general population, the risk factor groups with the best predictive ability were demographics (AUC 0.78, 95% CI 0.77-0.78), cognitive tests (AUC 0.72, 95% CI 0.72-0.73), and IADL (AUC 0.71, 95% CI 0.70-0.71). The most predictive risk factors and the existing models performed inconsistently across socioeconomic groups and had significantly higher AUCs when making predictions for female individuals and older adults with no formal education compared to male individuals and older adults with some education.

Our study showed that the 3 existing prediction models had significantly lower AUCs when predicting future cognitive impairment among male, urban-dwelling, and educated Chinese older adults compared to female, rural-dwelling, and not formally educated older adults. Despite the only shared risk factors in all 3 models being age and baseline MMSE score, significant differences in predictive ability were consistent across every model. One explanation is that risk factor differences between those who developed cognitive impairment and those who did not were larger among the groups where more accurate predictions were made. For example, the difference in average age between female older adults who did and did not become cognitively impaired was 11.8 years, whereas for male older adults, it was 7.0 years. Similarly, among those with no formal education, the difference in baseline MMSE score between those with and without cognitive impairment at follow-up was 2.67 compared to 1.45 among those with some education. In addition, the prevalence of cognitive impairment in our sample was higher among female, rural-dwelling, and not formally educated older adults, meaning that the models had more events to learn from. Indeed, previous studies using nationally representative data have also reported higher prevalence estimates among these groups [3]. Our results indicate that targeted prediction models for specific socioeconomic groups are needed in China to make equally accurate risk assessments across sex, residential status, and education level. Several studies have called for such models [28,29], but as of this writing, none have been developed in China.

Out of the 9 risk factor groups, we found that demographics, cognitive tests, and IADL best predict future cognitive impairment in the general Chinese population and across sex, residential status, and education level. Demographics are often included in prediction models for cognitive impairment [28,30-32], and we suggest that they continue to be leveraged because of their predictive power and ease to collect. Associations between chronic diseases, ADL, psychological factors, and diet with cognitive impairment among Chinese older adults have been established [13,33-39], but such factors showed moderate predictive ability in our study. To our knowledge, dietary factors have not been incorporated into existing prediction models in China, but they had higher AUCs than commonly used risk factors such as psychological factors, ADL, and chronic diseases. In fact, chronic diseases did not make significantly better than random predictions among male (AUC 0.50, 95% CI 0.49-0.51), urban-dwelling (AUC 0.50, 95% CI 0.50-0.51), and not formally educated (AUC 0.51, 95% CI 0.49-0.52) older adults. Hence, in addition to providing a ranking of the most predictive risk factor groups, our study is the first to show that dietary factors warrant consideration when predicting future cognitive impairment among Chinese older adults.

Many risk factor groups had significantly different AUCs across population subsets. Similar to the existing models we recreated, our study revealed that the most predictive risk factors (demographics, cognitive tests, and IADL) had significantly higher AUCs when making predictions among female and not formally educated older adults compared to male and educated older adults. As was the case with the recreated models, this likely resulted from the distributions of risk factors being more separable between those who developed cognitive impairment and those who did not in the groups where more accurate predictions were made. Given the lack of available evidence, it is unclear whether the discrepancies in predictive ability found in our study generalize outside of China, and future work may seek to perform similar analyses elsewhere.

Our study has several limitations. The source code was not available for the models we selected to recreate in this study, but we explicitly followed all preprocessing, variable derivation, and model creation procedures described in the original papers during the model replication process. The AUCs of each model in the general population in this study were consistent with the reported AUCs in the original papers, suggesting that the models were properly recreated from scratch. To facilitate future research, we have further made our code publicly available. Second, the CLHLS is not nationally representative, although it does include older adults from 23 of China’s provinces. The exercise and sleep group did not include objective measurements of physical activity and sleep. Self-reported exercise and sleep are often inaccurate, and we suggest that the results be interpreted with caution for the exercise and sleep group. Lastly, the data used in this study were from 2011 to 2014. Future studies may wish to collect new data and further validate the results presented herein.

Out of the 9 risk factor groups, our study found that demographics, cognitive tests, and IADL best predicted future cognitive impairment among Chinese older adults and had significantly better predictive ability among female and not formally educated older adults compared to male and educated older adults. Similarly, every existing model we recreated made significantly better predictions among female, rural-dwelling, and not formally educated older adults. Our study suggests that more targeted predictions models for cognitive impairment are needed to make equally accurate risk assessments across different socioeconomic groups in China and provides foundational evidence that can support variable selection for such models.