We conducted a retrospective cohort study to develop machine-learning models for predicting postoperative functional disability and in-hospital mortality in older patients with cancer. This study used data between April 2016 and March 2023 from 70 Japanese hospitals within the National Hospital Organization (NHO) database across 6 regional groups: Hokkaido-Tohoku group, Kanto-Shinetsu group, Tokai-Hokuriku group, Kinki group, Chugoku-Shikoku group, and Kyushu group (Figure 1) [18].

The NHO maintains 2 databases: (1) an administrative claims database based on the Diagnosis Procedure Combination–based Per-Diem Payment System [19] and a clinical information database based on the standardized structured medical record information exchange [20]. The administrative claims database contains patient information, such as age, sex, cost, comorbidities, complications, diagnosis, medical procedures, and medications. The clinical information database includes medical charts, laboratory data, and vital signs on a daily basis.

The study included patients aged ≥65 years who were admitted to NHO hospitals between April 2016 and March 2023 and underwent surgery for major cancers, including lung, stomach, colorectal, liver, pancreatic, breast, and prostate cancers. These cancer sites were selected because of their high incidence [2] and mortality rates [21] in Japanese and global cancer statistics [1]. The surgical procedures included both scopic and open surgeries under general anesthesia.

We excluded patients who had missing Barthel Index data at admission or discharge, were first included in the database at admission (no medical history available), underwent surgery more than 1 week after admission, or had the Barthel Index of 0 at admission. These exclusion criteria were applied because: (1) patients missing Barthel Index data could not be evaluated for outcome, (2) most of the predictor variables were missing if patients had no medical history, (3) we eliminated the effect of hospitalization on physical function from admission to surgery, and (4) the Barthel Index change from admission to discharge was an outcome variable, but patients with a minimum score of 0 at admission cannot show further decline [22].

The primary outcome was worse discharge, defined as either in-hospital death or postoperative functional disability. Postoperative functional disability was characterized as hospital-associated disability [16] (≥5-point decrease in the Barthel Index between admission and discharge) [17]. The Barthel Index consists of 10 items, including transfer, bathing, and stair climbing, used to evaluate activities of daily living on a scale of 0‐100, with lower scores indicating a decline in physical function.

The secondary outcomes were health care costs and postoperative length of stay (LOS) in patients predicted to be at high risk and low risk for worse discharge. The optimal cut-off point for high-risk or low-risk classification was determined using the Youden index [23] on the receiver operating characteristic (ROC) curve.

We identified the following 37 potential predictors of worse discharge based on previous studies: age (65‐74, 75‐84, and ≥85 years) [9], sex [24], underweight (BMI <18.5 kg/m²) [12] and obesity (BMI ≥30 kg/m²) [11], route of admission (home or nonhome), emergency admission, an estimated household income in the lowest tertile based on post-code (low income) [6], smoking (Brinkman Index ≥200) [14], functional dependence (Barthel Index ≤60) [22], surgical factors (open surgery, scopic surgery, and combined general and epidural anesthesia), the presence of gastrointestinal cancer (colorectal, liver, pancreas, and stomach), the cancer staging (0–II or III–IV) [25], the presence of recurrent cancer, Charlson Comorbidity Index (CCI) ≥3 [26,27], Hospital Frailty Risk Score (HFRS) ≥5 [28], comorbidities common in older patients (cerebrovascular disease, chronic pulmonary disease, congestive heart failure, dementia [8], diabetes, liver disease, myocardial infarction, peptic ulcer disease, peripheral vascular disease, and renal disease), medical history about cancer within 8 weeks before surgery that allowed the evaluation of preoperative chemotherapy [29] and radiotherapy [30] (chemotherapy, radiation, and surgery), vital signs (body temperature ≥38°C [31], systolic blood pressure >180 mm Hg [32]), and laboratory test values (albumin <3.5 g/dL [13], total bilirubin ≥2.0 mg/dL [33], creatinine ≥ 2.0 mg/dL [33], platelet <10⁵/μL [33], and hemoglobin <11 g/dL [10]).

These predictors were measured as follows: age, sex, BMI, route of admission, emergency admission, income, cancer staging evaluated using the Tumor Nodes Metastasis (TNM) classification system [34], the presence of recurrent cancer, Brinkman Index, and Barthel Index were assessed at admission. Surgical factors were assessed during the surgery. The type of cancer was determined during the surgery using the International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10) coding system [35]. Comorbidities were identified from ICD-10 codes within 8 weeks before surgery (Table S1 in Multimedia Appendix 1). Notably, dementia was determined if it was either identified from ICD-10 codes within 8 weeks before surgery or documented in the clinical summary at admission, as Japanese hospitals are required to include the dementia status of inpatients aged ≥65 years at admission [8]. Vital signs were measured closest to surgery after admission, and laboratory values were obtained using those measured closest to surgery within 8 weeks before surgery.

We randomly selected one of the 6 NHO regional groups for the external validation set [36], while hospital data from the remaining 5 groups were randomly divided into the training (70%) and the internal validation (30%) sets [37]. The missing predictor variables were imputed using the “missRanger” [38], which is a random forest–based algorithm [39], assuming that the data are missing at random.

To summarize patient characteristics, continuous variables were expressed as mean (SD) or median (IQR), depending on the distribution of variables. The Wilcoxon rank-sum test or the Welch test was used to assess between-group differences. Categorical variables were expressed as proportions and compared using the χ² test.

In the training set, a double penalty was implemented by eliminating unnecessary variables to create a more practical model for clinical use [40]. The first penalty involved selecting predictor candidates with a crude odds ratio (OR) at P<.1 [41]. The second penalty to further narrow down the predictor candidates used the least absolute shrinkage and selection operator (Lasso) method, which allowed for the selection of clinically relevant variables with consistent relationships [36,42].

The selected factors were incorporated into 6 machine learning models: category boosting (CatBoost) [36,43], extreme gradient boosting (XGBoost) [36,42], logistic regression, neural networks [44], random forest [36], and support vector machine (SVM) [36]. Model performance was evaluated using the area under the ROC curve (AUC) with 95% CIs). Similarly, we calculated the accuracy, sensitivity, specificity, F₁-score, and the area under the precision-recall curve (PRAUC) to assess the performance of the models [36,37]. The precision-recall curve of the models was also shown.

We used the synthetic minority oversampling and random undersampling techniques to avoid overfitting owing to the imbalance between the positive and negative events [45]. The minority class was oversampled at 50%, 100%, and 200% of its original size, followed by random undersampling of the majority class to achieve equal numbers between classes. The models were trained using 10-fold cross-validation with grid search for hyperparameter optimization. Of these sampling ratios and hyperparameter combinations, those yielding the largest AUC were selected. We ranked the predictor variable using the Shapley additive explanations (SHAP) method [46] to assess the contribution of the predictors to the models. Moreover, considering higher cancer stages are associated with poorer postoperative outcomes [25], a multiple logistic regression was conducted to evaluate the interaction between cancer staging and other predictor variables using OR (95% CI). We examined interactions between cancer stage III–IV and the top 5 features based on the mean absolute SHAP value.

We analyzed the AUC difference between the models using the DeLong method [47], which was based on the model with the largest AUC in both internal and external validation. Sensitivity analyses were used to assess the impact of missing data and assessing 2 outcomes simultaneously (death or functional disability) on the models. For missing values, a complete case analysis was performed to confirm the robustness of the results obtained from the imputed data set. We also evaluated the AUC of the models when the models predicted only death in all patients or functional disability in patients with survival discharge. We conducted subgroup analyses by LOS, cancer type (breast, colorectal, liver, lung, pancreas, prostate, or stomach), and cancer staging (stage 0–I, II, III, or IV). For the LOS analysis, we calculated the 75th percentile of LOS for each cancer type separately and divided patients into LOS for each cancer type <75th percentile (short-stay) and LOS ≥75th percentile (long-stay) groups [48].

For the analysis of secondary outcomes, we compared LOS and health care costs between patients at high and low risk based on the model with the highest AUC in both internal and external validation. Health care costs were assessed across various categories, including total, medical consultation, medication, medical procedure, surgical procedure, laboratory tests, hospital stay, and others. The currency conversion rate was 150 JPY to US $1.

All hypothesis tests had a 2-sided significance level of .05. All statistical analyses were performed using R version 4.3.1 (R Foundation for Statistical Computing) .

Our study was approved by the Institutional Review Board of Showa University (approval number 2023‐129-A). Individual consent was not required because this was an opt-out study. This study conforms to the principles outlined in the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.

In this study, we developed and validated machine-learning models to predict postoperative functional disability and mortality in older patients with cancer. Our CatBoost model achieved good performance using routinely available preoperative factors from electronic health records, indicating the potential for clinical implementation. Although ethical training for hospital staff is essential to prevent unauthorized disclosure of prediction results, implementing this model within closed electronic health record systems could provide protection for patient privacy.

The previous model for lower-extremity surgery had an AUC of 0.72 in external validation [6], similar to our model; however, our model directly predicted functional disability using the Barthel Index rather than using nursing home discharge as a surrogate. The model performance remained consistent across sensitivity analyses for death and functional disability separately, and complete cases, indicating the robustness of our findings. Notably, the model demonstrated higher predictive performance in patients with stomach or colorectal cancer than the other cancers, making it especially valuable for surgical decision-making in these patients.

The CatBoost model identified patients at high risk who had longer LOS and higher health care costs; however, surgical procedure costs were comparable between patients at high and low risk. These findings suggested that the increased cost was based on the varying postoperative course. Our model can support decision-making for older patients with cancer and their families regarding cancer surgery by providing insights into potential postoperative QOL and care burden. Moreover, if patients at high risk choose to undergo cancer surgery, our model may enable health care providers to implement targeted interventions such as intensive postoperative rehabilitation. Early identification of patients at high risk, such as those aged ≥85 years with dementia, can help health care providers prepare support systems, including caregiver education, social work consultation for home health support, and coordination with multidisciplinary teams [15]. This proactive approach may help reduce caregiver burden and improve outcomes for both patients and their families.

Of the 6 machine-learning models, the CatBoost and XGBoost models, with AUC ≥0.80, had the same combination of 14 features in the top 15 influential factors. These factors include established risk factors for poor postoperative outcomes in older patients as identified in previous studies: dementia [7], older age [9] (≥85 y and 75‐84 y), male sex [24], anemia [10] (hemoglobin <11 g/dL), low income [6], underweight [12] (BMI <18.5 kg/m²), diabetes [49], and cancer staging [25]. In addition, several factors serve as proxies for known risk factors. For frailty [5], the factors include (1) open surgery, which generally results in a more pronounced postoperative functional disability compared with scopic surgeries; (2) nonhome admission, likely indicating that patients are too frail to live independently; and (3) Barthel Index ≤60 at admission, indicating severe dependence [22]. For malnutrition [13], the proxies are (1) albumin <3.5 g/dL, a marker of malnutrition, and (2) gastrointestinal cancer, which often involves a long time to restart food intake after surgery, increasing the risk of malnutrition compared with other cancer types. The consistency between the 2 models in identifying these factors further validates their importance in predicting postoperative outcomes. Although chemotherapy and creatinine ≥2.0 mg/dL were not common in both models, these factors, identified as influential factors in previous studies [33,50], might also be included as predictor variables in future models. Moreover, our models included several factors associated with social vulnerability, such as age ≥85 years, dementia, and low income. Without ensuring model transparency to health care providers, our model could unconsciously contribute to reduced surgical care access for vulnerable populations. Therefore, when implementing our model in clinical practice, health care providers should consider these model characteristics to ensure fair allocation of health care resources [51].

Our study has some limitations. First, we only validated all models using Japanese data. While our CatBoost model showed moderate accuracy (AUC: 0.7‐0.9) [52] in both internal and external validation, the AUC in the external validation was lower than that in the internal validation as observed in a previous study [6]. Further studies using data from other countries and ethnic groups are necessary to evaluate model robustness, including potential bias and overfitting [51], and confirm their applicability to different health care systems. However, our study used data from 70 hospitals across Japan, which may enhance generalizability within the country. In addition, considering the global trend of population aging, our models may prove valuable for other countries in the future, particularly when these countries reach levels of demographic aging similar to Japan’s current situation.

Second, we did not have information on predictor variables such as marital status [6] because of the retrospective nature of the study. Despite this limitation, our models had good predictive performance in the validation sets. While our analysis showed no significant interactions between stage III–IV cancer and the top features, future studies incorporating additional variables may evaluate such interactions to enhance the predictive performance of models.

Finally, the long-term prognosis of patients classified as high-risk by our models remains unclear. Further research is required to determine the extent of functional recovery and mortality in these patients. At a minimum, postoperative functional disability in patients at high risk indicates an increased immediate post-discharge burden on family caregivers and health care resources.

Our CatBoost model achieved good performance for predicting postoperative functional disability and mortality in older patients with cancer. This model could support surgical decision-making for patients and families while guiding targeted interventions by health care providers. This model, which is based on routinely available preoperative factors, has the potential for implementation in clinical settings through electronic health records.