Drawing on the frameworks proposed by Eila et al [18], Rychetnik et al [25], and Ferrante et al [26], we developed the “importance for physiological and psychological impact” (IPPI) questionnaire to systematically evaluate participants’ values and preferences regarding concerns related to PSA screening. The instrument consisted of 10 items across two dimensions: 4 assessing physiological impacts and 6 evaluating psychological impacts. The full questionnaire is available in Multimedia Appendix 1.

This 10-item questionnaire underwent expert validity assessment by 10 urologists, each with more than 7 years of clinical experience. Reliability analysis, based on data from the first 300 patients, demonstrated strong internal consistency, with Cronbach α values of 0.838 for the physiological impact dimension and 0.900 for the psychological impact dimension.

Factor analysis of the IPPI was conducted using the maximum likelihood method, followed by Varimax rotation. Maximum likelihood estimation was selected for its superior parameter estimation and statistical testing capabilities [27]. Varimax rotation was used to maximize the variance of squared factor loadings, creating a simpler and more interpretable factor structure that approaches the Thurstone principle of simple structure [28,29]. This orthogonal rotation preserves factor independence, providing greater statistical stability and replicability compared to oblique methods [29]. The analysis extracted two factors: personal values concerning physiological impacts and personal values concerning psychological impacts.

In addition, the Chinese versions of the Decisional Conflict Scale (DCS) and the International Prostate Symptom Score (IPSS) were validated and used in this study (Multimedia Appendix 2). The short form of the Spielberger State-Trait Anxiety Inventory (STAI) and the Decision Satisfaction Questionnaire were translated into Chinese by 2 independent bilingual experts. Comprehensive details of all research instruments are provided in Multimedia Appendices 3 and 4.

This study used a closed, face-to-face recruitment strategy at Taiwan St. Joseph Hospital. We recruited men who were eligible if they (1) were aged 50 years or above, (2) presented for routine care between October 2017 and April 2018, and (3) were able to provide informed consent and communicate in Mandarin Chinese. This age threshold aligns with US Preventive Services Task Force and American Urological Association guidelines recommending PSA screening discussions starting at age 50, with SDM particularly emphasized for men aged 55‐69 years [30,31]. Men were excluded if they had (1) any documented psychiatric disorder that might impair decision-making capacity, (2) prior PSA testing (to ensure treatment-naïve status), or (3) a history of prostate surgery or prostate cancer. Participant identity and eligibility were verified against electronic medical records by research staff to prevent multiple entries. Notably, computer and internet literacy were not required; although the intervention used a web-based interface, all assessments and digital components were administered face-to-face by trained research staff. This supervised approach ensured accessibility for older adults regardless of technological proficiency and guaranteed that the study team had direct contact with all participants.

A total of 520 participants were initially recruited for model development. After excluding 13 individuals due to low-quality responses—defined as ten consecutive identical scores or rhythmic scoring patterns—the remaining 507 participants formed the model establishment group (MEG) for comprehensive training and validation. To ensure progressive optimization, the ML model was iteratively retrained after every batch of 20 new participants. This development phase continued until the model achieved a predefined performance criterion (area under the curve [AUC] ≥0.80), indicating sufficient predictive accuracy for potential clinical application.

Subsequently, we conducted a parallel-group randomized controlled trial to evaluate the impact of ML-based decision aid recommendations. An additional 380 participants were recruited and randomly assigned with a 1:1 allocation ratio to either the ML suggestion group (MLSG, n=190) or the control group (CG, n=190). The randomization sequence was computer-generated and integrated directly into the web-based study platform to ensure allocation concealment. Group assignment was revealed to the research staff only after each participant’s baseline data had been fully entered and locked, thereby preventing any foreknowledge of the allocation sequence. The MLSG received standard SDM support supplemented by ML-assisted recommendations, while the CG received standard SDM alone. After excluding participants with low-quality responses, the final analysis included 185 participants in the MLSG (n=5 excluded) and 182 in the CG (n=8 excluded). Prior to study commencement, all participating medical staff completed a standardized 10-minute training session to ensure proficiency in navigating the DA. The study was conducted in accordance with the original protocol, and there were no changes to the methods (including eligibility criteria and outcomes) after trial commencement.

This study was conducted in accordance with the Declaration of Helsinki and relevant ethical guidelines for research involving human subjects. The study protocol was reviewed and approved by the Institutional Review Board of Fu Jen Catholic University (approval C105142), and written informed consent was obtained from all participants before enrollment. The trial was registered in the Chinese Clinical Trial Registry (ChiCTR 2000034126). Only the patient medical record number was retained to protect privacy, and all study data were accessible exclusively to authorized research personnel; nonresearch staff were not permitted to access the dataset. To promote transparency and fairness, participants received a cash payment of NT $300 for their participation, which is equivalent to approximately US $9.38 based on an exchange rate of US $1=NT $31.97.

Baseline characteristics of the MEG cohort (n=507) are shown in Figure 4 and summarized in Table 1, with comprehensive results provided in Multimedia Appendix 11. Self-reported knowledge of PSA testing (KnowPSA score) was significantly higher in the accept group (mean 2.23, SD 0.81) than in the not now group (mean 2.07, SD 0.69; P=.05). Marital status distribution also differed significantly between the 2 groups (P<.001), with a notably higher proportion of married individuals in the accept group. Regarding the IPSS, the accept group reported significantly higher scores for IPSS items 1‐5 (P values ≤.02) and item 7 (P<.001). Furthermore, a significant difference was observed in the IPSS Q3 (quality of life) score (P=.008), whereas no significant difference was found for IPSS item 6. For the IPPI questionnaire, the accept group scored significantly higher across all domains except for domain G (P=.34).

Regarding decision-making priorities, item A was most frequently identified as the primary factor in the accept group (n=32, 24.62%), which was significantly higher than in the not now group (14.85%; P=.01). In contrast, item B was more commonly prioritized as the first concern by the not now group (19.89%) compared to the accept group (7.69%; P=.001). Significant between-group differences were also noted for items D (P=.004) and E (P<.001). For the second most important factor, item E was significantly more endorsed by the accept group (20.77%) than the not now group (4.24%; P<.001), a trend also observed for item F (13.08% vs 3.18%; P<.001). Conversely, item H was less favored in the accept group (3.08%) compared to the not now group (11.41%; P=.004).

As for the third most important factor, item C was the leading choice overall (22.09%). However, item F was significantly emphasized in the accept group (14.62%) relative to the not now group (2.39%; P<.001). Additionally, significant between-group variations were observed for items E (P=.009) and J (P=.01). These findings highlight the differential influence of items A, B, C, E, F, and J on decision-making, suggesting nuanced variations in patient priorities as captured by the IPPI.

Figure 5 shows the multivariable LGR results for participants in the MEG group, presenting odds ratios (ORs) and 95% CIs for each covariate, with PSA screening uptake as the outcome. Statistically significant estimates (P<.05) are highlighted in blue, whereas nonsignificant estimates are shown as hollow markers. In this model, KnowPSA (OR 0.54, 95% CI 0.34‐0.86; P=.01) and IPSS_7 (OR 0.61, 95% CI 0.43‐0.87; P=.006) were statistically significant. Moreover, several IPPI subscales were also significantly associated with the outcome, including IPPI_A (OR 0.66, 95% CI 0.47‐0.93; P=.02), IPPI_D (OR 0.62, 95% CI 0.43‐0.90; P=.01), IPPI_F (OR 0.53, 95% CI 0.35‐0.81; P=.003), IPPI_G (OR 0.53, 95% CI 0.35‐0.81; P=.003), and IPPI_J (OR 0.69, 95% CI 0.47‐1.00; P=.048). Together, these results suggest that both knowledge-related and psychological factors are associated with PSA screening decisions in this group. Among the significant predictors, KnowPSA, IPPI_A, IPPI_D, IPPI_F, and IPPI_G had the smallest ORs, and their 95% CIs were entirely below 1, supporting their independent associations with the outcome. Overall, these findings underscore the importance of addressing both informational and psychological dimensions when supporting patient decision-making in the context of ML-enhanced guidance.

A total of 367 participants were included in the analysis, consisting of 185 in the MLSG group and 182 in the CG group (Table 2). Detailed results are provided in Multimedia Appendix 12. Regarding lower urinary tract symptoms measured by the IPSS, significant between-group differences were observed in specific domains. Specifically, scores for IPSS item 3 (0.93±1.28 vs 0.63±0.88; P=.008) and item 4 (0.93±1.33 vs 0.58±0.85; P=.003) were significantly higher in the MLSG group compared with the CG group.

When ranking primary decision-making concerns, item A was most frequently identified as the top priority overall (31.06%). Notably, a significantly larger proportion of the CG group selected item A as their first concern compared with the MLSG group (36.76% vs 25.27%; P=.01). Regarding subsequent priorities, item E was the most common second-ranked choice overall (19.89%), while item F was the most frequently selected third-ranked priority (22.89%).

Table 3 summarizes the impact of ML-based recommendations on participants’ final decision-making outcomes. In the CG, 24.18% of participants selected “accept” as their final decision, while 75.82% chose “not now.” In contrast, participants in the MLSG who received an “accept” recommendation were significantly more likely to select “accept” compared to those in the CG (50.75% vs 24.18%; P<.001).

Furthermore, when the ML system suggested “not now,” only 17.8% of participants chose to “accept,” which was lower than the baseline acceptance rate in the CG. These findings indicate that ML recommendations exerted a statistically significant influence on participants’ decisions, substantially increasing the likelihood of selecting the recommended option.

For interpretability, lower scores on the DCS and Sa scale reflect better outcomes (ie, reduced conflict and enhanced satisfaction). Regarding the primary outcome, the total DCS score was significantly lower in the MLSG compared with the CG (MD –3.77, 95% CI –5.55 to −1.99; Cohen d=–0.44; P<.001), indicating superior overall decision quality. Item-level analysis revealed significant improvements in several key dimensions: participants in the MLSG reported greater perceived decision support (DCS7: adjusted P=.03), more adequate advice (DCS9: adjusted P<.001), higher decision confidence (DCS10: adjusted P=.03; DCS11: adjusted P<.001), and found the decision-making process significantly easier (DCS12: adjusted P<.001).

For decision satisfaction, the MLSG demonstrated a large and statistically significant improvement in the total Sa score (MD –7.38, 95% CI –8.54 to −6.18; d=–1.29; P<.001). Notably, the MLSG reported significantly higher satisfaction across all 5 items (adjusted P values <.001, Cohen d ≤ –1.10). These gains were most pronounced in information adequacy (Sa1: MD −1.46, 95% CI –1.72 to −1.20), confidence that the decision was the best personal option (Sa2), alignment with personal values (Sa3), and a stronger intention to implement the chosen decision (Sa4).

Regarding anxiety, although the total STAI score did not differ significantly between groups (P=.37), item-level analysis identified significant improvements in specific psychological states. Participants in the MLSG reported feeling significantly less worried (STAI 6: MD –0.98, 95% CI –1.20 to −0.76; d=–0.89; adjusted P<.001) and significantly calmer (STAI 1: MD 0.30, 95% CI 0.06 to 0.54; d=0.25; adjusted P=.01) compared with the CG. Detailed item-level comparisons are presented in Table 4.

In this randomized controlled trial of 367 men aged 50‐75 years, adding personalized ML recommendations to a structured SDM workflow was associated with significantly improved decision quality for PSA screening. Supported by a high-performing prediction model (AUC 0.93), the intervention group (MLSG) reported lower decisional conflict and reduced anxiety compared with standard care. Specifically, participants receiving ML-supported recommendations experienced greater calmness (STAI 1: adjusted P=.01) and less worry (STAI 6: adjusted P<.001). Beyond anxiety reduction, the intervention produced large, consistent improvements in decision satisfaction across all domains (Cohen d >1.10; adjusted P values <.001), including perceived adequacy of information and alignment with personal values.

Importantly, the intervention was designed to complement—rather than replace—clinician counseling. The ML output was presented as an advisory “second opinion” and reinforced by standardized clinician communication, ensuring that the final screening decision remained patient-directed. This approach is particularly relevant for older adults, who often face cognitive load and uncertainty when weighing the complex trade-offs of PSA screening [32].

Although conventional prediction approaches (eg, LGR) can identify determinants associated with PSA screening intentions and uptake, they are generally not designed to deliver individualized, real-time recommendations at the point of decision-making [33]. Unlike prior decision-aid evaluations that focused primarily on presentation format or generic personalization, our intervention explicitly targeted decisional conflict and anxiety [34,35]—key barriers to informed choice in adults and aging populations [3,36]. Prior studies also suggest that education, age, symptom-related, and psychosocial factors influence screening behavior, and that incorporating constructs such as self-efficacy and emotional well-being may improve the use of decision support tools [37-39]. Watson et al [33] reported that men’s intention to undergo PSA testing was closely related to baseline knowledge, attitudes, and perceived risk, underscoring the central role of informational and psychological determinants.

Consistent with this literature, our multivariable LGR analysis in the MEG cohort confirmed that screening decisions were driven by a complex interplay of informational and preference-related factors. Specifically, higher PSA-related knowledge (KnowPSA) was independently associated with lower screening uptake (OR 0.54; P=.01). This inverse association suggests that participants with limited understanding may be more inclined to accept screening without fully weighing potential harms (eg, false positives, overdiagnosis), whereas greater awareness appears to foster a more cautious approach, reducing uninformed compliance.

In addition to knowledge, clinical symptoms and personal values were robust predictors of decision-making. Greater urinary symptom severity (IPSS item 7; OR 0.61; P=.006) was associated with lower screening uptake, possibly reflecting a prioritization of symptom management over preventive screening in symptomatic individuals. Furthermore, several values-clarification subscales (IPPI items A, D, F, G, and J) significantly predicted screening behavior, indicating that participants who placed higher importance on specific physiological, psychological, or social impacts were more likely to decline or defer screening.

These results suggest that when patients are equipped with sufficient knowledge and an opportunity to clarify their values, they may become more discerning, potentially leading to informed decisions to decline screening based on their personal context. This aligns with the core tenet of SDM, which prioritizes value-concordant choices over passive adherence. The robust predictive power of these factors underscores the use of our ML model in capturing nuanced, individualized determinants, thereby moving beyond generic guidelines to support truly patient-centered care.

Recent evidence suggests that fewer than 25% of men aged 40 and older report discussing the benefits and harms of PSA testing with their providers, a gap even more pronounced among those with lower educational attainment [40]. The selection of the delivery mode for DAs is critical to addressing this disparity. Given that the target population for prostate cancer screening primarily consists of older adults, web-based DAs may present usability challenges due to age-related declines in fine motor control or limited digital literacy [41]. To mitigate these barriers, our intervention used a facilitated delivery approach. By providing personalized, accessible recommendations through a supported interface, our ML-driven decision aid ensures that screening decisions align with individual values rather than being driven by incomplete information or physician preference alone. Compared with conventional decision aids, ML recommendations significantly enhanced decisional confidence and yielded modest yet consistent increases in screening acceptance. This aligns with prior evidence that technology-supported tools can shape patient preferences and improve decision quality [24,42].

However, we emphasize that ML should be viewed as a decision-support tool, not a decision-maker. The modest increase in screening acceptance observed in the MLSG suggests that the tool facilitated decisive action, but ethical implementation requires ensuring that this influence does not coerce patients. Future integration into routine practice must prioritize transparency and physician training to maintain the delicate balance between algorithmic guidance and patient autonomy.

Taken together, these findings indicate that embedding personalized, ML-driven recommendations within SDM frameworks may not only improve decision quality but also enhance psychological outcomes. Before integrating ML-assisted tools into routine clinical practice, further research is warranted to evaluate their psychological and physiological safety. Such investigations are essential to determine not only the efficacy of ML recommendations in enhancing decision quality but also their broader implications for patient autonomy and well-being [43].

Several limitations should be noted. First, while we provided uniform technical assistance, we did not formally assess baseline eHealth literacy or track the frequency of support required, limiting our ability to analyze effects based on digital proficiency. Future studies should incorporate validated measures like the eHealth Literacy Scale. Second, although our model demonstrated high discrimination (AUC >0.90), calibration metrics (eg, Brier score) were not assessed during the development phase; future work should include calibration plots to ensure risk estimates are accurate across subgroups. Third, unmeasured confounding remains a possibility, as factors such as physician influence or access barriers were not fully captured.

Finally, the current user interface provides binary recommendations (agree vs not now) without visualizing feature contributions. Integrating Explainable Artificial Intelligence techniques, such as Shapley Additive Explanations values, will be critical for future iterations to enhance transparency and trust.

This study demonstrates that integrating personalized ML recommendations into SDM frameworks can enhance decision quality while simultaneously reducing psychological burdens, such as anxiety and decisional conflict. Participants exposed to ML-supported recommendations reported greater confidence, reassurance, and satisfaction with their choices, underscoring the unique psychological value of individualized algorithmic guidance in prostate cancer screening.

Importantly, these findings suggest that ML-based decision support should be regarded as a complement to, rather than a replacement for, traditional decision aids and physician-patient discussions. By enriching the decision-making process with nuanced, personalized insights, ML has the potential to foster more supportive and less distressing decision environments.