This study comprises a self-administered cross-sectional staff survey and a pre-post cross-sectional patient survey. The pre-post patient surveys were conducted among 2 different patient groups. The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guideline [18] was used to strengthen the reporting process (Checklist 1).

The study involved 4 piloting public tertiary hospitals representing 3 out of 7 geographical clusters of Hong Kong. The PDIS was introduced in a phased manner within the geriatric and medicine department in January 2018 [16]. For the staff survey, all physicians and nurses involved in the PDIS implementation were invited to participate in the study. The surveys were conducted at least 6 months after the PDIS implementation, which spanned from August 2018 to June 2019. Paper-based promotion leaflets, invitation letters, and questionnaires were distributed through designated coordinators in each hospital. This survey was conducted anonymously and on a voluntary basis.

For the patient survey, the sample size was determined based on the inpatient discharge statistics for patients aged 65 years or older in 2015, as provided by the Hospital Authority. In order to achieve a precision level of ±4% with a 95% CI, a minimum of 1450 respondents was required for pre-post rounds. Assuming a 50% response rate, at least 2900 patients were randomly selected from the discharge records for each round. Responses from caregivers acting as surrogates were accepted if patients were unable to respond independently. Readmitted cases and day patients were excluded. Within 14 days of their discharge, patients were contacted by telephone. The pre-post survey was conducted from June to December 2017 and May to December 2018, accordingly.

The staff survey used English and the patient version used Chinese. We used English in staff survey because English is their working language and they are proficient in English. Furthermore, staff surveys are typically conducted in English in Hong Kong, so it was assumed that participants would feel comfortable with this language. To ensure language did not pose a barrier to participation or affect the responses collected, we decided to use Chinese for the patient survey. We do not anticipate any language concordance issues because the researchers who were responsible for collecting patient responses were well-trained before the survey to ensure they can correctly convey the meaning of the questions.

The staff survey collected information on providers’ practicing behaviors and user experience, adapted by the technology acceptance model (TAM) [19]. TAM has been designed to investigate why individuals adopt a specific technology and has been widely used in different settings [20].

According to the TAM, the perceived usefulness of the technology can impact users’ behavior intention, which can be a determinant of users’ actual behaviors. We added another domain named design quality to capture providers’ perceived information quality of the tool. We also measured the overall satisfaction of the PDIS experience. Existing literature suggests that design quality can significantly influence staff adoption and implementation of the technologies [21]. The traditional TAM component of “Perceived Ease of Use” was not fully applicable in this context, as our primary aim was to compare the perceptions of patients and staff regarding their user experiences and perceptions of the PDIS. However, “Perceived Ease of Use” tends to be more relevant to staff, given that staff interact with the information system while patients interact with a paper-based version rather than a technology. Consequently, we adopted “Design Quality” as a domain to facilitate a meaningful comparison of perspectives between these 2 groups. In addition, we sought to streamline our survey to enhance response rates, particularly among staff who may have limited time to participate. Thus, we decided to use design quality as a domain in place of perceived ease of use.

We hypothesized that (1) design quality would have an impact on perceived usefulness, satisfaction, and behavior intention; (2) perceived usefulness would impact satisfaction and behavior intention; (3) satisfaction would impact behavior intention; and (4) behavior intention would impact actual behavior (Figure 1).

The pre-post patient surveys consist of the following sections.

The statistical analysis was performed utilizing R version 4.0.5 (R Project for Statistical Computing) and Stata version 18 for Mac (StataCorp). Provider and patient demographic information were summarized as means and percentages using descriptive statistical analysis. The staff survey and patient survey were analyzed separately. For the staff survey, subgroup analysis was conducted to evaluate the differences among physicians and nurses. The practicing behavior frequency and PDIS experiences were compared using the Pearson chi-square test and Mann-Whitney U test. The Mann-Whitney U test was used due to the outcome variables were skewed. In order to understand the relationship between the determinants of the behaviors like design quality, perceived usefulness, behavior intention, satisfaction of the PDIS, and the actual behavior, as well as the relationship between different determinants, the covariance-based structural equation modeling (CB-SEM) was applied. The goodness of fit index, comparative fit index (CFI), root mean square error approximation (RMSEA), Tucker-Lewis index (TLI), and standard root mean squared residual (SRMSR) were used to evaluate the goodness-of-fit of the model. The direct effects, indirect effects, and total effects between the core variables were assessed using a bootstrapping approach (n=2000). Standardized path coefficients (β) were used to estimate the path relationships. Statistical significance was set at P=.05.

For the patient survey, changes in patient experience regarding medication information before and after PDIS implementation were compared using the Mann-Whitney U test due to the outcome variables were skewed. The difference between self-reported side effects encounters and compliance was assessed using the Pearson chi-square test.

To identify the disparity between staff and patient. the shared questions related to PDIS experience (eg, information clarity, adequacy, and usefulness, and the overall experience of the PDIS) in the staff and patient postsurveys were compared using the Mann-Whitney U test. For the qualitative comments from both staff and patient, thematic synthesis [26] was applied to identify and compare the common themes in free-text comments for the PDIS program between patients and providers. Furthermore, 2 coders independently coded all the utterances of patients and staffs to generate the initial codes to achieve consensus on coding. The disagreement were formally resolved through several discussions with a senior researcher experienced in qualitative study. After that, the inductive analysis was performed by one coder to generalize initial codes into overarching themes. The initial summary themes were developed and discussed with the senior research to reach consensus. The final list of themes was reviewed and consented by the research team. The theme list were identical for patient and staff and the frequency of each theme were counted in order to compare the pattern for patients and staffs.

Ethical approval has been provided by the Joint Chinese University of Hong Kong – New Territories East Cluster Clinical Research Ethics Committee in compliance with the Declaration of Helsinki (CREC 2019.436). For the staff survey, implied consent was applied as participating staff members returned the completed questionnaires to the research team. For the patient survey, verbal consent was obtained from patients over the phone before the survey. All data were deidentified to protect participants’ privacy and confidentiality.

A total of 1375 providers completed the survey with a 76% response rate, comprising 966 (72%) female participants, 650 (50%) participants aged 18‐29 years old, 595 (50%) participants with 0‐5 years of working experience, and 1216 (88%) nurses (Table 1).

From the patient side, we received 2353 valid responses, including 1109 (47%) and 1244 (53%) responses collected through the pre- and postsurveys, respectively. The response rate was 55.5% for the presurvey and 59.4% for the postsurvey. The demographic composition was similar between the pre- and postsurvey groups, except that 6.4% (presurvey group: 853/1106; postsurvey group: 1023/1226) more participants were receiving the government subsidy in the postsurvey group (P<.001) (Table 2).

The comparative evaluation showed that patients consistently provided significantly higher ratings for their overall PDIS satisfaction compared with providers (mean 8.28, SD 1.60 vs mean 6.29, SD 1.88, respectively; P<.001) (Figure 2). Specifically, patients reported higher ratings for information clarity (mean 8.58, SD 2.50 vs mean 6.54, SD 1.86, respectively; P<.001) and usefulness of the PDIS (mean 8.14, SD 2.46 vs mean 6.50, SD 2.02, respectively; P<.001). On the contrary, patients were inclined to receive more information through the PDIS (mean 5.33, SD 1.35 vs mean 6.14, SD 2.08, respectively; P<.001).

Figure 3 displays the similarities and differences between patients and providers regarding their comments on PDIS experiences. A total of 538 comments were received (421 were from providers and 117 were from patients). The most frequently commented aspect was PDIS features for both providers (147/421, 35%) and patients (73/117, 62%). However, the specific area of the feature was different. Providers emphasized the need for broader coverage of the drug databases (60/147, 41%) and the lack of multiple language versions (59/147, 40%), while patients’ concerns revolve around the inconvenience of medication names and follow-up information being in English (45/73, 61%), the discomfort with medical jargon (9/73, 13%), and font size (7/73, 10%). In addition, providers frequently commented on the content of the PDIS form. For example, 27% (114/421) of the comments were related to the medication listed on the PDIS and emphasized the need for additional medication details such as medication changes, indications, and instructions.

Analysis of the practicing behaviors (Table 3) revealed that 72.1% (n=922) of the providers reported being able to consistently distribute the PDIS form to patients or caregivers, whereas 56% (n=667) stated they could consistently explain its content. Subgroup analysis demonstrated significant variations across different roles. Regarding distribution, 78% (n=915) of nurses reported always doing so, compared with 6.4% (n=7) of doctors (P<.001). Similarly, 57% (n=666) of nurses reported always explaining the content, compared with 4% (n=1) of doctors (P<.001).

Physicians and nurses indicated moderate satisfaction with the design quality and perceived usefulness of the PDIS to their work, as reflected by mean agreement scores ranging between 5.96 and 6.54 (Table 3). The subgroup analysis did not identify any differences between professional roles regarding user experiences. The CB-SEM analysis (Figure 4) showed that design quality significantly impacted their perceived usefulness (β coefficient=0.96, 95% CI 0.90-1.01) and behavior intention (β coefficient=0.14, 95% CI 0.06-0.21). In addition, perceived usefulness had a significant impact on behavior intention (β coefficient=0.48, 95% CI 0.26-0.70). Furthermore, behavior intention had a significant impact on the actual behavior (β coefficient=0.16, 95% CI 0.10-0.23). In addition, satisfaction can be significantly impacted by the design quality (β coefficient=0.24, 95% CI 0.08-0.40) and perceived usefulness (β coefficient=0.57, 95% CI 0.43-0.71). The structural equation modeling (SEM) model presents a good fit overall, with all indicators exceeding the recommended thresholds (Table 4). The results of the indirect effects and total effects can be found in Multimedia Appendices 2 and 3. In total, 3 mediating pathways were identified: (1) an indirect pathway from design quality through satisfaction to behavior intention (β coefficient=0.770, 95% CI 0.700‐0.841, proportion mediated=15.1%); (2) an indirect pathway from perceived usefulness through satisfaction to behavior intention (β coefficient=0.228, 95% CI 0.185‐0.272, proportion mediated=32.5%); and (3) an indirect pathway from design quality through perceived usefulness to satisfaction (β coefficient=0.544, 95% CI 0.482‐0.605, proportion mediated=30.8%).

Table 5 showed significant improvements in patient experience related to overall discharge information clarity (mean 8.18, SD 1.69 vs mean 7.93, SD 1.84, respectively; P=.002), adequacy (mean 8.15, SD 1.76 vs mean 7.92, SD 1.93, respectively; P=.01), and usefulness (mean 8.26, SD 1.70 vs mean 8.06, SD 1.80, respectively; P=.02). In addition, a significant increase was found in information adequacy for both side effects (mean 9.6, SD 2.0 vs mean 8.6, SD 3.4, respectively; P<.001) and warnings (mean 9.7, SD 1.8 vs mean 9.2, SD 2.7, respectively; P=.004). Only warning information reached statistically significant improvement in clarity (mean 8.77, SD 2.32 vs mean 8.45, SD 2.45, respectively; P=.03) and usefulness (mean 8.7, SD 2.45 vs mean 8.44, SD 2.39, respectively; P=.03).

No statistically significant difference was found between pre- and postsurvey groups in the percentage of self-reported medication compliance. Notably, the postsurvey group had a significantly lower percentage of self-reported side effects encounters (126/1083, 11.6% vs 111/1235, 9%, respectively; P=.04). Among participants who reported encountering side effects, the majority (96/111, 86.4%) still followed the medication instructions as prescribed.

This is the first study to compare the perceptions of older adult patients and health care providers regarding the use of large-scale EHR-based discharge communication tools with concordance measures. There was a noticeable difference in ratings between patients and providers, with patients giving higher ratings in terms of design quality and overall experience of this tool. Qualitative comments indicated that patients and providers have different areas of concern regarding this communication tool. Furthermore, from the health care providers’ side, inconsistent practicing behaviors were found, which were significantly influenced by the implementation intentions (represented as behavioral intentions in the SEM model), overall satisfaction, design quality, and perceived usefulness of the program. However, from the patients’ side, older adults who received the written summary reported improved experiences with discharge information, including information clarity, adequacy, and usefulness.

Providers assigned significantly lower scores to the perceived information clarity of the PDIS to their patients than patients themselves. This difference may be due to the providers’ concern about the challenges associated with older adults’ health literacy [27] and the potential negative consequences of sharing information on side effects, such as patients’ anxiety and non-compliance [28-30]. However, the improved patient-reported ratings of information clarity on medication warnings and overall medication information, significantly fewer side effects encounters, and no evidence of patient noncompliance found by our study and others [31] suggested that taking action is no worse than inaction but yields better outcomes, contrary to the biased perception held by staff members [32]. In order to address staff concerns, facilitate their implementation, and fulfill patients’ needs, rephrasing risk information by using lay language, shorter sentences, supplementing verbal descriptions with visual aids, and presenting medication benefits along with side effects can be considered [33-35].

Providers rated information usefulness for patients or caregivers lower than patients in this study, further impacting their implementation. The lower beliefs on the value of the communication tool for patient care from the provider side may be due to the beliefs that patients may not effectively follow the advice due to a lack of skills or inability to recall, despite clinicians appropriately delivering the instructions [36]. Therefore, it is suggested that using cognitive aid strategies such as teach-back techniques, repetition, demonstration, and reducing the complexity of the information to enhance patients’ capacity to perform self-care tasks and recall of information [10]. The discrepancy can also be attributed to providers’ lack of awareness regarding patient needs, which was also reported by previous studies [37]. Our study results, as well as other research [38], suggest that patients view information about medication side effects and warnings as crucial when making decisions about seeking professional assistance. Patient-provider information gap may not only lead to patient dissatisfaction but also levy stress on providers [37]. Therefore, it is important to leverage patient voices as credible sources and build long-term patient-provider relationships to address this gap.

It is important to note that older adults tend to rely more on health care providers and perceive them as trustworthy sources of information, as reflected in higher satisfaction with received medication information among older adults than younger individuals in a previous study [39]. Therefore, it is crucial to address provider-reported barriers in PDIS implementation to ensure a higher level of program satisfaction and implementation fidelity. Our study found that the design quality and perceived usefulness to providers’ work can hinder their implementation, which is in accordance with previous studies [40,41]. This can be attributed to the low compatibility of the service with providers’ existing workflow [21]. Our study found that providers expressed a preference for enriching the PDIS content with additional medication elements, such as medication changes, indications, and instructions, indicating their priorities for discharge education are not fully met. This suggested that involving frontline implementers as program designers and developers is crucial for program fit, staff self-efficacy, capacity, and commitment [21]. Other than the perceived usefulness and design quality of the program reported in our study, a comprehensive understanding of the complex elements involved in implementation, including context, stakeholders, and organizational factors, is needed [42]. This knowledge facilitates the creation of customized strategies and policies that have a higher likelihood of achieving success [43]. Therefore, conducting implementation research is essential in identifying the barriers and facilitators linked to the implementation of the PDIS to ensure providers’ user experiences are optimized, leading to improved patient access to high-quality care and maintaining a high level of patient experiences among the older adult population.

This study has several limitations. First, self-reported data may be subject to inaccuracies due to social desirability bias. However, including participants from diverse backgrounds may mitigate this limitation to some extent. Second, patient outcomes were not measured in this study, as our focus was on exploring and comparing patient and provider experiences with technology-based communication tools. Given the positive experiences reported by patients, future research could investigate clinical outcomes to further enhance the evidence base. Third, the pre-post survey design for a patient survey without a control group limits our ability to determine whether the observed changes are directly attributable to the PDIS. The cross-sectional design for the staff survey can limit our ability to establish causal inferences regarding factors influencing providers’ inconsistent implementation. Subsequent studies using longitudinal or experimental designs are warranted to understand the causal mechanisms and develop effective strategies to enhance staff performance. Finally, as we did not impose a strict designation of specific professional roles (doctor or nurse) for the tasks involved in PDIS implementation, there may be an underestimation of the effects of the determinants on behavioral intention and actual behaviors.

EHR-based discharge communication tools have the potential to improve the patient experience with discharge information. However, there is a notable difference in user perceptions between patients and providers. This difference may hinder the full benefits of the program for patients. These findings have implications for future research, particularly in implementation research, where barriers and strategies to enhance staff performance can be investigated. In addition, the study provides valuable insights for organizations seeking to improve patient-provider shared understanding of postacute care plans among older adult patients during hospitalization, particularly through technology-based interventions.