The authors are researchers at the mHealth Impact Lab [23] who contract with Routinify, Inc, [24], a vendor that delivers the Convoy-Pal intervention. Routinify offers a variety of software and hardware tools that are publicly available; costs vary based on the tools provided and can range from US $50 to $100 per patient. In this case, Routinify assists with the delivery of the Convoy-Pal intervention to older adults and their social convoys. However, Routinify is only permitted to deliver Convoy-Pal in contract with the mHealth Lab and is not engaged in the clinical research (ie, they are not involved in the study instruments, data collection, management, analysis, or designing the protocol).

Convoy-Pal is designed with the following care domains: physical, psychological, social, spiritual, near end of life, ethical and legal, and knowledge about palliative care overall. Convoy-Pal includes a palliative care assessment with self-monitoring and resource tools for each domain. For example, the near end-of-life aspects of care (Figure 1) includes information regarding grief support and self-care and provides an opportunity for life review activities. This also includes resources on how to communicate unaddressed concerns and identify a support group for social support. Convoy-Pal tools and content are designed to be delivered via WellAssist by Routinify, Inc. WellAssist is a personal point-of-care app and associated internet-connected medical devices. The app’s core is based on behavioral modifications in line with the overall plan of care. The app is designed so that all members within the social convoy can access and use the Convoy-Pal intervention.

Study procedures were approved by the Colorado Multiple Institutional Review Board (number 18-0973). All participants electronically consented to participate. Convoy-Pal collects assessments, including smartwatch-captured vital information, regarding mental health and overall well-being data for the older adult patient and caregivers. It is ethically sound to obtain consent from older adults to share their health information with their caregivers as we did for this study; however, older adult patients also have the option not to share their health information with their caregivers if desired.

Potential patient participants were identified from the UCHealth University of Colorado Hospital health system’s electronic medical record. Potential participants were aged at least 65 years at the time of recruitment and had been hospitalized at the UCHealth Hospital more than 2 times for HF in the year prior (January 2020-2021). Participants were currently living in their homes and receiving follow-up HF care. We mailed a study invitation letter with an opt-out contact option. Patients who did not opt out were then contacted by phone for recruitment and asked to self-identify social convoy caregivers.

Two research coordinators (JPV and IM) interviewed participants via Zoom to gain feedback on Convoy-Pal. Participants were exposed to sections of Convoy-Pal throughout the interview process, which lasted between 40 minutes to 1 hour. Participants were shown the Convoy-Pal hardware, which consisted of the tablet, watch, and charging station, and the web-based system portal and mobile app during the interview. Participants were also shown Convoy-Pal features such as goal setting and planning, monitoring options, daily check-in and calling features, convoy caregiver assessments, and palliative care resources. During the exposure to Convoy-Pal, participants completed a self-report acceptability measure and were asked to provide open-ended feedback.

For self-report acceptability, participants completed the Mobile Application Rating Scale: User Version (uMARS) survey [25]. The uMARS survey comprises 4 objective quality subscales—(1) engagement with the app, (2) functionality and users’ perceived functioning of the app, (3) aesthetics, and (4) users’ perception of the quality of the information [25]—to determine app quality mean score. uMARS has 2 optional subscales that can be used depending on the aims of the research. These 2 subscales are the app subjective quality scale, which can be reported as individual items, and the perceived impact scale, which obtains information on the knowledge, attitudes, and behavior change toward improving health behavior [25]. All items are assessed on a 5-point scale, with a uMARS score of 5 considered excellent while a score of 1 is inadequate [26]. The uMARS is shown to have high interrater reliability for evaluating the quality of mHealth apps on well-being, for example [25,27].

For open-ended feedback, the tablet and smartwatch were introduced and displayed over Zoom to the participants. We used a semistructured interview guide (Multimedia Appendix 1) to ask the participants questions and their opinions regarding the hardware, goal setting and planning, monitoring options, daily check-in and convoy calling options, portal and mobile app for convoy, convoy assessments, and convoy resources. Notes, recommendations, and opinions from participants were archived into Qualtrics (Qualtrics) [28], data management software, as the interview was being conducted. The final data captured were stored and saved in Qualtrics with their study ID numbers.

The uMARS survey data was analyzed using Excel (Microsoft Corp) calculation mechanisms and descriptive frequencies including mean scores for both caregivers and patients. Once all the interviews were complete, the qualitative data were moved from Qualtrics to NVivo 12 (QSR International) [29], a qualitative research software package, for analysis. A preliminary codebook was created, incorporating explicit domains from the interview guide (deductive themes) by a research assistant (JPV). A descriptive qualitative approach [30] was then used to identify themes and subthemes [31,32]. The codebook and final data interpretation were discussed in a group with all authors. No member checking was conducted.

We recruited 26 participants (16 patients and 10 caregivers) from the University of Colorado Denver and its affiliate, the University of Colorado Hospital. Patients and convoy caregivers participated together or separately depending on the patient’s ability to participate in the interview. Patients were primarily males (9/16, 56%), White (14/16, 88%), and had a mean age of 76 (SD 5) years. Caregivers were predominantly female (7/10, 70%), White (7/10, 70%), and had a mean age of 71 (SD 10) years. Patients were married (12/16, 75%) and had a postgraduate degree (8/16, 50%), with 44% (7/16) having an income of US $30,000 or more and 82% (13/16) owning an iPhone, Android, or a regular or basic phone (Table 1). Similarly, most caregivers were married (8/10, 80%) and had a college or postgraduate degree (7/10, 70%), with 50% (5/10) making US $30,000 or more; 40% (4/10) of caregivers chose not to answer the question regarding their income. All of the caregivers owned an iPhone, Android, or a regular basic phone (Table 1). Due to small cell sizes, demographic categories were collapsed and are not reported in the table.

Convoy-Pal is designed to increase access to palliative care resources and self-management in the setting of HF. Acceptability testing is essential because it results in a better quality product. In this acceptability and usability study, the Convoy-Pal is considered acceptable and a good quality app, based on the uMARS scores among older adults with HF and their caregivers. Older adult patients and caregivers also provided recommendations for improving Convoy-Pal, which included adding comment sections, designing a checklist for medications, including thresholds on graphs for interpretation, and adding features such as fall detection. Based on this feedback, authors will update and continue to assess Convoy-Pal for usability and feasibility.

Although there are few high-quality HF mobile apps [33] assessed for acceptability, functionality, and efficacy [33,34], our findings are supported by other empirical studies. For example, palliative care patients found a mobile mortality risk tool acceptable to use [35]. Additionally, using wearables for monitoring palliative care was also feasible [36,37], a tool that Convoy-Pal offers with its smartwatch. Similar to the feedback provided for Convoy-Pal, a commentary article, systematic meta-review, and qualitative study [15,38,39] report the need to track relevant information, receive education pertinent to health for older adults, and provide information sharing such as medication use.

Aside from HF apps specifically, digital health interventions overall have the potential to improve the accessibility and effectiveness in palliative care, as reported by a recent systematic meta-review [38]. Palliative care is one area where technologies are increasingly being deployed. Although leveraging existing resources for palliative care is one approach, mHealth interventions targeting palliative care enable patients increased access to this resource without spending time or traveling to locations [40]. mHealth palliative care allows older adults to participate in and govern their care. For example, they do this by self-reporting symptoms and needs, which improves communication with providers and caregivers [38,41,42]. Traditional palliative care resources do not provide a self-governing element in this unique way.

Additionally, HF mobile interventions rarely target the social convoy or palliative care domains. Therefore, Convoy-Pal would contribute to the advancement of palliative care and HF mHealth while also advancing a team approach to information sharing and targeting family- and caregiver-specific issues [43]. Convoy-Pal has the potential to support older adults with HF and their social convoy in the management of physical, psychosocial, and spiritual concerns.

First, due to university and state COVID-19 restrictions, research assistants were not able to meet with older adult and caregiver participants to physically interact with Convoy-Pal on the Routinify tablets or complete assessments in person. Researchers therefore collected acceptability and usability data by displaying Convoy-Pal and all its features remotely to participants for about 1 hour through Zoom. Second, the uMARS survey, for this reason, was modified by our team to reflect the following 2 optional responses for all subscales of the survey: (1) “Optional: Missing due to lack of time with app” and (2) “Optional: Did not feel comfortable answering.” If the participant did not feel comfortable answering the uMARS questions due to their belief that there was not enough time to explore Convoy-Pal, then they could select either optional response. The minor modifications made to the uMARS had not previously been tested and therefore may have reduced the validity of the original items. Physical interaction with the hardware may have yielded additional user feedback. Last, the study was further limited by small sample size and a single health system as well as lack of diversity representative of the local community. Acceptability and usability of Convoy-Pal may differ in other regional areas and varying access to health care.

HF is a leading cause of death in the United States, and mHealth provides opportunities for patients and their social convoy to participate in palliative care. In our study, 16 older patients and 10 caregivers were interviewed and asked to complete a uMARS assessment and provide open-ended feedback. Overall, older patients and their caregivers perceived Convoy-Pal as acceptable with good usability. Although in-person usability testing is needed, Convoy-Pal was perceived acceptable and may ultimately increase access to palliative care resources and facilitate self-management among older adults with HF and their caregivers.