One in 6 adults living in the United States is an older adult (aged ≥60 y). This number is expected to double in the next 40 years [1]. While older adults possess the wisdom of experience and are often actively involved in taking care of themselves, their family members, and their communities, they also face a wide range of unique health challenges that come with aging [2]. A disproportionate number of older adults live with chronic health conditions: 85% have 1 chronic health condition, and 60% report managing at least two [3]. One in 4 older adults reports at least 1 fall every year, and falls remain a leading cause of death and injury among this age group [4]. These health issues—combined with social determinants of health such as older adults’ built environment, social context, and access to medical care—put older adults at risk for premature death and poorer quality of life [5]. While only accounting for 17% of the population, older adults make up 35% of health care costs, according to 2019 Medical Expenditure Panel data [6]. Furthermore, 86 cents of every dollar of health care spending goes toward chronic conditions [7,8], and the burden of chronic disease is unevenly borne by women; older adults; people of color; and people living in poverty who experience disparities in access to, and quality of, care [9].

To address these challenges, many health promotion programs have been created based on the Chronic Care Model [10], to improve access and quality of care through community-based programs that teach knowledge, skills, and self-efficacy to enhance older adults’ health and well-being in their daily lives. Among these are evidence-based programs (EBPs) that have been researched and recognized by national and federal agencies, such as the Administration for Community Living (ACL) and the Centers for Disease Control and Prevention (CDC) Arthritis Management and Well-Being Program, as being effective in promoting health outcomes through standardized interventions [11,12].

Before the COVID-19 pandemic, many of these EBPs were primarily offered in person. This was not possible during the pandemic due to safety guidelines regarding physical distancing. At the same time, there was an increased need for programs because older adults became more isolated and less physically active, and they had less access to an overtaxed medical system [13,14]. Seeing this need, EBPs quickly pivoted to remote delivery by mail, telephone, videoconferencing, or a combination of these modes. While researchers suggest that EBPs will work as intended using different forms as long as core functions are not modified [15,16], it is unknown whether program effectiveness is maintained when switching from in-person to remote delivery. To date, there have been limited studies investigating the effectiveness of remote EBPs that were originally designed to be offered in person [17-21]. As such, we conducted a longitudinal national outcome evaluation of several EBPs to assess changes in older participants’ health.

We used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) for equity framework [22] to evaluate the potential impact of remote EBPs on older adults’ health and well-being. The equity lens means that in addition to evaluating impact, we looked at outcomes across programs, sampled organizations that reach older adults who are underserved, reported EBP reach, and assessed whether there were any unintended consequences. As such, we used mixed methods to “give voice to participants as well as report statistical trends” [23]. We conducted a multisite single-group pre-post evaluation. Multiple sites were selected to facilitate the generalizability of the findings, and a single-group design was chosen to make it feasible to conduct the evaluation during the COVID-19 pandemic and due to the descriptive nature of this study.

This study was considered exempt from University of Washington Institutional Review Board review because the activities fell under category 2 with regard to quality improvement and program evaluation (STUDY00011549). Participants who completed the preprogram survey were given a US $10 gift card, and participants who completed the follow-up survey were given a US $20 gift card (we provided electronic gift cards unless participants requested a physical card). Providers who completed the survey received a US $10 electronic gift card.

EBPs are health promotion programs that have been evaluated and proven to be effective. We included 5 EBPs in our evaluation: Chronic Disease Self-Management Program (CDSMP), Diabetes Self-Management Program (DSMP), EnhanceFitness (EF), Chronic Pain Self-Management (CPSMP), and Walk With Ease (WWE). These EBPs are all currently recognized by the national social services agency for older adults (ACL) and public health agencies (CDC) [11,12]. Before the COVID-19 pandemic, these programs were offered in person in group formats; they were adapted as follows for remote delivery in response to the COVID-19 pandemic: videoconferencing (CDSMP, DSMP, and EF), telephone plus mailed materials (CPSMP), and self-directed mailed materials plus enhanced support (WWE).

These 5 EBPs were selected because they had sufficient program reach and represented different remote modes (telephone, videoconferencing, mail, or a combination of these modes) and health topics (chronic disease self-management, falls prevention, and physical activity). The adaptation process largely involved planned changes to modify delivery context (eg, smaller class size; adding a cofacilitator to support engagement with, and the usability of, technology; and providing telephone options for people without access to reliable internet or videoconferencing technology) as well as some unplanned changes that occurred organically during field experience in consultation with the program leads (rather than changes to program content) [24]. It should be noted that WWE had created an enhanced self-directed program before the pandemic; we included this program in the evaluation to assess different modes of remote EBP delivery, given partners’, policy makers’, and older adults’ interest in diverse ways to engage in remote health promotion. The enhanced self-directed WWE is delivered via a cohort, with a trained leader providing web-based motivation and support to individual participants during the program period. More information about the included programs is provided below (Figure 1) and on each program’s website, including guidance for remote delivery.

In early 2021, we recruited organizations delivering remote EBPs with a brief web-based interest survey and webinars through several networks: EBP training listserves, the CDC Arthritis Program and ACL grantees, the Evidence-Based Leadership Collaborative, and regional EBP networks. The organizations included social services, public health, and health care agencies that were largely community based, although some were located in clinical settings. We used maximum variation purposive sampling [25] to identify organizations of diverse sizes and types, varied provider characteristics, and from different geographic areas to aid the generalizability of the evaluation findings. This sampling prioritized organizations engaging older populations with low-income status with multiple chronic conditions who are vulnerable to both COVID-19 infection and poor health outcomes and care (eg, people of color, those with disabilities, and those living in rural areas). The eligibility criteria for organizations was the delivery of at least 1 of the 5 remote EBPs from January 2021 through March 2022.

We then used convenience sampling to invite all remote EBP participants to take part in the evaluation. Participant surveys were primarily conducted on the web using REDCap (Research Electronic Data Capture; Vanderbilt University) [26], with options to complete surveys by telephone or by mail. Participants were surveyed when they enrolled in the program and again 6 months after program enrollment (regardless of when they finished the program). At both time points, a link to the survey was emailed to participants, and they received up to 3 additional reminder emails and 3 reminder calls during the month of survey eligibility. In addition, community-based organizations encouraged baseline survey completion as part of program intake and orientation.

Furthermore, all EBP providers at participating organizations were invited to take part in a 1-time survey eliciting their perspectives regarding the impact of remote EBPs on participants as well as providers. The EBP providers included leaders (people who delivered remote programs directly) and managers (people who coordinated and oversaw program delivery).

We used joint display tables [52] and compared quantitative and qualitative results to see where the findings converged, diverged, or expanded [23].

A total of 586 older adults participated in the evaluation over the study period, of whom 289 (49.3%) completed the 6-month follow-up survey (n=25, 8.7% completed the survey over the telephone). The primary reason for noncompletion was our inability to contact the older adults by telephone or email; only 10 (1.7%) of the 586 older adults declined to participate in the follow-up survey after being contacted.

Table 1 shows remote EBP evaluation participant demographics for those who completed both baseline and 6-month follow-up surveys. The mean age of the participants was 65.4 (SD 12.0) years. The majority were female (241/289, 83.4%) and White (184/289, 63.7%). A little more than one-third (108/289, 37.3%) identified as people of color: American Indian or Alaska Native (2/289, 0.7%), Asian (12/289, 4.2%), Black or African American (77/289, 26.6%), and Latinx (17/289, 5.9%). One in 7 (41/289, 14.2%) lived in a rural area, and 1 in 3 (99/289, 34.3%) found it “somewhat hard” or “very hard” to pay for basics such as food and housing. Of the 289 participants, 113 (39.1%) lived alone, and 58 (20.1%) were providing caregiving. The participants reported a mean of 2.5 (SD 1.7) chronic conditions, with the most common being hypertension (145/289, 50.2%), arthritis (131/289, 45.3%), diabetes (130/289, 45%), and mental health conditions (80/289, 27.7%). The prevalence of all chronic conditions is provided in Table 1, including more rare but still impactful conditions such as Crohn disease and ulcerative colitis.

Participants who completed the 6-month follow-up survey had similar demographic characteristics and baseline health status as those who completed only the baseline survey, with a few exceptions. A larger proportion of the CDSMP and CPSMP survey completers identified as Black compared to non–survey completers (15/69, 22% vs 13/98, 13%, and 23/47, 49% vs 31/75, 41%, respectively). WWE survey completers were less likely to be living alone (18/40, 45% vs 24/41, 59%). EF survey completers were more likely to be caregivers than not (2/12, 16% vs 0/10, 0%). Finally, across all programs except EF, survey completers were less likely to be living in a rural area than non–survey completers (37/274, 13.5% vs 109/284, 38.4%).

A total of 198 remote EBP providers (n=123, 62.1% leaders; n=75, 37.9% managers) from 107 EBP organizations in 33 states participated in the evaluation. The majority of the leaders identified as female (113/120, 94.2%). Furthermore, 4.3% (5/117) identified as Asian, 12.8% (15/117) as Black or African American, 11.7% (14/120) as Latinx, and 0.9% (1/117) as biracial. One-quarter of the leaders (31/113, 27.4%) lived in rural settings, one-third (36/120, 30%) were caregivers, and one-third (37/123, 30.1%) had ≥2 chronic conditions. One-fourth (30/121, 24.7%) identified as certified health professionals and 43.3% (52/120) as community health workers, promotoras, or other lay health providers. The leaders had a range of experience in EBP delivery: a little more than half (65/123, 52.8%) had delivered both in-person and remote programs before the survey, while 36.6% (45/123) were conducting remote EBPs for the first time. In addition to completing the survey, 26 EBP administrators, managers, and leaders took part in qualitative interviews. Most of the interview participants (22/26, 85%) identified as female and worked at community or government organizations.

Our evaluation found that remote EBP participants showed improvements on various outcomes from program enrollment to 6-month follow-up, including their self-rated health, energy, sleep quality, loneliness, depressive symptoms, and technology anxiety, within and across programs. Some program participants also reduced their anxiety, pain, pain interference, physical activity and self-efficacy, and enhanced their technology usability. The quantitative findings yielded mainly small effect sizes. This may be due to the heterogeneity of the participants; in particular, participants who did not enter the program as lonely or inactive would have no room to improve over time. The qualitative findings suggest that remote EBP participants improved knowledge, attitudes, and skills on how to live healthier lives; reduced social isolation and loneliness; and gained better access to programs. In addition, providers shared that they too benefited from delivering programs remotely by staying connected, having access to technology, and improving their own health and well-being.

In some cases, the findings support previous research on the effectiveness of the remote EBPs that were part of our evaluation; for example, 1 study of 213 videoconferencing CDSMP participants in rural and remote Ontario, Canada, found similar improvements in self-rated health, energy, and psychological well-being (a measure related to depressive symptoms) 4 months after their last class [54]. Another study of 97 telephone plus tool kit CPSMP participants in Cleveland, Ohio, United States, also reported better pain outcomes immediately after program completion [17]. Furthermore, a study of self-directed WWE participants (n=270) in rural and urban North Carolina, United States, who were living with arthritis found that participants also reduced fatigue at follow-up 1 year after program enrollment [21]. Our findings are also comparable to those of similar health promotion programs, such as increased physical activity and reduced depression for a remote DSMP [55].

In other cases, our findings contrast with those of previous research; for instance, 1 study of remote-delivered EF [20] found that the participants (n=15) decreased their knee pain as measured by the Knee Injury and Osteoarthritis Outcome Score; however, all study participants had symptomatic knee osteoarthritis, and the postprogram outcome was measured directly after active intervention at 12 weeks. It may be that our evaluation participants did not significantly improve pain outcomes due to their less severe baseline pain. In addition, the study of CPSMP participants mentioned previously [17] found statistically significant improvements in sleep and depression; our evaluation participants too improved their sleep quality and reduced their depressive symptom severity, but this change was not statistically significant. Our differing findings may be due to our smaller sample size (47 vs 97), which lowered our power to detect significant change, or a longer follow-up period (6 months vs 6 weeks), during which improvements may have been attenuated.

Furthermore, it should be noted that our outcome evaluation was not designed to compare in-person EBP delivery with remote EBP delivery. We used a common set of outcome measures of interest to older adults, EBP delivery organizations, and policy makers; as such, some of these measures look at similar constructs as the in-person EBP effectiveness studies but use different instruments, and some of these measures that are newly being evaluated (eg, loneliness and depression) were not evaluated in the in-person EBP research studies. In addition, we used a different time period compared to studies that evaluated the effectiveness of in-person EBP delivery: we looked at changes between program enrollment and 6-month follow-up, regardless of program duration, whereas prior effectiveness research looked at pre-post change over time based on program duration [21,56-59]. Our sample sizes were also smaller than those of previous in-person EBP research studies; therefore, the lack of statistical significance may be due to a lack of power to detect change. That said, it may be helpful to contextualize our remote EBP outcome evaluation findings with those from in-person EBP outcome evaluations. Remote CDSMP evaluation participants reported improvements in self-efficacy, health, energy (fatigue), sleep quality, physical activity, and depression, all outcomes that were also reported in previous studies of in-person CDSMP [37,60-62]. Furthermore, remote CDSMP evaluation participants did not show the significant improvements in pain outcomes that were seen in research with in-person program participants. Remote CPSMP evaluation participants reported outcome improvements similar to those of in-person CPSMP participants regarding pain and pain interference [56,63], while our evaluation added evidence for remote CPSMP participant improvements in energy (fatigue), anxiety, and physical activity; these 3 outcomes were not assessed in in-person CPSMP studies. However, remote CPSMP participants did not report significant improvements in self-efficacy, as was reported in previous in-person CPSMP studies. Looking at the DSMP, both remote and in-person delivery participants showed reduced hypoglycemia symptoms, health, and depression [57]. Earlier research with in-person program participants found improved self-efficacy as well, and remote program participants showed improvement in terms of their physical activity days. For EF, both remote and in-person program participants reported improvements in self-rated health as well as physical activity [64,65]. This evaluation adds novel findings on improved sleep quality and the usability of technology for remote program participants. In addition, in-person EF participants have reported reduced depression in other studies [64]. Looking at WWE, both in-person and remote program participants have demonstrated improvements in health, fatigue, and physical activity [21]. Remote WWE participants reduced their loneliness, which was not assessed in previous studies of in-person WWE [21]. Previous research of in-person WWE also found participant improvement in pain and self-efficacy [21].

Although EBPs shifting to remote delivery was necessitated by the pandemic-related lockdown and other restrictions [24], the findings suggest that remote delivery can improve access to quality health promotion programs outside the COVID-19 pandemic context; for example, a caregiver for a person living with dementia can join a physical activity program from home to support their own health while not having to find respite care for their loved one, an older adult living with chronic pain can join a class even if they are experiencing elevated pain levels that would make it hard to leave even their bedroom, and a program leader can deliver the program across a region in Spanish to participants who may not have access to language-specific health promotion. This is similar to other studies of remote EBPs (eg, the value for people living with cancer to be able to participate from a distance despite living with weakened immune systems [66]).

Adapting the format and channels through which an EBP is delivered [67] is considered an appropriate modifiable intervention characteristic to better align with the needs and preferences of participants, as well as organizational and geographic contexts in which a remote option improves program feasibility and sustainability [68]. Implementation science increasingly recognizes the need for adaptations [69] to improve the intervention-context fit (eg, increasing both practical and value fit [70]; increasing EBP feasibility to a given context [71]; and making sure that the EBPs can be delivered for different systems, organizations, providers, and participants than was originally studied [72]), support people living with multiple chronic conditions because comorbidity is the rule rather than the exception [73], and ensure EBP effectiveness by evaluating the intervention in different settings with varying provider and participant attributes [74]. Our complementary process evaluation found that delivering in-person EBPs remotely did not require modifications to core program components [53], suggesting that fidelity to the active ingredients of the program models was maintained. That said, some of the observed lack of improvement in health outcomes by remote EBP participants may be due to failures in implementation (eg, inadequate intervention dose due to technology issues) that negatively impacted program effectiveness [42].

Our evaluation comes with several strengths. First, we conducted a pragmatic evaluation across the country in partnership with policy makers and delivery organizations so that the findings would have direct implications for improving older adults’ health equity. Specifically, we built upon trusting relationships and took time to foster new relationships to engage people most impacted by implementing remote EBPs; we also gathered qualitative data to put quantitative outcomes in context for future quality improvement [75]. Second, we included multiple brief, validated health outcomes across various EBPs and diverse participants and providers, centering outcomes that are important and meaningful to participants, providers, organizations, and policy makers that were not measured in previous studies [76]. These include social factors such as social isolation and loneliness and mental health factors such as depression and anxiety, which lead to premature mortality for older adults [77,78]. While measuring multiple outcomes across heterogeneous populations and programs made it harder to see large effect sizes, it better reflects how organizations work (delivering multiple programs) and how participants view health more holistically rather than as just 1 primary outcome. It may also be that small effect sizes suggest that these programs provided primary or secondary prevention to delay the onset of more impairing symptoms and conditions that was not picked up from our measures or relatively study time frame [79]. Third, collecting qualitative as well as quantitative data from the perspectives of both participants and providers provides consistent measures to compare across studies as well as stories and unanticipated outcomes to explore more broadly in future research.

Looking at limitations, first, using a single-group design with the lack of a comparison group or randomization means that we cannot attribute change in health outcomes over time to participating in the remote EBP. Second, we recognize that the lack of statistical significance when assessing for whether change was due to chance or program participation may be due to small sample sizes that lack power. This is partly why we collected qualitative data from both providers and participants: we wanted to understand the magnitude of change from their perspectives. Third, our response rate was lower than is typical in controlled research studies, with only half of the remote EBP participants (289/586, 49.3%) providing follow-up data. This was due to pandemic-related logistical and methodological challenges faced by our evaluation team, program providers, and older adult participants; conducting an evaluation during the COVID-19 pandemic has been deemed more challenging than conducting evaluations in conflict areas [80]. This suggests limitations to both internal validity through biases such as selection bias (eg, persons who liked the program or had access to a mobile phone or the internet to complete the evaluation were overrepresented) and external validity, with the COVID-19 pandemic being a unique context, because of which our findings may not be generalizable (eg, were improvements in social connection due to the high level of disconnectedness faced by everyone during the pandemic-related lockdown?). Finally, our convenience sample only included group-based programs and as such may not be applicable to one-on-one EBPs. We also did not have access to data on the entire remote EBP delivery population; therefore, we cannot comment on the representativeness of our outcome evaluation sample. Our process evaluation [24] suggests that persons who have been historically underserved by EBPs (eg, people of color, those with disabilities, and those living in rural areas) can be reached through remote delivery; however, care must be taken to ensure that remote delivery does not widen the health inequities caused by the digital divide.

In closing, the findings from this outcome evaluation suggest that participating in remote EBPs can improve health, social, and technology outcomes of interest for older adults, providers, and policy makers (Textbox 1). Future policy and practice can better support remote EBPs, improving access for all older adults (regardless of income, geography, and ability) and strengthening delivery organizations.