The persistent and progressive nature of dementia has deleterious effects not only for the care recipient but also for the caregiver, adversely affecting key aspects of psychosocial functioning [1,2]. The protracted burdens of caregiving may lead to elevated depression or anxiety [3], with increased duration and severity of symptoms heightening this risk among caregivers [4]. This has a tandem effect on outcomes for care recipients, where increased caregiving burden and stress results in diminished quality of care and quality of life for the patient [5].

To address the issue of caregiver burden, researchers are using new technologies to deliver resources to caregivers. One such intervention, Life Enhancing Activities for Family Caregivers, is a web-based positive emotion regulation intervention delivered through videoconferencing [6-8]. A randomized controlled trial of Life Enhancing Activities for Family Caregivers found that participation led to significant increases in positive emotion (Cohen d=0.58; P<.01) and positive aspects of caregiving (Cohen d=0.36; P<.01) and decreased symptoms of depression (Cohen d=−0.25; P=.02) and anxiety (Cohen d=−0.33; P<.01) compared with an emotion-reporting waitlist control group [6]. During this study, participants met one-on-one with a trained facilitator to learn the skills via videoconference. However, such delivery formats can be costly in terms of the time, effort, and logistics required to meet participants individually. To maximize the scalability of the intervention, there is a need for other delivery options that are more time efficient and cost efficient.

One such delivery format that has been widely adopted is the self-guided eHealth intervention, in which participants have access to the content on their own with minimal guidance from facilitators. However, although such interventions reduce barriers to participation, they are beset by high rates of attrition and poor adherence [9,10]. Parallel studies of other self-guided resources, namely Massive Open Online Courses, have found that the construct of social presence—or the perception of others being present in a web-based environment—can improve retention and engagement [11]. Features that enhance social presence include welcome messages, sharing participant profiles, and discussion boards [12]. Therefore, the application of such social features may improve engagement with eHealth and mobile health interventions.

Social features may be particularly beneficial for caregivers, who experience high levels of social isolation and loneliness compared with noncaregivers [13,14]. This study builds on data collected from focus groups and interviews conducted with providers and caregivers about the social features that might be most helpful for participants (Kwok, I, unpublished data, 2022). For example, focus group participants made suggestions to create podcast content, framing our reminder messages in an encouraging tone, and noted the potential challenges of creating a “buddy system” in which participants would be paired up. On the basis of these suggestions, we adapted the existing caregiver intervention and created Social Augmentation of Self-Guided Electronic Delivery of the Life Enhancing Activities for Family Caregivers (SAGE LEAF)—a socially enhanced, self-guided, web-based intervention for dementia caregivers that incorporated features that would enhance participants’ sense of social presence. In this study, our goal was to conduct a pilot study to determine the feasibility and acceptability of SAGE LEAF.

A total of 15 dementia caregivers were recruited from social media advertisements and caregiver support groups and organizations, which often serve as the first point of contact for caregivers who are seeking resources and support. Although the age of onset and disease progression varies across different forms of dementia, caregivers experience a high level of emotional stress and burden, with similar support needs that include respite care, psychotherapy, and support groups.

Interested individuals were sent a screener survey to determine their eligibility based on the following inclusion criteria: (1) identify as the primary family caregiver of a person with dementia, (2) speak and read English, and (3) have access to high-speed internet. In our screener survey, we explained that a primary family caregiver is the caregiver who spends most time with the care recipient. However, we did not define what constitutes a family member to ensure that we were being inclusive of diverse familial arrangements. Caregivers were ineligible if they had participated in a previous version of the intervention. Study staff contacted eligible participants to explain the requirements of the study and seek informed consent. Once consented, participants were sent a link for the SAGE LEAF platform with instructions for creating their password. They were then able to access the intervention content, which included 8 positive emotion skills unlocked over the course of 6 weeks (description of the skills is given in the following section). During this period, they had access to all the social features of the platform.

Previous studies have found that 5 users may be sufficient to identify 80% of usability issues, with diminishing returns from additional testing [15]. Hence, the first 33% (5/15) of the participants were invited for a feedback interview at week 3 to identify any critical usability issues at the halfway point of the intervention that might significantly affect participation. This allowed them to provide feedback about the various features while they were still accessing the website. The subsequent 67% (10/15) of the participants were invited for a feedback interview at the end of the entire course. This would allow us to collect feedback from participants who had completed the whole intervention. However, the structure of the interview remained the same for all the participants (15/15, 100%).

The SAGE LEAF intervention consists of 8 empirically validated positive emotion regulation skills that were delivered over 6 weeks. Each week, new content was unlocked for participants, with daily home practice exercises to reinforce the skills delivered that week. Similar to the previous versions of the intervention [6,16-19], the skills include noticing positive events, savoring, gratitude, mindful awareness, personal strengths, positive reappraisal, self-compassion, and attainable goals. SAGE LEAF is distinct from previous versions in that it has an emphasis on social features that are specifically designed based on the feedback we received from focus groups and interviews conducted with caregivers and clinicians who work with patients with dementia (Kwok, I, unpublished data, 2022). Such features include the design or functional elements of the platform that may enhance participants’ perceptions of social presence—the extent to which they sense the presence of others during their participation in the intervention. This includes sensing the presence of other caregiver participants and members of the study team. Textbox 1 shows a list of social features that were added to SAGE LEAF.

An example of how we incorporated the feedback from the focus groups and interviews (Kwok, I, unpublished data, 2022) was adopting a more encouraging tone in our automated support features. For instance, if a participant did not log in for a week, an encouraging message was sent to them through their preferred communication method (email or SMS text message). Other social features that we created based on feedback include the multimedia elements of the intervention. For example, in addition to an introduction video shown at the beginning of the study, the study team members recorded a skill-building video at the beginning of each lesson, briefly describing the skill and how it may be potentially helpful for caregivers. Furthermore, we recorded podcasts at the end of each lesson, which provides caregivers with optional content that they can review at their own schedule. The podcasts comprised interviews with study team members explaining how caregivers could overcome some of the challenges with applying the skills in everyday life and how to enhance their practice of the skills. On the basis of the feedback received, we also added more fields for the user profiles, so that participants could choose to share various aspects of their caregiving experience, for example, the relationship between the caregiver and recipient and some of the challenges and positive experiences that they may have had as a caregiver.

All participants completed a 45- to 60-minute phone interview to provide feedback about their experience of using the platform, with a focus on the social features being implemented. Participants dialed into an audio-only Zoom (Zoom Video Communications) session that was recorded. The interview followed a semistructured guide and was conducted with a trained facilitator who has experience with conducting focus groups and interviews (Kwok, I, unpublished data, 2022). The questions evaluated participants’ (1) use of the social features, (2) facilitators of or barriers to using the social features, and (3) recommendations for future improvements for each of the social features: for example, “One of the features of the website was the discussion board. Were you able to use the discussion boards? How often did you use them? If not, what kept you from using them? What are some aspects of the discussion boards that you found helpful? What are some aspects of the discussion boards that you didn’t find helpful? What do you think we could do to encourage people to participate in the discussion boards? How might the discussion boards be improved?” These questions were repeated for each social feature.

All interviews were recorded and automatically transcribed using Zoom’s cloud recording feature. Content analysis was conducted by categorizing user feedback as positive, negative, or implementation suggestions corresponding to each social feature. In addition, we noted any issues that participants might be having with the other aspects of using the SAGE LEAF website.

Participants completed the surveys at baseline and after the intervention at week 8. We added a 1-week buffer before and after the 6-week intervention, so that we could ensure that participants had fully completed the study before sending out the postintervention surveys. The postintervention survey included our primary measures that assessed the (1) usability, (2) usefulness, (3) feasibility, and (4) acceptability of the SAGE LEAF platform. Furthermore, we measured the participants’ perceptions of others on the SAGE LEAF platform by using an adapted version of the original Social Presence Scale [20]. Although the study was not designed to determine the efficacy of the intervention, we included measures to examine pre-post changes in measures of caregiving burden and psychological well-being. A list of measures is shown in Table 1.

We calculated the medians and IQR for primary measures that assessed the (1) usability, (2) usefulness, (3) feasibility, and (4) acceptability of the SAGE LEAF platform. In addition, we calculated supplementary user metrics based on data collected by the platform (eg, percentage of home practice activities completed and percentage of videos watched) that reflect various aspects of engagement. We calculated the means and SDs for the Social Presence Scale and applied them as a correlate with measures of usability, usefulness, feasibility, and acceptability. For measures related to preliminary outcomes, we performed paired, 1-tailed t tests on the data collected during the baseline and postintervention assessments to examine changes in means. Analyses were conducted using Excel (Microsoft Corporation) and R Studio (Posit).

The Northwestern University Institutional Review Board reviewed and approved the protocol for this study (Reference number: STU00215548).

Participants were recruited from a group of individuals who responded to social media advertisements for another caregiver study but were not eligible (3/15, 20%) and from dementia caregiver support groups (12/15, 80%). A total of 25 participants were screened, of which 10 (40%) were excluded owing to the following reasons: the care recipient did not have a diagnosis of dementia (4/25, 16%), they were no longer interested in participating in the study (2/25, 8%), and they dropped out before enrollment (4/25, 16%). Eligible participants provided web-based informed consent; completed a baseline assessment; and were subsequently sent an email with instructions about how to set up their account on the SAGE LEAF website, which would give them access to the 6-week program. Upon the completion of the SAGE LEAF course, they completed a follow-up assessment. In addition, all participants (15/15, 100%) completed a feedback interview, with (1) the first 33% (5/15) of the participants scheduled to complete the interview at week 3 to determine if there were significant usability issues early in the progression through the program, and (2) the next 67% (10/15) of the participants scheduled to complete the same feedback interview after they had completed the intervention. Initially, the qualitative feedback from the first 33% (5/15) of the participants was examined separately. However, given that there were no significant usability issues identified among them and that the subsequent 67% (10/15) of the participants completed exactly the same version of the intervention, the qualitative feedback and quantitative data were combined for analysis. The baseline characteristics of participants are shown in Table 2.

User metrics and measures for feasibility, acceptability, usability, and usefulness were collected upon completion of the intervention. Feasibility of the SAGE LEAF intervention was determined as the average percentage of number of web pages viewed out of the total 139 pages available in the intervention. These data were collected by the SAGE LEAF website. Participants demonstrated good adherence, completing an average of 72% (SD 42%; 100/139 pages) of the pages of the intervention.

Other measures for acceptability, usability, and usefulness were collected using the postintervention survey. As these data were not normally distributed owing to a relatively small number of participants, we calculated the medians and IQRs for these variables. Acceptability was evaluated to be good, with participants rating their willingness to recommend the SAGE LEAF program to a friend (median 8; IQR 6-8) or dementia caregiver (median 8; IQR 6.25-9.5), with 1 indicating definitely not and 10 indicating definitely yes. Usability was assessed using the System Usability Scale, with a median score of 73.75 (IQR 57.5-85.63) out of a total of 100, which can be interpreted as “good” from a usability perspective [29]. In terms of feature-specific usefulness, on a scale of 1 to 5, participants rated the discussion board with a median score of 3 (IQR 2-3), email reminders with a median score of 3 (IQR 2-4), user profiles with a median score of 2 (IQR 2-3), videos with a median score of 3 (IQR 3-3.75), podcasts and audio recordings with a median score of 3.50 (IQR 3-4), and social notifications (ie, alerts about likes and comments) with a median score of 3 (IQR 2-3). The ratings suggest that the podcast and audio recordings were determined to be the most useful feature. In terms of system-wide usefulness, participants rated SAGE LEAF with a median score of 3.70 (IQR 3-4.75) on a scale of 1 to 5.

Regarding social presence, participants rated the SAGE LEAF program with a median score of 3 (IQR 2.66-3.33) on a range of 1 to 5. Studies of the Social Presence Scale have not yet established benchmarks for what constitutes optimal levels for web-based platforms. However, past studies demonstrate strong associations between social presence scores and student’s satisfaction and learning, in the context of e-learning platforms [11]. Hence, we performed exploratory Pearson correlational analyses with social presence scores as a correlate with feasibility, acceptability, usability, and usefulness ratings. Our analyses found moderate, positive correlation between social presence scores and participants’ willingness to recommend the program to others (ie, friends and other caregivers; r₉=0.672; P=.03). We also found positive correlation between social presence scores and participants’ perceptions about the overall usefulness of the intervention (r₉=0.773; P=.009). We found no statistically significant associations with other feasibility, usability, and usefulness measures. Table 3 shows the results of these measures.

Of the 15 participants, 5 (33%) were lost to follow-up. The remaining 67% (10/15) of the participants completed both the baseline and postintervention surveys, which included measures relating to preliminary outcomes (Table 4). One-tailed paired t tests demonstrated a statistically significant (1) decrease in negative affect from baseline (M=1.71, SD 0.78) to after the intervention (M=1.34, SD 0.72; t₉=−2.49, P=.03); (2) decrease in perceived stress from baseline (M=9.00, SD 1.63) to after the intervention (M=7.50, SD 1.78; t₉=−2.29, P=.05); and (3) decrease in anxiety from baseline (M=61.42, SD 5.00) to after the intervention (M=57.45, SD 7.96; t₉=−2.52, P=.03); we also found a statistically significant (4) increase in meaning and purpose from baseline (M=43.56, SD 7.33) to after the intervention (M=47.19, SD 5.63; t₉=2.60, P=.03). No other statistically significant changes were found between the baseline and postintervention findings. However, no definitive conclusions about efficacy can be made, because our pilot study did not have a control group and was not sufficiently powered.

In this study, we adapted an existing intervention that has been previously tested with caregivers by incorporating social features into the design of the platform, which include podcasts and videos from the study team, multimedia library, discussion board, community tab, emotion feedback chart, social notifications, automated reminders, and user profiles. We then conducted a pilot test to determine its feasibility and acceptability. Participants’ ratings showed that the SAGE LEAF intervention demonstrated a high level of feasibility and acceptability.

Participants provided feedback about how to improve the social features in a way that would foster great social presence. Participants liked the format and content of the videos and podcasts, which enhanced their perception of the presence of study team members involved in the intervention. However, these could be improved by featuring caregivers through case studies or testimonials or discussing how the skills could be used in acutely stressful situations. The interviews revealed that most participants used the discussion board to post their home practice answers. However, there was less interaction among participants in response to each other’s posts. A suggestion was to highlight new or relevant posts that might entice users to interact with each other. The feedback indicates that the participants’ experience with using the discussion board may be enhanced if they had had more understanding about how other related social features work, including the user profiles, social notifications, automated reminders, and community tab—which may have synergies in enhancing the perception that there are other caregivers using the platform. Several participants also liked the emotions feedback chart because it provided feedback about their emotion data, but some participants were not aware about this feature.

In general, participants suggested that it would be important to explain why the features might be beneficial for caregivers early in the intervention. Hence, future versions of SAGE LEAF may incorporate this information in a tutorial video or slides during the registration process or encourage participants to select an avatar and complete their profile when they first log into the website, so that it can help them connect with other caregivers. Hence, we hope to enhance the onboarding process in future versions of SAGE LEAF, so that it sets up participants for successful social interactions during their time in the program. Given the web-based, self-guided format of the intervention, this initial setup process may be particularly crucial owing to the lack of direct contact between participants and study staff members. Other implementation suggestions from participants include enrolling small cohorts of participants and sharing relevant aspects of their profiles with the group. This may help establish a more personal context for interacting over the web and foster a strong sense of social connection.

Building on these early findings, future studies may include a randomized controlled trial of the SAGE LEAF intervention, in which a large pool of participants may be assigned to different combinations of the social features through a factorial design. A previous iteration of the intervention for individuals with depression showed that the combination of internet-based rewards and brief facilitator contact resulted in participants completing more of the intervention compared with those who only had access to only 1 feature [30]. A factorial design may be helpful in understanding which SAGE LEAF social features may have unique synergies for dementia caregivers. For example, for SAGE LEAF, the combination of user profiles with the discussion board may allow participants to learn more about the other participants who they are communicating with, thereby enhancing a sense of social presence. It is also plausible that other social feature combinations may work against each other such that a lot of social features may be burdensome for participants because our feedback interviews revealed that receiving daily reminders or completing the videos, skill lessons, and podcasts in one sitting is difficult for many caregivers. In addition, participants might feel pressured to share information or compete in internet-based challenges if all the activities have a social component. Hence, this future study will be helpful in teasing apart how such social feature combinations may influence outcomes.

Moving forward, once shown to be efficacious, the program may be broadly disseminated through dementia clinics across the country. For example, a local clinic may implement the SAGE LEAF intervention by enrolling a cohort of caregivers living in the same geographic area, which may complement their ongoing support group programming. Furthermore, dementia clinics may use the SAGE LEAF platform to connect participants who provide care for recipients with more uncommon forms of dementia. The web-based format of SAGE LEAF could facilitate participation across clinics in different geographic locations, allowing caregivers to connect with others who might share their unique circumstances.

One of the strengths of this study is that the social features that we incorporated were based on a previous study in which we collected qualitative feedback from clinical providers and caregivers about which features would be most helpful for caregivers (Kwok, I, unpublished data, 2022). We also performed a pilot test in which participants were given access to a fully functional version of the SAGE LEAF website, which would allow them to engage in these social features as if they were actual users with other participants enrolled at the same time. Typically, user experience evaluation studies are conducted through brief momentary evaluations or episode tests, with a trained facilitator having participants go through various tasks, collecting participants’ comments in real time, and recording the time that it takes to complete these tasks and their completion rates [31]. Instead, the pilot test allowed participants to evaluate the usefulness of our social features in a more naturalistic way and help us identify any significant usability issues.

A limitation of our study is our small sample size (N=15), which limits the generalizability of the findings. The small sample size allowed for a detailed and in-depth examination of usability and feasibility concerns through a mix of quantitative and qualitative methods. However, a large cohort of participants may have revealed more insights about the usability of the social features.

Another limitation of the study is that our sample lacked diversity in the type of diagnosis of dementia, with most participants being caregivers of a family member with frontotemporal dementia. There are >100 known forms of dementia, with Alzheimer disease being the most common diagnosis [32], and there is significant variation in the presentation of these diagnoses. In terms of differences in the presentation of dementia, Alzheimer disease affects individuals at an older age compared with frontotemporal dementia. In addition, Alzheimer disease results primarily in memory impairments as compared with the significant changes in language skills and personality exhibited by individuals with frontotemporal dementia. However, although care needs may differ, caregiving burden remains high across diagnoses, and the positive emotion skills may be helpful in managing the stress of caregiving. Therefore, the insights collected about feasibility and acceptability from this pilot study are likely to be generalized across diagnoses. Future studies may involve comparing cohorts of participants with care recipients with similar diagnoses and confirming that the intervention is effective across diagnoses. These cohorts may also be recruited based on the relationship between the caregiver and care recipient (eg, children caring for their parent or parent-in-law vs caregivers who are partners or spouses), which may enhance a sense of social connection among participants who share similar caregiving circumstances.

Furthermore, our sample lacked ethnic diversity, with non-Hispanic, White participants comprising 87% (13/15) of all participants. This warrants recognition because racial and ethnic minority communities are disproportionately affected by dementia. For example, dementia is approximately 50% more prevalent among Mexican American older adults [33] and Alzheimer disease is twice as prevalent among Black older adults than among their White counterparts [34]. In addition, as internet access varies across ethnic groups, our sample may have been more familiar with using the internet and email to participate in the study. Future collaboration with community groups relating to older adult services, family services, faith-based groups, and so on may help with outreach for underserved caregivers [35]. For large-scale evaluations of SAGE LEAF, relevant recruitment strategies may include tailoring our efforts according to geographic location, providing monetary incentives for engaging in more specific components of the study (eg, completion of each lesson and home practice activity), affording great flexibility in how we contact participants, and providing more extensive recruitment training [36-38].

Through our pilot test, we found the SAGE LEAF intervention to have a high level of feasibility and acceptability. In general, participants liked the social features that were implemented and expressed feedback about how they could be improved to enhance their usability and foster a sense of social connection. Future iterations of the SAGE LEAF intervention may include a more extensive onboarding process describing how the social features work and how they may benefit caregivers. We hope that future refinements to the intervention will enhance the perception that there are other caregiver participants and study staff involved, thereby fostering great engagement among participants.