According to the methodological approach of scoping reviews [18], we aimed at mapping evidence of implementation success to provide a comprehensive overview. We followed the methodological guidance [18] of the Joanna Briggs Institute and structured our report according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) [19].

This scoping review is embedded in the project “Technology-based counselling in dementia (TeCoDem),” for which a protocol has been developed [7] and registered with the international prospective register of systematic reviews (PROSPERO CRD42021245473; see the section Deviations From the Protocol).

We included studies, irrespective of their design in English and German, that reported on technology-based counseling interventions for people with any type and severity of dementia or their informal caregivers. Interventions had to be tailored to individuals and provided remotely by professionals using various information technologies. Studies on people with mild cognitive impairment as well as studies on standardized interventions, such as cognitive behavioral therapy, on genetic counseling, and on counseling regarding diagnostics or screening for dementia were excluded. We also excluded studies describing interventions that focus mainly on care coordination or case management. Furthermore, publications reporting exclusively on the development of interventions were excluded [7].

We searched CINAHL, the Cochrane Library including the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE via PubMed, PsycINFO via Ovid, and the Web of Science Core Collection databases (last search: April 22, 2021) without any filters and limiters. We systematically developed a search strategy that contained 3 components: dementia, technology, and counseling. Corresponding search terms and synonyms (eg, dementia/Alzheimer, technology/electronic, counselling/counseling/consultation) were identified through an orienting search using MEDLINE via PubMed, and we checked entry terms given in the Medical Subject Headings browser. The strategy was peer-reviewed by applying the Peer Review of Electronic Search Strategies [20]. In addition, we performed forward and backward citation searches of included studies and pertinent reviews via Scopus (last search: October 7, 2021) and a web search via Google and Google Scholar (last search: November 26, 2021) [21,22]. Full database-specific search strategies are provided elsewhere [7].

Titles, abstracts, and full texts were independently screened by 2 reviewers (out of AB, JH, and DB) using the Rayyan web application (Rayyan) [23]. Any discrepancies in the decisions were resolved by discussions within the review team.

A targeted and uniform extraction sheet was developed and consented to by the research team. We extracted study and design characteristics (year of publication, country where the study was conducted, objectives, number of participants or contacts) and assessed the technology-based counseling interventions by applying criteria from the Template for Intervention Description and Replication checklist [24] and from the revised Criteria for Reporting the Development and Evaluation of Complex Interventions guideline [25] to obtain a comprehensive overview of the interventions and their components. Data extraction on implementation success was guided by the conceptual framework for implementation outcomes, which comprises the implementation outcomes acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability [17]. We adapted the definitions of the outcomes by specifying the intervention of interest and adjusting it to the research interest of our scoping review. Adaptations were consented to by the review team. The original and adapted definitions are listed in Table 1.

Quotations from the included studies were extracted and assigned to the outcomes by 1 reviewer (out of DB or JW) and cross-checked for accuracy by another reviewer (out of DB or JW). Any discrepancies were resolved by consensus between these 2 reviewers.

Data on the implementation success of technology-based counseling interventions were summarized by applying the framework mentioned in the preceding section [17]. The following 8 conceptually distinct implementation outcomes were used as conceptual categories to operationalize implementation success: acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability.

Our approach to data synthesis involved the following steps:

Each synthesis step was cross-checked (DB and JW) and consent was obtained from the review team.

Study characteristics and characteristics of the included interventions are presented in narrative and tabular forms.

The prespecified method of conducting a Qualitative Comparative Analysis on the conditions of successful implementation of technology-based counseling interventions [7] could not be realized because of the heterogeneity of the data found in the literature. In addition, reports that were not written in English or German were excluded because of a lack of professional translation resources.

The electronic database yielded 6387 records. After removing duplicates, we screened the titles and the abstracts of 3775 records, reviewed 277 full texts for eligibility, and included 35 records. We identified 3614 records from additional sources and assessed 151 full texts, of which 17 were included. Finally, 52 publications [29-80] reporting on 27 technology-based counseling interventions were included (Figure 1).

Interventions ranged from single counseling interventions, such as helpline services, to multicomponent programs combining nontechnology-based components, such as day care for people with dementia, with technology-based counseling for informal caregivers. All 27 interventions [29-80] focused on informal caregivers, mostly in combination with people with dementia, and 4 also addressed professional caregivers [29-32,34,35,37,40]. Counseling was provided by professionals from different disciplines, such as psychologists, geriatricians, or nurses. If volunteers were involved, they were professionally supervised or had received training. The technologies used for delivery included telephone, email, videoconferencing, social media, and chats, as well as web-based platforms. Additional (personalized) information material was frequently offered and delivered via email or postal mail. We differentiated the following types of interventions (refer to Table S1 in Multimedia Appendix 1): counseling via telephone or email and counseling via videoconferencing; web-based psychosocial intervention: information, communication, and counseling; videoconference- or telephone-based counseling combined with tele-monitoring or psychoeducation; and technology-based counseling as part of a comprehensive program with nontechnology-based components.

The design of the included studies varied (refer to Table S2 in Multimedia Appendix 1). Studies applying a quantitative descriptive design mostly focused on users’ demographics, topics discussed and advice provided in counseling sessions, and satisfaction with services. Few of the included studies exclusively focused on implementation, and we found process evaluation reports [79,80] related to 2 interventions. Furthermore, the publication type of the included reports varied greatly, as we aimed to depict the broad spectrum of interventions. In addition to research reports, we identified abstracts, letters to the editor, and practical project reports.

The interventions are displayed in Table S2 in Multimedia Appendix 1 and characteristics of the included studies are described in Table S3 in Multimedia Appendix 1.

Table 2 provides an overview of the data extracted from the included studies.

The label (“✓”) indicates the presence of data without any information on content or scope. None of the interventions included reported data in all categories. The information available ranged from 1 to 7 conceptual categories for each intervention (also refer to the analysis matrices in Multimedia Appendix 1).

Figure 2 shows the graphical synthesis of the data. Data on the conceptual categories of implementation success were sought for 27 interventions (blue line). The number of interventions for which data were extracted is indicated by the red line. While the categories appropriateness and acceptability are largely covered, substantial parts of the other areas remain unconsidered (refer to Figure 2).

We aimed to review the knowledge about the implementation success of technology-based counseling interventions for people with dementia or their caregivers. In our scoping review, we included 52 publications that reported 27 interventions. Interventions were heterogeneous and ranged from single counseling interventions, such as helplines, to counseling as part of multicomponent programs. To operationalize implementation success, we used the 8 outcomes of the conceptual framework for implementation outcomes [17] as conceptual categories. Only a few studies evaluated the implementation. Reporting on implementation outcomes was found to be fragmentary, and the comprehensiveness of the information varied widely. Overall, the focus of reporting was on the outcomes of appropriateness and acceptability, which may be because great efforts were made to adapt the interventions to a vulnerable target population.

As our data show, reporting on acceptability is inconsistent in terms of the perspective taken: data reports on measures to promote acceptability, but only from the perspective of the provider. At the same time, reporting on impact from this perspective was underrepresented. As already discussed in the literature, there are difficulties in unifying the wealth of perspectives in the context of implementation research [16].

The information reported on the dimension of adoption illustrates the importance of the fit between organizational motives and the type of intervention chosen for successful implementation. Little data are available on the mode of decision but differences in organizational culture can be identified that may influence the success of implementation efforts. We found data indicating that decisions from administrative stakeholders had an impact on sustainability by limiting the duration of the implementation of an intervention. Increasing the administrators’ enthusiasm for implementing the intervention by promoting familiarity with the specific intervention and using the effect of name-brand recognition may facilitate long-term commitment [82]. In some cases, the uptake of a specific intervention was based on evidence of its effectiveness, and there is a need for further effectiveness trials to expand the evidence base for decision-making regarding the implementation of technology-based counseling interventions.

The data provided on appropriateness comprised the largest amount of information extracted for categories and document the efforts undertaken to fit the target population. In addition to general measures to enhance the perceived fit of individualized support services, providers’ skills and instruments, as well as concepts applied by individual providers, were described. Tailoring and personalizing counseling services to individuals’ needs has been associated with the usefulness of information and support [55-65]. The benefits and limitations of using technology for delivering counseling are discussed against the background of statements by consumers who would have preferred a different mode of communication with counselors [36]. On the basis of participants’ attrition, González-Fraile et al [9] reported that remote support or training interventions appear to be less acceptable to informal caregivers of people with dementia than control interventions, which may limit their applicability in community settings. Further research is needed to determine whether services that are accessible both face to face and technology-based can provide appropriate accessibility and improve the perceived fit of the target populations.

Information on feasibility comprised the successful implementation of interventions. Although factors impeding feasibility, such as legal issues and technical challenges, were reported, we found no information on failed or unsuccessful implementation. Barriers to the implementation of eHealth interventions described in the literature are, among others, the lack of digital literacy in the target population and staff’s uncertainties and insecurities about their coaching competences [83]. According to Proctor et al [17], the concept of feasibility is typically “invoked retrospectively as a potential explanation of an initiative’s success or failure.” Thus, a more comprehensive reporting of factors promoting or impeding feasibility may inform the implementation of interventions in future projects and may contribute to increasing the implementation success of technology-based counseling interventions.

Ways of formalizing the interventions to ensure fidelity in the delivery of interventions were mentioned, but manuals or guidelines were not made accessible along with publications. In addition, measurements to ensure fidelity were described for some interventions, but the results of assessments, as well as details on aspects where deviations occurred, were not reported. After critically reviewing the literature on the use of fidelity implementation frameworks in early intervention, Lemire et al [84] also stated gaps in defining and assessing implementation fidelity. Drawing on preexisting conceptualizations, the authors proposed a definition of fidelity that comprises the 4 components: adherence, exposure, quality, and participant responsiveness [84].

The cost impact of implementation efforts was rarely reported for the included interventions. Factors that influence the costs are the complexity of the specific intervention, the strategy used for implementation, and the delivery setting [17]. Despite the costs incurred in setting up the technical infrastructure, the costs for remote delivery were lower than when counseling was provided face to face [36,68,69]. Owing to the lower costs, eHealth interventions are considered suitable for widespread implementation [16]. The provision of information on implementation costs is essential to compare the cost impact of different interventions and to inform decisions regarding the uptake of a specific intervention [17].

The data reported on penetration often indicates access to the service in multiple ways, which seems to match the preferences and capabilities of consumers. As reported by Jelly et al [85], caregivers use dementia support services primarily when services are able to meet consumers’ individual needs. However, it is important to keep in mind that, from an organizational point of view, these extended access options need to be served simultaneously. In particular, cooperation with other stakeholders seems to be central to integrating counseling services into a service setting, but this was only highlighted by some authors in the included publications.

The sustainable anchoring of diffusion processes is described as a difficult phase in the implementation process of support services for caregivers of people with dementia. The reasons for this include a lack of understanding of the barriers to sustainable implementation in practice and a lack of long-term funding [86]. The problem is substantiated in that, as long as researchers focus on measuring the effectiveness of the interventions, the potential goal is not fully realized. However, there are models that can support this sustainable implementation [87].

There are several theoretical approaches, such as generalized theories, models, or frameworks, that address different aspects of implementation [88]. Frameworks “describe more loosely structured constellations of theoretical constructs... or prescriptive approaches for accomplishing implementation goals” [88]. By providing clarity in terms and definitions [88], frameworks contribute to shared language in implementation research. There are different types of frameworks focusing on processes or determinants or evaluations [88]. Evaluation frameworks, such as the conceptual framework for implementation outcomes introduced by Proctor et al [17] and the Reach, Efficacy, Adoption, Implementation, Maintenance (RE-AIM) planning and evaluation framework [89,90], offer guidance on identifying results that can be used to evaluate implementation efforts [88]. While the RE-AIM framework describes outcomes across 5 domains (reach, effectiveness, adoption, implementation, and maintenance) [90], Proctor et al [17] present the concept of 8 implementation outcomes, which are differentiated from service system and treatment outcomes. Implementation outcomes are defined as “the effects of deliberate and purposive actions to implement new treatments, practices, and services” and are reported from different levels of analysis (eg, the provider or the consumer perspective) [17]. Serving as conceptual categories of the implementation success [17], these outcomes provided the appropriate approach to operationalize the object of interest—the implementation success of technology-based counseling interventions in dementia—in our review.

To increase the clarity of terminology used in implementation research, Proctor et al [17] proposed the definitions of 8 conceptually distinct implementation outcomes as a “working taxonomy,” including different aspects of implementation success and thus creating a comprehensible framework.

The use of outcomes as conceptual categories was sometimes challenging in our case. The mapping of the extracted data, in particular, was sometimes difficult because of the conceptual similarity of some outcomes, for example, penetration and sustainability, and the inconsistent use of terminology found in the literature. When determining the levels of analysis, we sometimes included additional perspectives, as described by Proctor et al [17].

Altogether, the lack of process evaluation studies, the fragmented reporting, and the unclear use of terms and concepts made it impossible to determine the extent of implementation success of technology-based counseling interventions in dementia care. Because of the impaired comparability of data, we were not able to assess how the different types of interventions affect the conceptual categories of acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability. For instance, we found data on appropriateness for all interventions, but the consumer-perceived usefulness of services was referred to for only 5 interventions. These 5 interventions comprised helplines that provided counseling via telephone [29-32,55] and web-based psychosocial interventions that provided information, communication, and counseling [56-65]. The reported information does not allow any conclusions to be drawn on how the types or components of interventions have an impact on implementation success. Gaining further insight into this issue is important for developing future interventions that can be implemented successfully.

Adherence to the framework for developing and evaluating complex interventions [6] may help overcome these problems, as proper process evaluation and exploration of conditions for implementation are recommended. The update to the Medical Research Council guidance states that “[e]arly consideration of implementation increased the potential of developing an intervention that can be widely adopted and maintained in real-world settings” [6] and thus can increase the success of implementation efforts. In addition, the conceptual clarity of the terms and concepts used in implementation research is needed to enhance transparency. This can be achieved by applying theoretical approaches that are “encapsulated as generalized theories, models, or frameworks” [88]. The consistent use of terms not only creates clarity but also forms the basis for better reporting on the success of implementation efforts, as Lengnick-Hall et al [91] proposed as the first of 6 practical recommendations for improved implementation outcomes reporting.

We followed a theory-driven approach to review the available evidence on implementation success. As we examined a broad topic with evidence emerging from studies in various designs, a scoping review proved appropriate. We performed a comprehensive and methodologically rigorous systematic literature search and included a variety of technology-based counseling interventions for people with dementia and their informal caregivers. Differentiating counseling from interventions focusing on education and information or from psychotherapeutic approaches brought challenges that we overcame through intensive discussions in the review team. Although we were able to include a considerable number of publications, it was not possible to make reliable statements about the implementation success of technology-based counseling interventions in dementia because of the inconsistent database as well as the heterogeneity in terminology and concepts.

We applied 8 conceptually distinct categories to operationalize the implementation success of technology-based counseling interventions for people with dementia and their informal caregivers. We found considerable data for the categories appropriateness and acceptability, and limited data on sustainability, feasibility, adoption, penetration, fidelity, and implementation cost. There is an imbalance in the scope and depth of the reported data on the conceptual categories, and the data extracted from the included publications only partially covered the concept of implementation success.

This highlights the need for a systematic evaluation accompanying the implementation of technology-based counseling interventions in the context of dementia. Adherence to guidelines for the development and evaluation of interventions and to guidelines or recommendations for reporting conceptualizations, measurements, and results on implementation outcomes is needed to expand knowledge on the effectiveness of implementation efforts and may foster the implementation of complex interventions in diverse contexts.