This paper is in the following e-collection/theme issue:

Assisted Living for the Elderly and Nursing Home Care (162) Usability of Apps and User Perceptions of mHealth (974) Mobile Health (mhealth) (2939) Human Factors and Usability Case Studies (787) Emerging Technologies for Rehabilitation (261) Web-based and Mobile Health Interventions (3297) Mobile Devices and Apps for Seniors and Healthy Aging (167) Aging in Place (283) Dementia and Cognitive Decline (481) Cognitive Training for the Elderly (95) Home Care (170)

Published on in Vol 6 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/46480, first published .
A Tablet-Based App to Support Nursing Home Staff in Delivering an Individualized Cognitive and Physical Exercise Program for Individuals With Dementia: Mixed Methods Usability Study

A Tablet-Based App to Support Nursing Home Staff in Delivering an Individualized Cognitive and Physical Exercise Program for Individuals With Dementia: Mixed Methods Usability Study

A Tablet-Based App to Support Nursing Home Staff in Delivering an Individualized Cognitive and Physical Exercise Program for Individuals With Dementia: Mixed Methods Usability Study

Authors of this article:

Jelena Krafft1 Author Orcid Image ;   Bettina Barisch-Fritz1 Author Orcid Image ;   Janina Krell-Roesch1 Author Orcid Image ;   Sandra Trautwein1 Author Orcid Image ;   Andrea Scharpf1 Author Orcid Image ;   Alexander Woll1 Author Orcid Image
Article Authors Cited by (2) Tweetations (7) Metrics

    Original Paper

    Institute of Sports and Sports Science, Karlsruhe Institute of Technology, Karlsruhe, Germany

    Corresponding Author:

    Jelena Krafft, MSc

    Institute of Sports and Sports Science

    Karlsruhe Institute of Technology

    Engler-Bunte-Ring 15

    Karlsruhe, 76131

    Germany

    Phone: 49 721 608 ext 48513

    Email: jelena.krafft@kit.edu


    Background: The promotion of physical activity in individuals with dementia living in nursing homes is crucial for preserving physical and cognitive functions and the associated quality of life. Nevertheless, the implementation of physical activity programs in this setting is challenging, as the time and expertise of nursing home staff are limited. This situation was further exacerbated by the COVID-19 pandemic. Mobile health apps may be a sustainable approach to overcome these challenges in the long term. Therefore, the Individualized Cognitive and Physical Exercise-App (the InCoPE-App) was developed to support nursing home staff in delivering and implementing tailored cognitive and physical exercise training for individuals with dementia.

    Objective: This study aims to assess the usability of the InCoPE-App in terms of user performance and user perception in a laboratory setting using a mixed methods approach.

    Methods: Nursing home staff were encouraged to perform 5 basic tasks within the InCoPE-App. Their thoughts while using the app were captured by implementing a think aloud protocol. Then, participants completed the System Usability Scale questionnaire. The think aloud transcripts were qualitatively evaluated to unveil usability issues. All identified issues were rated in terms of their necessity to be fixed. Task completion (ie, success rate and time) and perceived usability were evaluated descriptively.

    Results: A total of 14 nursing home employees (mean age 53.7, SD 10.6 years; n=13, 93% women) participated in the study. The perceived usability of the InCoPE-App, as assessed by the System Usability Scale questionnaire, can be rated as “good.” The main usability issues concerned navigation logic and comprehensibility of app content.

    Conclusions: The InCoPE-App is a user-friendly app that enables nursing home staff to deliver and implement cognitive and physical exercise training for individuals with dementia in nursing homes. The InCoPE-App can be used with little training, even by people aged ≥50 years, who may have low digital literacy. To achieve sustainable use and high user satisfaction of the InCoPE-App in the long term, it should be implemented and evaluated in a field study.

    JMIR Aging 2023;6:e46480

    doi:10.2196/46480

    Keywords

    dementia; individualized physical exercise; tailored exercise; physical activity; older adults; app; mobile health; mHealth; usability; mobile phone 


    Background

    More than 55 million people worldwide have dementia, with approximately 10 million new cases every year [1]. By 2050, the number of individuals with dementia is expected to increase to up to 150 million individuals worldwide [2,3]. As dementia is a noncurable disease, treatment possibilities to stop or slow the progression of disease-specific symptoms (eg, declining cognitive function and physical performance) are critical. In addition to pharmacological therapies, nonpharmacological approaches such as physical activity (PA) have gained increasing attention. A growing body of research has shown that PA may have a beneficial impact on cognitive and physical performance in individuals with dementia [4]. However, only small and mainly nonsignificant effects of PA on quality of life (QoL) among individuals with dementia have been reported [5,6]. Overall, results from studies are conflicting, mainly owing to heterogeneous sample sizes and characteristics and differing intervention contents, periods, frequency, and duration of PA training [4]. Some studies also pointed out the heterogeneous prerequisites of individuals with dementia such as varying interindividual degrees of cognitive and motor impairments. Thus, a one-size-fits-all PA approach may fall short [7]. In addition, individual vulnerabilities and needs of individuals with dementia may need to be considered when designing, planning, and conducting PA interventions [8-10].

    According to several studies [11-14] and a systematic review [15], up to 80% of individuals living in nursing homes in European countries experience dementia. Individuals with dementia residing in nursing homes often have decreased life expectancy [16], more advanced dementia stages, and more impaired physical performance compared with community-dwelling individuals with dementia [17]. Moreover, living in a nursing home is associated with negative changes in QoL [18]. Overall, promoting PA in nursing home settings is therefore crucial. In many nursing homes, PA promotion is not regarded as a task or responsibility of nursing home staff and is usually delegated to external providers (eg, physiotherapists) [19]. During the COVID-19 pandemic, this practice was no longer feasible, as many nursing homes in Germany and other European countries were closed to visitors or external service providers, and PA programs had been discontinued in many nursing homes owing to increased safety measures [19]. The resulting social isolation and restricted movement possibilities led to worsening of cognitive function and physical performance among individuals with dementia, as perceived by nursing home staff [20]. Moreover, some studies reported significant impact on the mental well-being of nursing home residents (eg, QoL) [21]. A conclusion that can be drawn from the COVID-19 pandemic with its far-reaching health consequences is that PA promotion in nursing homes should be designed and implemented in a way that allows continuation even as new challenges arise (eg, changing circumstances owing to the pandemic or similar events) and without access to external PA instructors. Therefore, mobile health (mHealth) apps may be a viable solution in this context. Various definitions of the term mHealth exist and most include key aspects such as mobile computing, medical sensor, and communications technologies [22], health information and services [23], patient monitoring devices, and personal digital assistants [24] to improve health outcomes. mHealth can be considered as a subsection of eHealth [23]. mHealth solutions are considered to be feasible, can be implemented at little or no cost [25], and have wide reach among various patient groups or populations.

    So far, a large number of mHealth apps for use in care settings are available, with most of them providing support for medication management or health information, and they can be accessed free of charge from app stores [26]. However, to the best of our knowledge, no mHealth app for individualized PA promotion in nursing homes is available so far [27]. mHealth apps are promising tools in this setting and may help alleviate nursing home staff shortages; for example, a standardized, mHealth-based training manual may facilitate the instructions of PA sessions. Moreover, such an app may contain pictures and detailed exercise descriptions and information about the possible risk factors of certain exercises. These advantages may reduce the potential barriers for nursing home employees to deliver PA programs to individuals with dementia and enable the implementation of PA even in times of a pandemic. Nevertheless, a recent Cochrane review showed that health care workers with limited experience in using mobile apps and low digital literacy had concerns about making mistakes when using a mobile device [28], which might, in turn, affect the usability and acceptability of such apps.

    However, to guarantee the long-term use and acceptability of mHealth apps in nursing homes, the feasibility and usability of an app must be considered, ideally in the design and development phase of the app [29]. Usability indicates how a product is perceived by an intended user to achieve a specific goal in a specific context of use [30]. Nevertheless, most of the currently existing mHealth apps have not been scientifically designed and empirically evaluated [31,32], and publications addressing their feasibility and usability are lacking [33]. This is a main research gap, particularly because theory-based design and development of apps with subsequent scientific evaluation of usability and acceptability may be among the most important criteria to ensure the long-term implementation of mHealth apps, particularly in special settings such as nursing homes [34]. Moreover, studies have shown that involving nursing home staff in the development process of a mobile app makes them feel valuable and appreciated, which, in turn, could have a positive impact on acceptance [35]. Therefore, an iterative development process of an app including qualitative and quantitative methods to integrate possible end users in the development process is recommended [36], where designing, testing, and redesigning of a mobile app are embedded in a regular circle [29]. Examples for qualitatively collected data could be the identification of specific problems. In contrast, quantitative data may provide insight into use times or success rates [33]. A multistep development approach is intended to increase end users’ acceptability of an mHealth app and to ensure long-term use.

    Objective

    To address the current need for a scientifically derived mHealth-based PA promotion for individuals with dementia in nursing homes, we developed the Individualized Cognitive and Physical Exercise-App (the InCoPE-App). The InCoPE-App is a tablet-based app aimed at assisting nursing home staff in delivering tailored cognitive and physical exercise training for individuals with dementia in a nursing home setting. The content of the InCoPE-App is based on previous studies of our research group on PA for individuals with dementia [8-10]. The goal of this study was to evaluate the usability of the InCoPE-App with possible end users, that is, nursing home staff, using a mixed methods approach in a laboratory setting. Specifically, we examined user performance and perception, existing problems, and possible solutions regarding the InCoPE-App by integrating qualitative and quantitative methods. The results of this study will be used for further improvement and adaption of the InCoPE-App with the ultimate goal of implementation and long-term use of the app in nursing homes. Furthermore, this procedure can be used as an example for future studies of app development in nursing home settings.

    If and when the InCoPE-App has high usability, we anticipate that its use by nursing home staff will likely increase PA among individuals with dementia residing in nursing homes, as the app is designed such that it empowers nursing home staff to administer tailored physical exercise training to individuals with dementia in an easy and low-threshold way. Importantly, the InCoPE-App can be used by staff without previous PA-specific training or expertise.


    Study Design and Participants

    To evaluate the usability of the InCoPE-App, we used a mixed methods approach. We used a combination of qualitative and quantitative methods and considered a sample of 14 individuals, as previous studies have shown that 8 participants are sufficient to identify the main usability problems of a system [37]. Participants were recruited in April 2021 from 5 nursing homes in South-Western Germany. To be included in the study, participants (ie, nursing home staff) were required to have had previous experience with PA programs for individuals with dementia in the nursing home setting. Before the study, eligible participants received a project description regarding the objectives, participation, and benefits of the study and provided written consent for participation. The study was registered in the German National Register of Clinical Trials (DRKS00024069).

    Ethics Approval

    The study was approved by the Ethics Committee of the Karlsruhe Institute of Technology (Karlsruhe, Germany).

    The InCoPE-App: Content and Development

    The InCoPE-App was designed to be used by nursing home staff and not by individuals with dementia themselves, as individuals with dementia in nursing homes would not be able to perform structured physical exercise alone, and they need supervision for safety reasons. Specifically, the InCoPE-App supports nursing home staff in assessing current levels of cognitive and physical performance of individuals with dementia and, based on this assessment, guiding and delivering physical exercise sessions to individuals with dementia, without the need of having completed specific training or certification in sports or exercise science or kinesiology. A unique feature of the InCoPE-App is its integrated algorithm that uses data from 1 cognitive (ie, Mini Mental Status Examination [38]) and 3 physical performance tests (ie, Frailty and Injuries: Cooperative Studies of Intervention Techniques [39], 6-meter walk test [40], and modified 30-second chair stand test [41,42]) to tailor the recommended exercise program to the participant’s individual needs (Figure 1). The cognitive and physical tests integrated into the InCoPE-App are oriented to recommendations for individuals with dementia [43,44]. On the basis of the individual performance results, each individual with dementia is assigned to one of four exercise clusters, which are integrated in the app [45]: (1) individuals with below-average cognitive and physical performance, (2) individuals with average cognitive performance and above average physical performance, (3) individuals with above average cognitive performance and below average physical performance, and (4) individuals with above average cognitive and physical performance. The clustering into these 4 groups is based on previous studies by our group that have demonstrated the need for individualization of PA programs for individuals with dementia [8-10,46]. Depending on the cluster assignment, the InCoPE-App generates an exercise plan that fits the current performance level and needs of the individual with dementia. To adjust the exercise plan to individual changes in cognitive and physical performance, the InCoPE-App reminds the nursing home staff to repeat and record cognitive and physical performance tests every 3 weeks. In general, the exercise plan integrated into the InCoPE-App consists of ritualized warm-up and cooldown and 2 individualized workout phases that integrate exercises for balance, mobility, and upper and lower body strength [45].

    The generated exercise plan is presented in the app through brief descriptions along with pictures of the exercises to provide guidance about how to perform the exercises correctly and avoid common mistakes (Figure 2). Each training session lasts 60 minutes and is intended to be performed in one-on-one sessions or small groups of up to 2 individuals with dementia. For more information about the main functions of the InCoPE-App, refer to Multimedia Appendix 1.

    The iterative development process of the InCoPE-App included several steps (Figure 3), of which 3 are already completed. First, we defined a general product vision of the InCoPE-App. We then conducted a web-based survey to collect information about sex, age, profession, and daily tasks from nursing home staff. Furthermore, we gathered information about potential previous implementations of PA programs or interventions in participants’ nursing homes. On the basis of the results of this study, we were able to sketch personas as possible end users of the InCoPE-App [26]. In the second step, based on our product vision and the design of personas, we developed the first prototype of theInCoPE-App 1.0 in collaboration with a software expert team. The InCoPE-App was developed on Android 9.0. For study purposes, an offline-capable version of the InCoPE-App was locally installed on tablets (Lenovo Tab M10; 10 inch). Currently, the app is available only in German. The usability of the InCoPE-App 1.0 was tested by 7 experts in the areas of psychology, IT, sports science, and software development using a think aloud protocol and the System Usability Scale (SUS) [47]. The expert review unveiled relevant information about the usability of the InCoPE-App. The experts rated the InCoPE-App as acceptable but also noted some usability problems.

    ‎
    Figure 1. Chair stand test. (A) Written and illustrated description of the test procedure; (B) description of the measurement recording; (C) input field for the measured value; (D) integrated stop watch; and (E) required tools or equipment and possible risks.
    ‎
    Figure 2. Exercise for lower limb strength. (A) Overview of the training schedule; (B) exercise sequence in pictures; (C) description of aims and correct conduct of the exercise; (D) training parameters (eg, repetitions), possible risks (eg, pain), and cognitive input (eg, counting the repetitions); and (E) further information (eg, required equipment).
    ‎
    Figure 3. The iterative development process of the Individualized Cognitive and Physical Exercise-App (InCoPE-App) (step 1 results are published in a paper by Barisch-Fritz et al [26]; step 3 results are published in another paper by Barisch-Fritz et al [47]).

    Outcomes and Procedure

    After the participants signed the consent form, demographic information and data about general smartphone, tablet, and app use were collected from each participant using a short survey. Usability was assessed qualitatively and quantitatively in individual sessions during the first use of the InCoPE-App. To collect qualitative usability data, the think aloud technique was applied as it was found to be the most frequently used qualitative approach in usability testing of eHealth applications [31]. At the beginning, we explained to the participants that they would be required to speak their running thoughts aloud while interacting with the InCoPE-App. To become familiarized with this method, participants received a sample task within the InCoPE-App (ie, “Go to ‘exercise pool,’ choose exercise ‘Rope Pulling’ and tell me possible risks of this exercise”). Then, they were asked to perform 5 tasks (Table 1) with the InCoPE-App along a standardized protocol. These tasks were representative of a real-world situation when using the InCoPE-App in the nursing home setting [37]. During the think aloud session, a researcher was present and only interrupted participants if they stopped talking for >10 seconds while performing the tasks. Running thoughts of the participants were recorded via a voice recorder. Following the think aloud session, participants were asked three final questions: (1) “Which parts of the InCoPE-App are well designed?” (2) “Which parts of the InCoPE-App need to be revised?” and (3) “Do you have any other further comments on the InCoPE-App?”

    For quantitative usability assessment, the time spent on each individual task and all tasks overall was assessed by using the screen recorder function of the tablet. Furthermore, the success rate of each task was coded as “success,” “problem,” or “failure,” as described by Ehrler et al [48]. After the think aloud protocol, participants completed the German version of SUS [49,50], which is one of the most frequently used questionnaires in usability research [31]. The German version of SUS has reasonable reliability (0.84), concurrent validity (0.74), and sensitivity (0.83) [50]. SUS comprises 10 statements about the usability of a system (eg, “I think that I would like to use this system frequently”), each rated on a scale ranging from “I don’t agree” to “I totally agree.” Negatively worded statements (even numbers) are coded from 4 to 0, whereas positively worded (odd numbers) statements are coded from 0 to 4 [51]. The items are added to a sum score (minimum 0; maximum 40 points), which is multiplied by 2.5 (sum score—minimum 0; maximum 100 points). Published literature suggests a mean SUS score of 68 as a useful “benchmark” [52]. Furthermore, the total SUS score can be interpreted as follows: scores <60 indicate substantial usability problems, scores between 60 and 80 indicate marginal to good usability, and scores >80 indicate good to excellent usability of a system [49]. According to the Subjective Rating Scale of Bangor et al [53], a mean SUS score of 71.4 indicates good usability.

    Table 1. Standardized “think aloud” protocol.
    TaskDescription of the task
    1“Create a new test person.”
    2“Start and complete cognitive and physical testing with the test person.”
    3“Create an exercise plan and replace two exercises.”
    4“Start and finish a training session with the test person.”
    5“Start and finish a training session with two participants simultaneously.”

    Data Collection and Analysis

    Each think aloud session and the 3 interview questions were recorded with a voice recorder (Philips DVT2050) and transcribed verbatim using a transcription software (software F4transkript, from audiotranskription, dr.dresing&pehl GmbH). The transcribed protocols contained time stamps to estimate the time for task completion. To identify usability problems, bottom-down developed categories (ie, navigation, screen layout, graphics, comprehensibility, and overall usability) were used to analyze the protocols divided according to the think aloud tasks. This categorization was adjusted and based on a proposal by Kushniruk and Patel [54]. Two researchers (JK and ST) coded the transcripts independently. In case of ambiguities and discrepancies, a third researcher (BBF) was consulted. The identified usability problems were further rated by 1 researcher (JK) using the Nielsen severity scale (0=I do not agree that this is a usability problem at all, 1=cosmetic problem only, 2=minor usability problem, 3=major usability problem, and 4=usability catastrophe) [55]. This allows ranking of the usability problems and helps to prioritize them for a further revision cycle of the InCoPE-App. For presentation in this paper, the quotations from the final interviews were translated from German to English.

    The total SUS score, time spent on each task and in total (derived from the screen records), and frequencies of identified usability problems were evaluated descriptively (mean, SD, and range) using SPSS (version 27.0; IBM Statistics). The success rate for each task was evaluated in percentages.


    Participants

    We included 14 employees (n=13, 93% women and n=1, 7% men) from 5 nursing homes. The mean age was 53.7 (SD 10.6) years. Data about general smartphone and tablet use showed that all participants (14/14, 100%) owned a smartphone, with 93% (13/14) of the participants reporting daily use. Only 21% (3/14) of the participants reported using a tablet. Of the 14 participants, 12 (86%) had several apps installed on their personal smartphones or tablets and 7 (50%) reported daily app use. For study purposes, all participants (14/14, 100%) used the InCoPE-App installed on a tablet. Participants’ demographics and information about technical experience are presented in Table 2.

    Table 2. Sample characteristics (N=14).
    CharacteristicsValues
    Age (years), mean (SD)53.7 (10.6)
    Sex, n (%)

    		Female13 (93)

    		Male1 (7)
    Age group (years), n (%)

    		20-291 (7)

    		30-391 (7)

    		40-491 (7)

    		50-596 (43)

    		>605 (36)
    Certificate of secondary education, n (%)

    		Hauptschule (diploma after 5 y)5 (36)

    		Realschule (diploma after 6 y)2 (14)

    		High school diploma (diploma after 8-9 y; university entrance qualification)5 (36)

    		University degree2 (14)
    Use of mobile devices, n (%)

    		Smartphone14 (100)

    		Tablet3 (21)
    Frequency of smartphone use, n (%)

    		Daily13 (93)

    		Several times/wk1 (7)

    		Several times/mo—a

    		Rarely—

    		Never—
    Frequency of tablet use, n (%)

    		Daily1 (7)

    		Several times/wk2 (14)

    		Several times/mo1 (7)

    		Rarely—

    		Never10 (71)
    Use of apps, n (%)12 (86)
    Frequency of mobile app use, n (%)

    		Daily7 (50)

    		Several times/wk3 (21)

    		Several times/mo—

    		Rarely2 (14)

    		Never—

    aNot applicable.

    SUS Scores

    The mean SUS score was 72.3 (SD 18.9; range 45-95), indicating good to marginal usability. According to the Adjective Rating Scale by Bangor et al [53], usability can be rated as good. When dividing the sample into 3 age groups (ie, nursing home staff aged <50 years: 4/14, 29%; aged between 50 and 60 years: 5/14, 36%; and aged >60 years: 5/14, 36%), the mean SUS scores were 77.5 (SD 16.2), 78 (SD 17.1), and 60 (SD 22.1), respectively, indicating better usability in participants aged <60 years. The results for single items of the SUS are presented in Table 3.

    Table 3. Scores for the single items of the System Usability Scale.
    ItemStatementScore of the total group (N=14), mean (SD)aScore of participants aged ≤50 years (n=4), mean (SD)bScore of participants aged 51-59 years (n=5), mean (SD)cScore of participants aged ≥60 years (n=5), mean (SD)d
    1“I think that I would like to use this system frequently.”3.2 (1)3.5 (0.6)3.6 (0.9)2.6 (1.1)
    2“I found the system unnecessarily complex.”2.9 (1.2)3.3 (1)2.6 (1.7)2.8 (1.1)
    3“I thought the system was easy to use.”2.8 (0.7)3 (0.8)3 (0.7)2.4 (0.5)
    4“I think that I would need the support of a technical person to be able to use this system.”2.7 (1.4)3.8 (0.5)2.6 (1.3)2 (1.6)
    5“I found the various functions in this system were well integrated.”3.1 (0.8)3 (0.8)3.4 (0.5)2.8 (1.1)
    6“I thought there was too much inconsistency in this system.”3.2 (0.7)3.3 (0.5)3.4 (0.9)3 (0.8)
    7“I would imagine that most people would learn to use this system very quickly.”3.2 (0.8)3.3 (1)3.6 (0.5)2.8 (0.8)
    8“I found the system very cumbersome to use.”2.9 (1.1)2.5 (1.3)3.0 (1.2)3.0 (1)
    9“I felt very confident using the system.”2.4 (1)2.8 (0.5)2.8 (0.8)1.6 (1.1)
    10“I needed to learn a lot of things before I could get going with this system.”2.9 (1)2.8 (1.3)3.2 (0.8)2.6 (1.1)

    aTotal mean 72.3 (SD 18.9).

    bTotal mean 77.5 (SD 16.2).

    cTotal mean 78 (SD 17.1).

    dTotal mean 60 (SD 22.1).

    Think Aloud Session and Final Interviews

    The mean duration of the think aloud sessions in total was 45 minutes and 56 seconds (SD 5 min and 42 s; range 33 min and 34 s to 53 min and 7 s), including the instructions and the familiarization task at the beginning. The most time-consuming part was cognitive and physical testing (mean 16 min and 26 s, SD 3 min and 44 s; Table 4). Creating a test person profile was completed by all participants without any problems. Most usability problems (n=71) arose with cognitive and physical testing. The last task (“Start and finish a training with two participants simultaneously”) could not be performed by any participant (Table 4).

    On the basis of the think aloud protocols, 71 different usability problems could be identified that were mentioned 134 times in total. The categorization of the usability problems according to Kushniruk and Patel [54] revealed most problems in the category, “navigation” (64/134, 47.8%), within the InCoPE-App. In particular, problems with finding the button to start a training for 2 participants simultaneously were mentioned by 79% (11/14) of the participants. The frequency of the mentioned problems and the most common examples are displayed in Table 5.

    Table 4. Task duration and task completion.
    TaskDuration, mean (SD)Completion (N=14), n (%)


    		SuccessProblemFailure
    “Create a new test person.”3 min, 53 s (2 min, 4 s)14 (100)0 (0)0 (0)
    “Start and complete cognitive and physical testing with the test person.”16 min, 26 s (3 min, 44 s)3 (21)10 (71)1 (7)
    “Create an exercise plan and replace two exercises.”3 min, 54 s (1 min, 36 s)3 (21)4 (29)7 (50)
    “Start and finish a training session with the test person.”5 min, 14 s (2 min, 19 s)2 (14)8 (57)4 (29)
    “Start and finish a training session with two participants simultaneously.”—a0 (0)0 (0)14 (100)

    aNot applicable.

    Table 5. Frequency and rating of the mentioned usability problems identified via the think aloud protocol.
    CategoryMentioned frequency (N=134), n (%)Most common problems and rating
    Navigation64 (47.8)
    • Finding the start button to initiate a training for 2 people—“Usability catastrophe”
    • Changing or replacing exercises in an exercise plan—“Major usability problem”
    • Noticing the stopwatch during assessment—“Usability catastrophe”
    Screen layout20 (14.9)
    • Small font type—“Major Usability Problem”
    • Overloaded screens during exercising—“Major Usability Problem”
    Graphics6 (4.5)
    • No “zoom in” function—“Cosmetic problem only”
    Comprehensibility28 (20.9)
    • Uncertainty in cognitive test procedures—“Usability catastrophe”
    • Unclear scientific terminology—“Major Usability Problem”
    Overall usability16 (11.9)
    • Drag-and-drop function is not intuitive—“Minor usability problem”
    • Lot of information on most of the screens, owing to which app use was perceived as time consuming—“Minor usability problem”

    Of the 71 identified usability concerns, 4 (6%) were rated as usability catastrophe according to Nielsen 48 and must be corrected before the InCoPE-App can be used in the field. Of the 71 problems, 29 (41%) were rated as a major usability problem with high priority to fix; 23 (32%) as minor usability with low priority to fix; and 8 (11%) as cosmetic problems only, which should only be fixed if there will be extra time for app development. Of the 71 problems, 7 (10%) mentioned usability concerns were rated as not a usability problem at all. Examples are displayed in Table 5.

    During the final interviews, participants were able to explain which parts of the InCoPE-App were well designed. They explicitly mentioned that creating a test person within the InCoPE-App was very simple and easy to conduct:

    I think, the beginning, when creating a participant profile – this was very good and clear.

    Moreover, the participants highlighted the good interface and the clear user paths within the InCoPE-App:

    I liked that it [the app] is well pictured.
    What I totally like is that something is highlighted in orange, when I have to do [enter] something...and it is suggested to me.

    The participants also liked the instructions on the training screens within the InCoPE-App:

    So you’re just being carried through the exercise plan, exercise by exercise. That is well designed.
    [The exercise plan] is already divided into what counts as warm-up, the workout part itself, and the cool-down. I found that to be very clear.

    Overall, the participants appreciated that using the app only needs little practice and is beginner-friendly:

    I generally have very little idea about a tablet or a smartphone... For me it was plausible. It [the app] has also actually indicated to me what I have to do next.
    You also tried to keep it as simple as possible.

    In addition to the question about the parts of the InCoPE-App they liked the most, the participants were asked to name the parts that need to be revised in their opinion. Regarding this aspect, it was mentioned that exercise videos instead of pictures would be more user-friendly:

    It [the training] would take too long with the participant. I would be lost in details. Videos and especially a voice explaining it [the exercises] to me briefly, that would be very helpful for me.

    This statement was accompanied by comments about information overload on the screens within the InCoPE-App:

    That is a lot of text. You lose a lot of time. By the time I read this, the participants no longer have any desire
    [to exercise]
    I would have liked it better if the text had been shortened and presented in sections.

    In contrast to the comments about the beginner-friendliness of the InCoPE-App, a person also mentioned barriers to the first use:

    Well, if you don’t use a tablet every day, you don’t know where to push [a button]. For me as a person with limited media experience, it was hard.

    Finally, when participants were asked for further comments about the InCoPE-App, they underlined that even though they had some problems with the app at first or with technologies in general, they liked the app:

    At the beginning, I was really concerned. I thought that I have no idea about computer and tablets and so on. ...And I think, this is a great application, even I can handle that.

    Principal Findings

    Promoting physical and cognitive exercise for individuals with dementia in nursing homes is critically important, particularly in terms of the reduction of PA in this setting during the COVID-19 pandemic. Nevertheless, most interventions available today have limitations regarding long-term use and implementation. With the InCoPE-App, we aimed to develop an effective and easy-to-use app that requires a multistage development process considering feedback from future end users. In this study, we analyzed the usability of the InCoPE-App, which assists nursing home staff in delivering a tailored cognitive and physical exercise program for individuals with dementia in nursing homes.

    Here, we applied a mixed methods approach to get an in-depth impression of how the InCoPE-App is perceived by potential end users. Our results show that the usability of the InCoPE-App can be rated as “good” [53]. Considering the results of the single items of SUS, the least agreement was given to the statement, “I felt very confident using the system.” In contrast, the highest agreement was given to the statements, “I would imagine that most people would learn to use this system very quickly” and “I think I would like to use this system frequently.” These results indicate that on the one hand, participants felt that they needed additional information or training with the InCoPE-App. However, in contrast, they assumed that app use can be learned quickly. Overall, participants would like to use the InCoPE-App frequently and did not find the app to be unnecessarily complex.

    On the basis of think aloud task completion, cognitive and physical testing required the most time. It can be assumed that this corresponds well with real-life situations, as conducting tests among individuals with dementia requires a rather large amount of time and personnel resources. We observed that, particularly, reading test instructions was time consuming. However, it is likely that time to read instructions within the InCoPE-App may decrease with more regular app use. The most difficult task (100% failure) was to start a simultaneous training of 2 individuals. This app feature needs to be revised with high priority and has to be placed more prominently within the app menu. Overall, we can assume that the InCoPE-App is a user-friendly tool and that most of the problems mentioned by participants could be solved by frequent app use.

    Comparison With Previous Studies

    Although mobile devices have become increasingly popular over the past decade [28], so far, there is no scientifically evaluated mHealth app available in the context of PA promotion in nursing homes [27]. To the best of our knowledge, our study is the first to evaluate the usability of an mHealth-based app, developed to assist nursing home staff in implementing tailored cognitive and physical exercise for individuals with dementia in nursing homes. A unique feature of the InCoPE-App is that it is not used by the group considered vulnerable (ie, individuals with dementia) directly but by nursing home staff who serve as a mediator. To the best of our knowledge, there are no studies that have used this approach.

    The methods used in our study are consistent with the current literature and recommendations for usability testing [37]. Both applied methods exhibit important advantages in gathering a comprehensive impression of the usability of the InCoPE-App. So far, SUS is the most frequently applied questionnaire in the usability testing of digital health solutions [31]. Although there are usability scales specially tailored for mHealth solutions (eg, mHealth Usability Questionnaire [56]), these newly developed scales have not been widely used, and only a few comparative studies exist [57]. As the sole administration of SUS as a stand-alone usability method is not recommended [58], using a think aloud protocol is a complementary approach that provides direct insight into a person’s cognitive and problem-solving processes while using an app and is therefore essential and effective for uncovering usability issues in addition to a quantitative questionnaire [37,58]. A recent systematic review showed that, even for the evaluation of usability among older participants, questionnaires and qualitative assessments such as think aloud protocols are commonly used and feasible methods [59]. Furthermore, other studies in the context of health care rehabilitation also used a mixed methods approach to assess usability [48,60,61].

    In our study, we obtained a mean SUS score that is slightly above the benchmark of 68 points according to Sauro and Lewis [52] and the mean SUS for “good” usability according to Bangor et al [53]. A recent meta-analysis by Hyzy et al [62] explicitly focused on the SUS sum scores of 114 digital health apps and reported a mean score of 76.16 (SD 15.12) for all the included apps. By further categorizing the included apps, they observed a mean SUS score of 83.28 (SD 12.39) for “physical activity” apps (n=66) and a mean SUS score of 71.3 (SD 12.72) for “health care” apps [62]. Owing to the unique content of the InCoPE-App, the content-related results of the think aloud protocols and task completion are not comparable with other studies. Nevertheless, a study by Ehrler et al [48], which examined a mobile app for nurses in a hospital setting, identified “navigation within an app” to be one of the major problems. This is consistent with our results, as 47.8% (64/134) of the problems mentioned by study participants were related to the navigation structure within the InCoPE-App. These results imply that mobile apps to be used by staff in health care settings should be intuitive to navigate because complex navigation is perceived as time consuming and may thus be a barrier for long-term use by the end users [63]. Nevertheless, as the usability results of our study can be interpreted as “good,” we assume that the InCoPE-App is well designed and suitable for its primary target group, that is, nursing home staff.

    The perceived usability of the InCoPE-App could also be related to the mean age and the experience with mobile apps in our sample, that is, participants aged <60 years had fewer problems with using the InCoPE-App when compared with those aged >60 years. This was also observed in another study, where older participants reported more usability problems than younger ones, who were also more likely to have used apps before study participation [48]. Furthermore, existing literature has already demonstrated generational differences and a high likelihood of problems when implementing digital (health) solutions among older adults [33,64]. Thus, an age-based digital divide in mHealth adoption has been proposed in the literature [65]. Moreover, individuals often experience a loss in digital literacy if and when they do not use digital devices on a regular basis [28]. To overcome possible age-related and experience-related barriers to app use, current literature recommends education and familiarization training [48,66]. Moreover, as the fear of making mistakes could also be perceived as a barrier [28], “undo” functions should be included in an app [48].

    Strengths and Limitations

    The main strength of our study is the novelty of the presented the InCoPE-App and its user-centered development and testing process. This helps to gain new insights into a, thus far, little-explored research field. Although our participants were predominantly women and aged >50 years, they can be considered to be representative of the population of end users (ie, nursing home staff) who will use the system in the future. It is very crucial to include a representative target group to generate valid usability data and to avoid biases [37]. In addition, our sample was heterogeneous in terms of age, education, and technical experience. This allowed us to detect usability problems from different perspectives and gave us a nuanced impression of the potential end users. Moreover, engaging individuals with less access to or knowledge about technology is very important to ensure high usability of a system for individuals with low digital literacy [67]. Another strength of the study is the mixed methods approach. Particularly in usability research, 1 method alone is not suitable to cover all the important aspects of a system’s usability. Combining SUS with the think aloud task and the interview questions therefore allowed us to gain deep insight into the usability problems, as opposed to only evaluating usability on the basis of a sum score.

    A limitation of our study is the relatively late inclusion of the end users in the direct development process of the InCoPE-App. Although we created fictitious end users on the basis of a questionnaire in early development stages [26], the main content and the basic structure of the data model has been developed and finalized without the input of nursing home staff. In other studies, end users were included from the very beginning of the app development process [60]. It is likely that some of the frequently mentioned usability problems (eg, navigation within the app) could have been avoided by the early inclusion of end users in the development process. Another limitation is that members of our research team ranked the usability problems according to the method of Nielsen [55], and it is possible that the end users would have rated the severity of the problems differently. Thus, the revision of the app based on the prioritization done by the researchers may not fully correspond to the expectations and wishes of the end users as they may have chosen another prioritization. Therefore, in future studies, end users should also be included in this step. Furthermore, it should be differentiated which usability problems should be further addressed from different perspectives (eg, experts, developers, researchers, and end users).

    Conclusions

    The InCoPE-App is a novel and innovative app that assists nursing home staff in delivering tailored cognitive and physical exercise to individuals with dementia residing in nursing homes. We showed that the usability of the current version of the InCoPE-App can be rated as good according to 14 potential end users. Furthermore, even older participants found the InCoPE-App as easy to use after some familiarization. Nevertheless, certain aspects such as navigation features within the app must be further improved to increase the usability of the app in the future. To overcome potential barriers to using the app, further development should follow a “less is more” approach, for example, by minimizing navigation screens or reducing the complexity and length of text on the screens. Overall, the inclusion of end users in the app’s development process continues to be critically relevant and highly important. Therefore, the InCoPE-App was further tested in an 18-week intervention study [68].

    Acknowledgments

    This project was financially supported by Dietmar Hopp Stiftung (St Leon-Roth, Germany). The sponsor does not have any role in the design of the study, execution, data collection, or analysis and interpretation of data. The authors acknowledge the support provided by the KIT-Publication Fund of the Karlsruhe Institute of Technology. At the time of publication, AS is no longer affiliated with the Institute of Sports and Sports Science at the Karlsruhe Institute of Technology.

    Authors' Contributions

    All authors (JK, BB-F, JK-R, AS, ST, and AW) contributed to the conception, design, and implementation of the study. JK, BB-F, and AS collected the data. JK, ST, and BB-F analyzed and interpreted the data. JK wrote the manuscript, and all authors provided critical feedback. All authors approved the final version of the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Main functions of the Individualized Cognitive and Physical Exercise-App.

    DOCX File , 29 KB
    1. Dementia: key facts. World Health Organization. 2023. URL: https://www.who.int/news-room/fact-sheets/detail/dementia [accessed 2023-06-12]
    2. Patterson C. World Alzheimer report 2018 - the state of the art of dementia research: new frontiers. Alzheimer's Disease International. 2018. URL: https://www.alzint.org/resource/world-alzheimer-report-2018/ [accessed 2023-06-12]
    3. Pickett J, Bird C, Ballard C, Banerjee S, Brayne C, Cowan K, et al. A roadmap to advance dementia research in prevention, diagnosis, intervention, and care by 2025. Int J Geriatr Psychiatry. Jul 2018;33(7):900-906. [FREE Full text] [CrossRef] [Medline]
    4. Forbes D, Forbes SC, Blake CM, Thiessen EJ, Forbes S. Exercise programs for people with dementia. Cochrane Database Syst Rev. Apr 15, 2015;2015(4):CD006489. [FREE Full text] [CrossRef] [Medline]
    5. Lam FM, Huang MZ, Liao LR, Chung RC, Kwok TC, Pang MY. Physical exercise improves strength, balance, mobility, and endurance in people with cognitive impairment and dementia: a systematic review. J Physiother. Jan 2018;64(1):4-15. [FREE Full text] [CrossRef] [Medline]
    6. Ojagbemi A, Akin-Ojagbemi N. Exercise and quality of life in dementia: a systematic review and meta-analysis of randomized controlled trials. J Appl Gerontol. Jan 2019;38(1):27-48. [CrossRef] [Medline]
    7. Müllers P, Taubert M, Müller NG. Physical exercise as personalized medicine for dementia prevention? Front Physiol. May 29, 2019;10:672. [FREE Full text] [CrossRef] [Medline]
    8. Barisch-Fritz B, Trautwein S, Scharpf A, Krell-Roesch J, Woll A. Effects of a 16-week multimodal exercise program on physical performance in individuals with dementia: a multicenter randomized controlled trial. J Geriatr Phys Ther. Jan 2022;45(1):3-24. [CrossRef] [Medline]
    9. Bezold J, Trautwein S, Barisch-Fritz B, Scharpf A, Krell-Roesch J, Nigg CR, et al. Effects of a 16-week multimodal exercise program on activities of daily living in institutionalized individuals with dementia. Ger J Exerc Sport Res. Oct 14, 2021;51(4):506-517. [FREE Full text] [CrossRef]
    10. Trautwein S, Barisch-Fritz B, Scharpf A, Ringhof S, Stein T, Krell-Roesch J, et al. Effects of a 16-week multimodal exercise program on gait performance in individuals with dementia: a multicenter randomized controlled trial. BMC Geriatr. Jul 16, 2020;20(1):245. [FREE Full text] [CrossRef]
    11. Hoffmann F, Kaduszkiewicz H, Glaeske G, van den Bussche H, Koller D. Prevalence of dementia in nursing home and community-dwelling older adults in Germany. Aging Clin Exp Res. Oct 2014;26(5):555-559. [CrossRef] [Medline]
    12. Auer SR, Höfler M, Linsmayer E, Beránková A, Prieschl D, Ratajczak P, et al. Cross-sectional study of prevalence of dementia, behavioural symptoms, mobility, pain and other health parameters in nursing homes in Austria and the Czech Republic: results from the DEMDATA project. BMC Geriatr. Aug 13, 2018;18(1):178. [FREE Full text] [CrossRef] [Medline]
    13. Schäufele M, Köhler L, Hendlmeier I, Hoell A, Weyerer S. Prevalence of dementia and medical care in German nursing homes: a nationally representative survey. Psychiatr Prax. May 2013;40(4):200-206. [CrossRef] [Medline]
    14. Kowalska J, Rymaszewska J, Szczepańska-Gieracha J. Occurrence of cognitive impairment and depressive symptoms among the elderly in a nursing home facility. Adv Clin Exp Med. Jan 2013;22(1):111-117. [FREE Full text] [Medline]
    15. Seitz D, Purandare N, Conn D. Prevalence of psychiatric disorders among older adults in long-term care homes: a systematic review. Int Psychogeriatr. Nov 2010;22(7):1025-1039. [CrossRef] [Medline]
    16. Brent RJ. Life expectancy in nursing homes. Appl Econ. 2022;54(16):1877-1888. [FREE Full text] [CrossRef]
    17. Król-Zielińska M, Kusy K, Zieliński J, Osiński W. Physical activity and functional fitness in institutionalized vs. independently living elderly: a comparison of 70-80-year-old city-dwellers. Arch Gerontol Geriatr. Jul 2011;53(1):e10-e16. [CrossRef] [Medline]
    18. Olsen C, Pedersen I, Bergland A, Enders-Slegers MJ, Jøranson N, Calogiuri G, et al. Differences in quality of life in home-dwelling persons and nursing home residents with dementia - a cross-sectional study. BMC Geriatr. Jul 11, 2016;16:137. [FREE Full text] [CrossRef] [Medline]
    19. Frahsa A, Altmeier D, John JM, Gropper H, Granz H, Pomiersky R, et al. "I trust in staff's creativity"-the impact of COVID-19 lockdowns on physical activity promotion in nursing homes through the lenses of organizational sociology. Front Sports Act Living. Oct 07, 2020;2:589214. [FREE Full text] [CrossRef] [Medline]
    20. Geissler T, Bezold J, Scharpf A, Barisch-Fritz B, Trautwein S, Krell-Roesch J, et al. Impact of lockdown restrictions on physical activity programs in nursing homes during the COVID-19 pandemic. In: Proceedings of the 26th Annual Congress of the European College of Sport Science. 2021. Presented at: ECSS '21; September 8-10, 2021; Virtual Event. URL: https://publikationen.bibliothek.kit.edu/1000139432
    21. Kaelen S, van den Boogaard W, Pellecchia U, Spiers S, De Cramer C, Demaegd G, et al. How to bring residents' psychosocial well-being to the heart of the fight against COVID-19 in Belgian nursing homes-a qualitative study. PLoS One. Mar 26, 2021;16(3):e0249098. [FREE Full text] [CrossRef] [Medline]
    22. Istepanian R, Jovanov E, Zhang YT. Introduction to the special section on M-Health: beyond seamless mobility and global wireless health-care connectivity. IEEE Trans Inf Technol Biomed. Dec 2004;8(4):405-414. [CrossRef] [Medline]
    23. Nacinovich M. Defining mHealth. J Commun Healthc. 2011;4(1):1-3. [FREE Full text] [CrossRef]
    24. mHealth: new horizons for health through mobile technologies: second global survey on eHealth. World Health Organization. 2011. URL: https://apps.who.int/iris/bitstream/handle/10665/44607/9789241564250_eng.pdf?sequence=1&isAllowed=y [accessed 2023-06-12]
    25. Bhattacharya S, Kumar A, Kaushal V, Singh A. Applications of m-health and e-health in public health sector: the challenges and opportunities. Int J Med Public Health. 2018;8(2):56-57. [FREE Full text] [CrossRef]
    26. Barisch-Fritz B, Barisch M, Trautwein S, Scharpf A, Bezold J, Woll A. Designing a mobile app for treating individuals with dementia: combining UX research with sports science. In: Proceedings of the 12th International Symposium on Computer Science in Sport. 2019. Presented at: IACSS '19; July 8-10, 2019; Moscow, Russia. URL: https://link.springer.com/chapter/10.1007/978-3-030-35048-2_22 [CrossRef]
    27. Diener J, Rayling S, Bezold J, Krell-Roesch J, Woll A, Wunsch K. Effectiveness and acceptability of e- and m-health interventions to promote physical activity and prevent falls in nursing homes-a systematic review. Front Physiol. May 20, 2022;13:894397. [FREE Full text] [CrossRef] [Medline]
    28. Odendaal WA, Anstey Watkins J, Leon N, Goudge J, Griffiths F, Tomlinson M, et al. Health workers' perceptions and experiences of using mHealth technologies to deliver primary healthcare services: a qualitative evidence synthesis. Cochrane Database Syst Rev. Mar 26, 2020;3(3):CD011942. [FREE Full text] [CrossRef] [Medline]
    29. Shackel B. Usability – context, framework, definition, design and evaluation. Interact Comput. Dec 2009;21(5-6):339-346. [FREE Full text] [CrossRef]
    30. Thirumalai M, Rimmer JH, Johnson G, Wilroy J, Young HJ, Mehta T, et al. TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mHealth app: participant-centered development and usability study. JMIR Mhealth Uhealth. May 24, 2018;6(5):e10181. [FREE Full text] [CrossRef] [Medline]
    31. Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: a scoping review. Int J Med Inform. Jun 2019;126:95-104. [CrossRef] [Medline]
    32. Nouri R, R Niakan Kalhori S, Ghazisaeedi M, Marchand G, Yasini M. Criteria for assessing the quality of mHealth apps: a systematic review. J Am Med Inform Assoc. Aug 01, 2018;25(8):1089-1098. [FREE Full text] [CrossRef] [Medline]
    33. Guo C, Ashrafian H, Ghafur S, Fontana G, Gardner C, Prime M. Challenges for the evaluation of digital health solutions-a call for innovative evidence generation approaches. NPJ Digit Med. Aug 27, 2020;3:110. [FREE Full text] [CrossRef] [Medline]
    34. Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst. Mar 2015;39(2):1. [CrossRef] [Medline]
    35. Tsertsidis A. Identifying digital solutions for people with dementia (PwD): lessons learned from a Swedish dementia care residence. Gerontechnology. Jan 01, 2021;20(2):1-14. [FREE Full text] [CrossRef]
    36. Cho H, Yen PY, Dowding D, Merrill JA, Schnall R. A multi-level usability evaluation of mobile health applications: a case study. J Biomed Inform. Oct 2018;86:79-89. [FREE Full text] [CrossRef] [Medline]
    37. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. May 2009;78(5):340-353. [CrossRef] [Medline]
    38. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. Nov 1975;12(3):189-198. [CrossRef] [Medline]
    39. Rossiter-Fornoff JE, Wolf SL, Wolfson LI, Buchner DM. A cross-sectional validation study of the FICSIT common data base static balance measures. Frailty and injuries: cooperative studies of intervention techniques. J Gerontol A Biol Sci Med Sci. Nov 1995;50(6):M291-M297. [CrossRef] [Medline]
    40. Graham JE, Ostir GV, Kuo YF, Fisher SR, Ottenbacher KJ. Relationship between test methodology and mean velocity in timed walk tests: a review. Arch Phys Med Rehabil. May 2008;89(5):865-872. [FREE Full text] [CrossRef] [Medline]
    41. Blankevoort CG, van Heuvelen MJ, Scherder EJA. Reliability of six physical performance tests in older people with dementia. Phys Ther. Jan 2013;93(1):69-78. [CrossRef] [Medline]
    42. Jones CJ, Rikli RE, Beam WC. A 30-s chair-stand test as a measure of lower body strength in community-residing older adults. Res Q Exerc Sport. Jun 1999;70(2):113-119. [CrossRef] [Medline]
    43. Bossers WJ, van der Woude LH, Boersma F, Scherder EJ, van Heuvelen MJ. Recommended measures for the assessment of cognitive and physical performance in older patients with dementia: a systematic review. Dement Geriatr Cogn Dis Extra. Jan 2012;2(1):589-609. [FREE Full text] [CrossRef] [Medline]
    44. Trautwein S, Barisch-Fritz B, Scharpf A, Bossers W, Meinzer M, Steib S, et al. Recommendations for assessing motor performance in individuals with dementia: suggestions of an expert panel - a qualitative approach. Eur Rev Aging Phys Act. Apr 13, 2019;16:5. [FREE Full text] [CrossRef] [Medline]
    45. Barisch-Fritz B, Bezold J, Scharpf A, Trautwein S, Krell-Roesch J, Woll A. Usability and effectiveness of an individualized, tablet-based, multidomain exercise program for people with dementia delivered by nursing assistants: protocol for an evaluation of the InCoPE-app. JMIR Res Protoc. Sep 26, 2022;11(9):e36247. [FREE Full text] [CrossRef] [Medline]
    46. Barisch-Fritz B, Bezold J, Trautwein S, Scharpf A, Woll A. Cluster analysis of motor and cognitive skills of institutionalized individuals with dementia: 4 phenotypes for developing individualized physical activity programs. In: Proceedings of the 24th Annual Congress of the European College of Sports Science. 2019. Presented at: ECSS '19; July 3-6, 2019;265; Prague, Czech Republic. [CrossRef]
    47. Barisch-Fritz B, Bezold J, Barisch M, Scharpf A, Geissler T, Trautwein S, et al. InCoPE-App: a digital solution to train individuals with dementia. Usability review based on expert user experiences. In: Baca A, Exel J, Lames M, James N, Parmar N, editors. Proceedings of the 9th International Performance Analysis Workshop and Conference & 5th IACSS Conference. Cham, Switzerland. Springer; 2022;147-150.
    48. Ehrler F, Weinhold T, Joe J, Lovis C, Blondon K. A mobile app (BEDSide Mobility) to support nurses' tasks at the patient's bedside: usability study. JMIR Mhealth Uhealth. Mar 21, 2018;6(3):e57. [FREE Full text] [CrossRef] [Medline]
    49. Brooke J. SUS: a "quick and dirty" usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation in Industry. London, UK. Taylor & Francis; 1996;189-194.
    50. Gao M, Kortum P, Oswald FL. Multi-language toolkit for the system usability scale. Int J Hum Comput Interact. 2020;36(20):1883-1901. [FREE Full text] [CrossRef]
    51. Lewis JR. The system usability scale: past, present, and future. Int J Hum Comput. 2018;34(7):577-590. [FREE Full text] [CrossRef]
    52. Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research. 2nd edition. Cambridge, MA. Morgan Kaufmann; 2016.
    53. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud. May 01, 2009;4(3):114-123. [FREE Full text]
    54. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform. Feb 2004;37(1):56-76. [FREE Full text] [CrossRef] [Medline]
    55. Nielsen J. Usability Engineering. Amsterdam, the Netherlands. Kaufmann; 2010.
    56. Zhou L, Bao J, Setiawan IM, Saptono A, Parmanto B. The mHealth app usability questionnaire (MAUQ): development and validation study. JMIR Mhealth Uhealth. Apr 11, 2019;7(4):e11500. [FREE Full text] [CrossRef] [Medline]
    57. Hajesmaeel-Gohari S, Khordastan F, Fatehi F, Samzadeh H, Bahaadinbeigy K. The most used questionnaires for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health. BMC Med Inform Decis Mak. Jan 27, 2022;22(1):22. [FREE Full text] [CrossRef] [Medline]
    58. Broekhuis M, van Velsen L, Hermens H. Assessing usability of eHealth technology: a comparison of usability benchmarking instruments. Int J Med Inform. Aug 2019;128:24-31. [CrossRef] [Medline]
    59. Wang Q, Liu J, Zhou L, Tian J, Chen X, Zhang W, et al. Usability evaluation of mHealth apps for elderly individuals: a scoping review. BMC Med Inform Decis Mak. Dec 02, 2022;22(1):317. [FREE Full text] [CrossRef] [Medline]
    60. Rai HK, Schneider J, Orrell M. An individual cognitive stimulation therapy app for people with dementia: development and usability study of thinkability. JMIR Aging. Nov 16, 2020;3(2):e17105. [FREE Full text] [CrossRef] [Medline]
    61. Reeder B, Drake C, Ozkaynak M, Wald HL. Usability testing of a mobile clinical decision support app for urinary tract infection diagnosis in nursing homes. J Gerontol Nurs. Jul 01, 2019;45(7):11-17. [CrossRef] [Medline]
    62. Hyzy M, Bond R, Mulvenna M, Bai L, Dix A, Leigh S, et al. System usability scale benchmarking for digital health apps: meta-analysis. JMIR Mhealth Uhealth. Aug 18, 2022;10(8):e37290. [FREE Full text] [CrossRef] [Medline]
    63. Ahmad NA, Mat Ludin AF, Shahar S, Mohd Noah SA, Mohd Tohit N. Willingness, perceived barriers and motivators in adopting mobile applications for health-related interventions among older adults: a scoping review. BMJ Open. Mar 09, 2022;12(3):e054561. [FREE Full text] [CrossRef] [Medline]
    64. Calvo-Porral C, Pesqueira-Sanchez R. Generational differences in technology behaviour: comparing millennials and generation X. Kybernetik. 2020;49(11):2755-2772. [FREE Full text] [CrossRef]
    65. Fox G, Connolly R. Mobile health technology adoption across generations: narrowing the digital divide. Inf Syst J. Jan 29, 2018;28(6):995-1019. [FREE Full text] [CrossRef]
    66. Mayer MA, Rodríguez Blanco O, Torrejon A. Use of health apps by nurses for professional purposes: web-based survey study. JMIR Mhealth Uhealth. Nov 01, 2019;7(11):e15195. [FREE Full text] [CrossRef] [Medline]
    67. Richardson J, Letts L, Sinclair S, Chan D, Miller J, Donnelly C, et al. Using a web-based app to deliver rehabilitation strategies to persons with chronic conditions: development and usability study. JMIR Rehabil Assist Technol. Mar 18, 2021;8(1):e19519. [FREE Full text] [CrossRef] [Medline]
    68. Barisch-Fritz B, Bezold J, Scharpf A, Trautwein S, Krell-Roesch J, Woll A. ICT-based individualized training of institutionalized individuals with dementia. Evaluation of usability and trends toward the effectiveness of the InCoPE-app. Front Physiol. Jul 08, 2022;13:921105. [FREE Full text] [CrossRef] [Medline]

    ‎
    the InCoPE-App: the Individualized Cognitive and Physical Exercise-App
    mHealth: mobile health
    PA: physical activity
    QoL: quality of life
    SUS: System Usability Scale

    Edited by R Yang; submitted 07.03.23; peer-reviewed by F Khordastan, A Hashim, E Baker; comments to author 09.05.23; revised version received 26.06.23; accepted 05.07.23; published 22.08.23.

    Copyright

    ©Jelena Krafft, Bettina Barisch-Fritz, Janina Krell-Roesch, Sandra Trautwein, Andrea Scharpf, Alexander Woll. Originally published in JMIR Aging (https://aging.jmir.org), 22.08.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Aging, is properly cited. The complete bibliographic information, a link to the original publication on https://aging.jmir.org, as well as this copyright and license information must be included.