Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 16.10.20 in Vol 3, No 2 (2020): Jul-Dec

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/19852, first published May 04, 2020.

This paper is in the following e-collection/theme issue:

    Original Paper

    Smartphone-Based Experience Sampling in People With Mild Cognitive Impairment: Feasibility and Usability Study

    1Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Alzheimer Centre Limburg, Maastricht University, Maastricht, Netherlands

    2Division of Occupational Therapy, Department of Neurobiology, Care Science and Society, Karolinska Institutet, Stockholm, Sweden

    Corresponding Author:

    Sara Laureen Bartels, PhD

    Department of Psychiatry and Neuropsychology

    School for Mental Health and Neuroscience, Alzheimer Centre Limburg

    Maastricht University

    Dr. Tanslaan 12

    Nivo 4

    Maastricht, 6229 ET

    Netherlands

    Phone: 31 387 41 75

    Email: sara.bartels@maastrichtuniversity.nl


    ABSTRACT

    Background: Daily functioning of people with cognitive disorders such as mild cognitive impairment (MCI) is usually depicted by retrospective questionnaires, which can be memory-biased and neglect fluctuations over time or contexts.

    Objective: This study examines the feasibility and usability of applying the experience sampling method (ESM) in people with MCI to provide a detailed and dynamic picture of behavioral, emotional, and cognitive patterns in everyday life.

    Methods: For 6 consecutive days, 21 people with MCI used an ESM app on their smartphones. At 8 semi-random timepoints per day, participants filled in momentary questionnaires on mood, activities, social context, and subjective cognitive complaints. Feasibility was determined through self-reports and observable human-technology interactions. Usability was demonstrated on an individual and group level.

    Results: Of the 21 participants, 3 dropped out due to forgetting to carry their smartphones or forgetting the study instructions. In the remaining 18 individuals, the compliance rate was high, at 78.7%. Participants reported that momentary questions reflected their daily experiences well. Of the 18 participants, 13 (72%) experienced the increase in awareness of their own memory functions as pleasant or neutral.

    Conclusions: Support was found for the general feasibility of smartphone-based experience sampling in people with MCI. However, many older adults with MCI are currently not in possession of smartphones, and study adherence seems challenging for a minority of individuals. Momentary data can increase the insights into daily patterns and may guide the person-tailored development of self-management strategies in clinical settings.

    JMIR Aging 2020;3(2):e19852

    doi:10.2196/19852

    KEYWORDS



    Introduction

    Clinical questionnaires are commonly retrospective in nature and are thus potentially affected by a memory bias and thought to have low ecological validity [1]. As already cognitively healthy individuals over or underestimate past emotions and situations [2], this method might distort reality even more when people experience cognitive deficits. Moreover, within- or between-day fluctuations of health aspects are rarely taken into account, even though emotions and well-being vary depending on daily circumstances [3,4].

    Momentary data collection, known as the experience sampling method (ESM) [5] or ecological momentary assessment [6], may offer a solution to this problem. The ESM uses diaries to gather information on symptoms, mood, activities, or social interactions in the moment they occur. Individuals fill in short questionnaires about current emotions and behaviors repeatedly over several days, which results in a high ecological validity and offers detailed insight into dynamic patterns [7]. Originally, ESM questionnaires had a paper-pencil format, but more recently, mobile devices such as smartphone apps have prevailed. Compared to paper-pencil diaries, technology-based ESM questionnaires can be filled in faster, reducing time burden and providing more details about the exact assessment time. Using the ESM, especially in combination with personalized feedback from a health care professional, increases awareness of and engagement in a healthy lifestyle and thus supports self-management [8,9]. The term self-management can be defined as “the individual's ability to manage their symptoms, treatment, physical and psychosocial consequences, and lifestyle changes inherent in living with a chronic condition” [10] and is a necessary skill to improve or maintain daily functioning.

    A recent review reported that technology-based self-monitoring such as the ESM is already applied in various populations, including people with depression, chronic pain, or other health issues, to study behaviors and promote health [9]. In cognitively healthy older adults, momentary data collection seems feasible and acceptable [11] and is promising in individuals with brain injury [12] and after stroke [13].

    One group of individuals that might also benefit from this diary approach are people with mild cognitive impairment (MCI). By definition, MCI is not thought to impact daily functioning greatly [14]; however, even small cognitive alterations can lead to changes in feelings, behaviors, self-perception, and social interactions [15]. Thus, self-management can be impaired when living with MCI.

    To our knowledge, using the ESM in people with mild cognitive impairment (MCI) is rare. Daily or weekly paper-pencil diaries have been used to study momentary stressors and affect in MCI samples [16,17], but we are not aware of technology-based ESM studies in this population. Assessing the general feasibility of an unfamiliar and technology-based method is necessary, as people with MCI are commonly older and have amnestic deficits. Thus, individuals with MCI may find it challenging to process new information or handle unfamiliar technologies. Research shows, for example, that people with MCI find it more challenging to use everyday technology than older adults without cognitive impairments [18,19], which might also impact the feasibility of smartphone-based ESM. If feasible, applying ESM in people with MCI may reveal valuable insight into daily patterns of their lives that traditional assessments have been unable to depict. Furthermore, the ESM may promote awareness and self-management in this population, thus ultimately contributing to maintained or improved well-being.

    This study aims to determine the feasibility and usability of smartphone-based experience sampling in people with MCI. An ESM app was installed on participants' smartphones and programmed with a high sampling frequency to capture various intra-individual states (ie, mood, subjective cognitive problems) and situations (ie, activities, social context). Self-reports of using the ESM and observations of the direct human-technology interaction were conducted as part of the feasibility assessment. Human-technology interaction refers here to the person's ability to manage the ESM smartphone app, including specific performance skills, environmental characteristics, and individual capacities.

    The usability of momentary data was studied on an individual and group level, focusing on subjective cognition, daily activities, and stress experienced in relation to those activities. Studying the data on a group level can provide valuable information on the daily functioning of the MCI population in general, while individual data can illustrate within-person fluctuations. This may result in helpful person-tailored insights that not only foster individualized therapy but also the diagnostic process [20] and the monitoring of early changes in cognitive or behavioral alterations in MCI.


    Methods

    Sample

    Participants were recruited from the memory clinic at the Maastricht University Medical Center (UMC) from June 2018 to January 2020. Inclusion criteria were (1) having a clinical diagnosis of MCI, according to Albert et al [14], (2) owning a smartphone with an Android or iOS operational system, (3) providing written informed consent, and (4) receiving written informed consent from a relevant other (ie, partner, family member, close friend) that was selected by the person with MCI and was recruited. Exclusion criteria were (1) insufficient abilities to participate in research (eg, a self-reported or relevant other–reported inability or lack of confidence to use a smartphone or to learn and remember the purpose of the study) and (2) severe health problems, such as a diagnosis of a somatic, psychiatric, or neurological disorder causing additional cognitive dysfunction. Both exclusion criteria were based on the clinical judgment of a psychologist or psychiatrist during the recruitment phase (eg, telephone conversations with a potential participant or relevant other).

    The Medical Ethical Committee from the Maastricht academic hospital (azM) and Maastricht University approved the study (NL64310.068.17 / METC173055), and the protocol is registered on ToetsingOnline (64310). The authors comply with the Helsinki Declaration of 1975, as revised in 2008. All participants, including people with MCI and their relevant others, provided written informed consent before study participation.

    Experience Sampling Smartphone App

    The PsyMate smartphone app [21] is a cloud-based platform developed at Maastricht University and Maastricht UMC (Multimedia Appendix 1). It is a tool for repeated momentary assessments in daily life that has been extensively studied and refined in mental health care [22]. In this study, the PsyMate was programmed to prompt participants 8 times a day over 6 consecutive days with an auditory and visual signal (“beep”) to fill in a short momentary assessment. A high sampling frequency of 8-10 beeps per day was thought to provide sufficient insight into various daily contexts while not disrupting the flow of everyday life. The duration of 6 days was chosen to capture both weekdays and weekend days. This set-up was based on previous feasibility studies [23,24]. Beeps occurred unpredictably in semirandom time blocks of 112.5 minutes between 7:30 AM and 10:30 PM and were available to be filled in for 15 minutes after the beep. In total, 27 ESM items were included and could be answered on a 7-point Likert scale or in a multiple-choice set-up (Multimedia Appendix 2) assessing mood (eg, “I feel cheerful”), physical well-being (eg, “I feel tired”), subjective cognition (eg, “Since the last beep, I had memory problems”), and context (eg, “Where am I?”). Participants classified their responses individually, meaning that “work,” for example, could mean paid employment for one individual while another individual selected this option for gardening or doing chores. A morning and evening questionnaire was also part of the ESM, asking the participant to reflect on the previous night (eg, “I slept well”) and day (eg, “Generally, I felt tense today”). These questionnaires were not prompted via beeps but were available during the morning and evening, to be filled in self-reliantly, and this data was not included in this study (Multimedia Appendix 3). The development of this questionnaire was based on previous ESM studies [22,25,26]. Questions on subjective cognition were added after consulting with ESM experts and clinicians (ie, psychologists, psychiatrists, and neuropsychologists from the UMC).

    Procedure

    Participants were approached via the Alzheimer Center Limburg research database, consisting of patients with cognitive impairments who had previously expressed interest in being contacted for research purposes and had been previously recruited through UMC or by their treating health care professional at the memory clinic. A member of the research team called potential participants, checked general eligibility, verbally explained the study, and sent out information sheets. Participants were called by phone 1 week later, and if willing to participate, a date for the orientation session was set. A standardized protocol was used: (1) an orientation session, (2) an ESM training session, (3) a 6-day ESM period, and (4) a debriefing session. Only the person with MCI participated in the ESM training, the ESM period, and the debriefing session, but both the person with MCI and their relevant other were present at the orientation session. Sessions took place either at the hospital or at the participant's home, depending on the participant's preference. Participants could drop out at any time without providing a reason.

    Orientation Session

    After the study procedure was explained once more and final questions were clarified, informed consent was signed by the person with MCI and their relevant other. Next, sociodemographic information was collected and questionnaires were filled in assessing characteristics of the person with MCI either with self- or proxy-reports. At the end of this session, a date for the ESM training session was set. The ESM training was not combined with the orientation session so as not to overburden participants (as filling in a range of questionnaires can potentially be intense, confronting, and tiring). Thereby, we hoped to prevent participants from forgetting the ESM-training instructions due to information overload.

    ESM Training Session

    During the 30-minute training session, the PsyMate app was installed on the participant's smartphone, and the participant was instructed on how to respond to beeps, operate the app, and interpret the momentary questions. An example ESM questionnaire was filled in to familiarize participants with the procedure. The management of the app was observed, guided by the Management of Everyday Technology Assessment (META), to get a detailed picture of the human-technology interaction and performance skills [27]. All participants were briefed individually. A leaflet containing all instructions and contact information was handed out.

    ESM Period

    The PsyMate started sending beeps from the moment of installation; participants could respond on this day to train for filling in the momentary assessments, but they were instructed that the official 6-day ESM period would start the following day. On the second ESM day, a researcher called to check-in and solve potential technical problems or provide clarification.

    Debriefing Session

    This session took place 1 day after the last day of the ESM period. Participants were asked to report their general experiences using the app, and they received travel reimbursements and a small gift after participation but no financial reward.

    Instruments

    Sociodemographic and Descriptive Information

    Next to the sociodemographic information of the person with MCI (age, sex, education, living situation, years since first symptoms) and their relevant other (age, sex, relationship to person with MCI), reliable and valid instruments were filled in with the purpose of describing the sample. The Mini–Mental State Examination (MMSE) provided information on cognitive functioning [28]. If the MMSE had been administered by a health care professional at the memory clinic in the past 3 months, these scores were used to reduce the burden. Otherwise, the MMSE was part of the orientation session. Furthermore, the Guidelines for the Rating of Awareness Deficits (GRAD) were included as a semistructured interview to assess the degree of awareness of one’s own cognitive problems [29]; the GRAD compares the patient's information and the relevant other's view on the patient's history. Impaired awareness is defined as the absence of knowledge recognition of cognitive deficits and its impact [29]. The Hospital and Anxiety Depression Scale (HADS) was included to generate scores for generalized anxiety and depression [30,31], and the Perceived Stress Scale (PSS) measured the perception of stress [32]. The relevant other filled in the Neuropsychiatric Inventory Questionnaire (NPI-Q) for information on a variety of neuropsychiatric symptoms [33,34] as well as the Amsterdam Instrumental Activities of Daily Living (Amsterdam IADL), which specifically measures problems in instrumental activities in individuals with mild cognitive problems [35].

    Feasibility Assessment: Self-Report

    The feasibility was determined through the compliance rate of the ESM assessments and was regarded as satisfactory when >70% of the momentary questionnaires were filled in [12,23]. The subjective experience of using the ESM tool was assessed during the debriefing session through a semistructured interview, including ratings of the difficulty, time burden, interference with daily activities, and overall acceptability of the methodology. This interview followed a standardized protocol and included questions such as “Was this a normal week?” or “Did the PsyMate app hinder your daily occupations?” which were discussed and then rated by the participant on a 7-point Likert scale or categorically.

    Feasibility Assessment: Observations

    The Management of Everyday Technology Assessment (META) [36] was used during the ESM training session. This tool aims to identify the ability to manage technology among older adults with and without cognitive impairments by observing the direct human-technology interaction. The META consists of 4 parts, assessing (1) observable performance skills, (2) environmental characteristics, (3) the person's capacity, and (4) the perceived importance of the used technology. The fourth part (the perceived importance of the technology), as well as general information about the technology (eg, years of possession, amount of use), is answered by the individual via an interview. The first 3 parts are scored by the investigator on a 4-point scale: 4=competent handling and management (ie, no deficits in this skill disturbs or hinders the person's use of the technology; no difficulty); 3=deficits in this skill occasionally or slightly disturb the person's use of the technology (minor difficulty/problems); 2=deficits in this skill obviously disturb the person's use of the technology (major difficulty/problem); 1= deficits in this skill hinder the person's use of the technology, or the person is in need of assistance to perform the skill competently. For the first part (observable performance skills), 6 out of 11 performance skills were selected and scored, as the other 5 were not part of using a smartphone app (eg, coordinate different physical parts of the technology).

    Statistical Analyses

    Descriptive analyses were conducted to summarize the sociodemographic information and background questionnaire scores. The compliance rate of the ESM day questionnaires, responses to the debriefing questionnaires, and META scores of the human-technology interaction were also analyzed using descriptive statistics. For the usability demonstration, only participants who filled in at least 30% of the ESM assessments were included, as a sufficient amount of information needs to be available to describe daily patterns [37]. Thus, momentary ESM data from dropouts collected via the PsyMate were not included in this part of the analysis. The momentary data were demonstrated on a group level using mood, context, feelings of tiredness, and subjective cognition items, and analyzed with descriptive statistics to assess the usability in this population in general [12]. The variable positive affect (PA) consisted of the ESM items “I feel cheerful,” “I feel energetic,” “I feel relaxed,” “I feel satisfied,” and “I feel enthusiastic.” The negative affect (NA) included “I feel down,” “I feel insecure,” “I feel irritated,” “I feel lonely,” “I feel anxious,” and “I feel guilty.” To demonstrate elements of daily functioning, an activity-related stress (ARS) variable was generated using “I can do this well” (reversed), “This requires effort from me,” and “I would rather do something else.” The ESM data collected with the PsyMate has a multilevel structure with beeps (level 1) nested within participants (level 2). Average scores of PA, NA, and ARS were thus person-mean centered to take the within-person effect into account. Factor analyses were conducted to ensure sufficient internal validity (Cronbach α=.86, .84, and .68 for the PA, NA, and ARS, respectively). PA, NA, and ARS were average, on a person-mean level. On an exemplary level, the subjectively experienced cognitive problems of 3 participants were demonstrated with line graphs over the course of the ESM period. The 3 participants were selected without specific criteria but with the aim of showing variation, giving a general impression of the ESM data, and demonstrating how the data can be used in clinical practice prospectively. Daily functioning using ARS was also demonstrated on an individual level by using data from 3 participants exemplary. Stata statistical software (version 13.0; StataCorp) was used for statistical analyses, and Excel (version 16.16.19; Microsoft) was used to create graphic visualizations.

    Data Availability Statement

    The data is stored at Maastricht University. Due to ethical and legal regulations, the data is only accessible for the MUMC+ research team. Sharing data with another research team needs to be approved by the Medical Research Ethics Committee azM/UM, or participants need to sign a new informed consent sheet.


    Results

    Group Characteristics

    A total of 152 people with MCI were approached to participate in this study; 21 people with MCI signed informed consent. The participant flow is illustrated in Figure 1. Their relevant others also agreed to participate; the relevant others of the study participants had a mean age of 63.3 (SD 8.9, range 47-78) years, 6 were men and 15 were women, and 19 were the partners of the participants while 1 was a sibling and 1 was a friend. Table 1 provides an overview of the characteristics of the total sample. Multimedia Appendix 4 shows the details of the study completers and dropouts.

    Figure 1. Flow chart of recruited participants with mild cognitive impairment (MCI). ESM: experience sampling method.
    View this figure
    Table 1. Descriptive information about the participants with mild cognitive impairment (n=21).
    View this table

    Dropouts, Compliance, and Self-Report

    In the study, 21 individuals started the ESM period, resulting in 673 beep records; 3 participants had problems using the ESM and did not complete the ESM period. These 3 dropouts had been eager to learn the app during the training session and their MMSE, other questionnaire scores, and general impression did not deviate outstandingly from the other participants. A statistical comparison between study completers and dropouts was not performed due to the small sample size.

    The reasons for dropout were the following: Person A had problems using the right force pressing app buttons during the training session, forgot hearing aids repeatedly (according to partner) and thus did not react to the beeps, did not carry the smartphone along at all times, forgot the appointment, and seemed to generally deny cognitive problems. Person B expressed being very busy, only heard “some beeps” (no hearing problems, technical problems are unlikely according to IT specialist, reason unclear), and forgot the appointment for the debriefing session. Person C seemed generally nervous during the ESM training session (while expressing strong interest to participate), required very detailed and simple explanations of app use, and had forgotten instructions when contacted the following day. These 3 participants had not filled in the required 30% (16 beeps) to be included in the usability analysis, leading to a loss of 17 records (2.3%).

    Of the 21 participants, 18 completed the ESM period and debriefing session, resulting in 656 valid beep records. On average, participants completed 38 beeps (SD 6.8; range 23-47) of the 48 beeps. The ESM compliance rate was 78.7%. Participants thought that the momentary questions reflected their experiences well (mean 4.83, SD 1.62) and that the PsyMate had little influence on their mood (mean 1.44, SD 1.15), activities (mean 1.61, SD 1.54), social interactions (mean 1.22, SD 0.73), or daily occupations (mean 1.39, SD 0.85). Filling in the momentary questions made participants marginally more aware of their activities (mean 2.17, SD 1.86) and moderately aware of their feelings (mean 3.56, SD 2.45) and memory (mean 4.56, SD 2.5). Of the 18 participants, 4 found increased awareness of their memory to be unpleasant, while 13 experienced it as pleasant or neutral. Table 2 provides detailed information on the general experience with the PsyMate and user-friendliness.

    Table 2. General PsyMate app and user-friendliness evaluation [n=18; dropouts (n=3) were not included because they did not participate in the debriefing session].
    View this table

    Observation of the Human-Technology Interaction

    The META revealed that most performance steps involved in using the PsyMate did not cause any difficulties (Table 3). However, using the appropriate force, tempo, and precision caused, on average, some disturbances (mean 3.48, SD 0.51). With regard to the environmental characteristics influencing the use of the PsyMate app during the training session, the contextual influence (ie, the presence of researchers, which could be potentially stressful) was observed as not hindering smartphone use (mean 3.9, SD 0.3; range 3-4), while the technological design (ie, screen and button size) was observed as somewhat disturbing (mean 3.38, SD 0.3; range 2-4). The overall judgment of the participants' capacity to use the app was reflected in the capacity to recall necessary information as not disturbing (mean 3.86, SD 0.36; range 3-4), just like the capacity to pay attention and focus (mean 3.81, SD 0.40; range 3-4) and the capacity to manage stress (mean 3.76, SD 0.45; range 3-4). Of the 21 participants, most participants (12/21) had had smartphones for more than 10 years; 5 had used a smartphone for 3-9 years, 1 had used it for 1-2 years, and 2 had it for less than 1 year (1 participant could not indicate the duration). All 21 participants experienced the technology as very important and not replaceable; 18 used their smartphones daily and 2 used it at least weekly (for the remaining 1 participant, there is a missing value).

    Table 3. Assessment of observable performance skills when using the PsyMate app during the experience sampling method (ESM) training session (n=18).
    View this table

    Usability of the Experience Sampling Data

    Daily Patterns on a Group Level Over ESM Period

    In general, the 18 participants experienced a high level of PA (mean 4.95, SD 0.66; range 3.94-6.13), a low level of NA (mean 1.95, SD 0.93; range 1.07-3.92) and a low to moderate level of ARS (mean 2.73, SD 0.74; range 1.71-4.05). They felt moderately tired (mean 3.64, SD 1.39; range 1-6.29) and had low to moderate problems with their memory (mean 3.01, SD 1.11; range 1.34-5.29), language (mean 2.04, SD 1.15; range 1-5.21), and concentration (mean 2.85, SD 1.36; range 1.05-4.96). With regard to their contextual patterns, participants spent most of their time at home (72%) (other locations: transport [9%]; at family's/friend's [5%]; at work [5%]; somewhere else [5%]; public place [4%]) engaging in household (22%) or relaxing activities (29%) (other activities: eating/drinking [10%]; something else [10%]; work [8%]; nothing [6%]; traveling [4%]; in conversation [4%]; self-care [4%]; sport/physical activity [3%]), and were often in company of their partner (45%) or alone (31%) (other social company: family [8%]; colleagues [4%]; co-occupants [4%]; friends [3%]; acquaintances [3%]; strangers [2%]).

    Individual Profiles

    To illustrate the variability that can be studied using momentary data, several descriptive examples are presented, focusing on the subjectively experienced cognitive problems, daily activities, and activity-related stress in everyday life. These participants were selected without specific criteria but with the aim of visually illustrating fluctuations within subjects, variables, and days (Figures 2-4). An unspecified heterogeneity is present, while no statistical differences within and between subjects were tested. Some suggestions for personalized feedback conversations between health care professionals and the individuals are provided as well.

    As shown in Figure 2 and Table 4, Person 1 reports mainly moderate memory problems, while language and concentration abilities are overall subjectively unimpaired. Person 1 engages in doing “nothing” 24% of the time. This activity shows personally higher levels of ARS, while “relaxing” has lower levels of ARS. The person engages in “work” (note: not necessarily paid) 10% of the time, which also shows a personally higher level of ARS. When discussing this data, increased engagement in relaxation and coping with work could be targeted.

    Figure 2. Person 1: subjectively experienced cognitive problems.
    View this figure
    Table 4. Person 1: daily activities and levels of activity-related stress.
    View this table

    As shown in Figure 3 and Table 5, Person 2 reports cognitive problems that fluctuate across all 3 domains. Conversations (2% of activity engagement) seem to be the most stressful (personally higher level of ARS). In this case, dealing with cognitive problems and developing coping strategies for conversations might be useful for the individual.

    Figure 3. Person 2: subjectively experienced cognitive problems.
    View this figure
    Table 5. Person 2: daily activities and levels of activity-related stress.
    View this table

    As shown in Figure 4 and Table 6, Person 3 has subjective cognitive impairments in all 3 domains that fluctuate somewhat simultaneously. On some mornings, the cognitive problems seem to be milder. Relaxation activities, which report low levels of ARS, are the main activity of Person 3 (48%). Nevertheless, doing nothing (12%) and working (7%) might be topics to discuss to optimize self-management.

    Figure 4. Patient 3: subjectively experienced cognitive problems.
    View this figure
    Table 6. Person 3: daily activities and levels of activity-related stress.
    View this table

    Discussion

    Principal Findings

    This study evaluates the feasibility and demonstrates the usability of a smartphone-based ESM in people with MCI. Several important findings emerged: (1) in study completers, the compliance rate was high and subjective ratings of the ESM procedure were positive; (2) the observable human-technology interaction between participants and the ESM app was generally unproblematic; (3) raising awareness for one’s own cognitive problems through ESM can be unpleasant for some individuals; and (4) cognitive issues (eg, forgetfulness) may lead to the inability to use the ESM.

    Previous research found that the compliance rate, also referred to as adherence, use, or engagement, to technology-based self-monitoring such as ESM lies between 51-86% in middle-aged and older adults [9,38]. Therefore, the reported 78.7% of completed assessments in this study is a strong indication for the feasibility of ESM in a majority (18/21, 85%) of our MCI sample. A high sampling frequency, which was applied in this study with 8 beeps per day, is not thought to hinder ESM use, while the length of the questionnaires can increase burden [39]. The overall positive participants’ feedback on the procedure, including the length and frequency, supports the chosen ESM set-up. Additionally, the human-technology interactions were observed as overall unproblematic. Occasionally, a participant's inappropriate tempo, force, or precision of clicking on app buttons disturbed usage slightly. As older adults may benefit from large buttons and screens without scroll functions [40], it would be advisable to rotate the screen, increase button sizes, or provide a touch-pen to ease the app use even further. In addition to dexterity, older adults might also have hearing issues. In this study, one of the dropouts did not wear their hearing aids, which might have contributed to their inability to use the ESM and discontinuation of the trial.

    According to the social cognition theory, self-monitoring can raise awareness for one’s own emotions or behaviors [41]. Repeated momentary assessments can use this increased awareness to promote behavioral changes towards healthy lifestyles [22,42] as well as improve mental well-being [43]. Within this study, there was no intention to change daily patterns, but nevertheless, participants became more aware of their memory abilities through repeated self-assessments. Of the 18 participants, 13 individuals experienced this as pleasant or neutral, but 4 reported this to be unpleasant. Similar side-effects of the ESM have previously been reported, and a suggestion could be to use positive formulations in the ESM questions [44]. For example, instead of asking about cognitive problems (eg, “Since the last beep, I had memory problems”), abilities could be targeted instead (eg, “I can remember well”). In this study, the experience of using ESM was discussed during the debriefing session, and in 1 case, a participant was advised to consult a health care professional for further treatment for cognition-related stress. In clinical settings, treating health care professionals may discuss experiences and increased awareness to develop coping strategies [45]. The individual profiles section highlights topics that may be discussed on an individual level, such as activities that elicit low stress (eg, relaxation) or the potential need for assistance or new coping strategies. Studies suggest that those feedback conversations could focus on positive emotions to increase resilience to stress [46] and stimulate goal-directed behavior [47]. This kind of feedback has shown to improve well-being, for instance, in an ESM-based intervention for carers of people with dementia [26]. The ultimate goal when using ESM is to support self-management through increase awareness for one's own abilities and orientate attention towards positive and meaningful aspects of daily life.

    A small number of participants were unable to complete the experience sampling period. In older adults with undiagnosed subjective cognitive concerns, nonadherence to momentary assessments is thought to be greatly influenced by cognitive issues [48]. In this study, predicting dropouts using standardized instruments such as the MMSE was impossible. A systematic review reports that the averaged MMSE score in MCI samples seems to range from 23.1 to 28.7 [19], indicating a great variability of cognitive abilities in this population and that participants of this study potentially had relatively mild MCI. However, the MMSE has a limited discrimination between cognitively health adults and people with MCI, and other tests with a higher sensitivity (eg, Hopkins Verbal Learning Test) could have been used to determine study eligibility [49]. All participants were eager to participate, while no clear indication for exclusion could be identified. Follow-up phone calls were helpful to notice difficulties early. Dropouts seem to blame their inability to participate on the technology (eg, “It did not beep”). Admitting problems with technology might be easier than admitting other cognitive deficits, as even young and healthy individuals may occasionally face difficulties with technology. Further, reduced illness insight and cognitive deficits could have influenced the ability to use the ESM. The latter is supported by reports from participants and relevant others, stating that smartphones or hearing aids were forgotten, thus interfering with the ESM use. To prevent injustice in health care, all individuals with MCI motivated to use the ESM should be given a chance to do so, and frustration can be prevented through follow-ups, close guidance, and open communication.

    Generally, the ESM group data revealed subjective problems with memory, concentration, and language in everyday life. This finding is in line with traditional neuropsychological assessments reporting a variety of cognitive deficits in MCI, of which memory is commonly most dominant [50]. A moderate level of fatigue has also been found in a healthy sample using the ESM [24] and may thus not be directly related to the cognitive deficits. To determine significant differences from healthy older adults, a control group is prospectively necessary. Furthermore, associations between daily fatigue, context, mood, and cognitive problems experienced by individuals may be studied using multilevel analysis [12].

    Future Directions

    On an individual level, cognitive fluctuations indicate trends of diversity both within and between subjects. The heterogeneity of the MCI group has been highlighted before [51], but this is one of the first studies to provide such a detailed insight into daily patterns using smartphone-based ESM. Next to the subjective evaluation of cognitive problems in everyday life, objective momentary cognition tasks can be added to this ESM app. The feasibility of 2 tasks has recently been tested in healthy individuals [52] and holds promise for future studies to describe a comprehensive picture of cognitive abilities. The ESM may also be useful to compare daily patterns of subjective or objective cognitive functioning in different neurological and psychiatric disorders.

    Additionally, activity-related stress levels seem to vary between activities as working, for example, shows a trend for high levels of stress. This study is unfortunately not able to statistically explore activity patterns in people with MCI, but future research might follow up on this idea. Research shows that complex tasks are affected early on in the process of cognitive decline [53], and cognitive difficulties may decrease the ability of individuals with MCI to work [54]. Our understanding of necessary adjustments and ways to support working, particularly employment, when living with MCI are limited [55], but the insights gained from the participants in this study highlight the need to study working and employment as a potential stressor in this population further. High levels of anxiety and depression are commonly observed in MCI [56], as they were in this sample, and may also stand in relation to stress and cognitive deterioration [57]. As highlighted above, developing coping strategies and focusing on positive emotions might support daily well-being, including work-related stress. The ESM can be a useful tool to relate functional fluctuations with contexts and activities and thus understand patterns and networks in people with MCI, both on a within- and between-subject level [58].

    Limitations

    Some critique regarding the ESM and study limitations need to be acknowledged. It is recommended not to overinterpret single items but rather to use momentary data as a starting point for a conversation about one's self-management and coping. Generally, many people (n=70) approached for the study had no smartphone or did not feel confident to participate in a smartphone-based study. This outcome indicates that there is a bias towards individuals with a higher technology familiarity to benefit from digital innovations in research and clinical work. Over the next decade, this bias might decrease, but researchers and clinicians need to be aware of this gap to not neglect individuals in need of support. Potentially, traditional paper-pencil diaries might be an alternative for people with MCI [16,17] that cannot or do not want to use smartphones. However, cognitive problems (eg, forgetting paper diary) or hearing problems (eg, not hearing the beeps from a prompting device) could still interfere. As learning and using a new technology is an intertwined process [59], and training is a key component for older adults to increase confidence and self-efficacy when using technology [60], prospective individuals with MCI who are not confident in their abilities to use a smartphone could receive training sessions and additional guidance. Unfortunately, this study is not able to determine if individuals with MCI would also be able to learn smartphone and ESM use together. Furthermore, the findings may be affected by a sex and education bias, as 76% (16/21) were men and only 10% (2/21) were low-educated. In addition, the etiology of MCI was not determined, resulting in an unspecified heterogeneity. As indicated by the MMSE, this MCI sample might have relatively mild cognitive problems, and a replication of our findings in a broader MCI sample might be necessary to increase the generalizability of the results. Detailed descriptive information about MCI subgroups could prospectively be added. The small sample size orientated on other feasibility studies [12,61] may limit the generalizability of the results, but the great number of assessments still result in a rich data set [22]. Finally, the study represents a specific group of people with MCI in possession of their own smartphones, and this recruitment criteria needs to be kept in mind when applying the ESM in future studies or clinical settings.

    Conclusion

    Technology-based ESM can be a useful addition to clinical questionnaires to reveal detailed moment-to-moment fluctuations, contextual patterns, and individual differences in subjectively experienced cognitive problems, affect, and activities. This feasibility study is a relevant step to better understand and support people with MCI in their everyday lives. Momentary data may prospectively be used to study individual and group-based patterns in this population and develop person-tailored self-management strategies.

    Acknowledgments

    The authors thank all participants that took part in this study, as well as Roos Roberts, Astrid Quist, and Karel Borkelmans for their contribution to data collection. This research was carried out as part of the H2020 Marie Skłodowska-Curie Actions Innovative Training Network (ITN) action, H2020-MSCA-ITN-2015, under grant agreement number 676265.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Interface of the experience sampling method (ESM) smartphone app "PsyMate.".

    PNG File , 89 KB

    Multimedia Appendix 2

    Experience sampling method (ESM) item list of the day questionnaire.

    DOCX File , 15 KB

    Multimedia Appendix 3

    Morning and evening questionnaires.

    DOCX File , 14 KB

    Multimedia Appendix 4

    Descriptive information of the study completers (n=18) and dropouts (n=3).

    DOCX File , 16 KB

    References

    1. Van den Bergh O, Walentynowicz M. Accuracy and bias in retrospective symptom reporting. Current Opinion in Psychiatry 2016;29(5):302-308. [CrossRef]
    2. Moore RC, Depp CA, Wetherell JL, Lenze EJ. Ecological momentary assessment versus standard assessment instruments for measuring mindfulness, depressed mood, and anxiety among older adults. Journal of Psychiatric Research 2016 Apr;75:116-123. [CrossRef]
    3. Geukes K, Nestler S, Hutteman R, Küfner A, Back M. Trait personality and state variability: Predicting individual differences in within- and cross-context fluctuations in affect, self-evaluations, and behavior in everyday life. Journal of Research in Personality 2017 Aug;69:124-138. [CrossRef]
    4. van der Krieke L, Blaauw F, Emerencia A, Schenk H, Slaets J, Bos E, et al. Temporal Dynamics of Health and Well-Being: A Crowdsourcing Approach to Momentary Assessments and Automated Generation of Personalized Feedback. Psychosom Med 2017;79(2):213-223. [CrossRef] [Medline]
    5. Csikszentmihalyi M, Larson R. Validity and reliability of the experience-sampling method. In: Flow and the foundations of positive psychology. Dordrecht: Springer; 2014:35-54.
    6. Shiffman S, Stone AA, Hufford MR. Ecological Momentary Assessment. Annu. Rev. Clin. Psychol 2008 Apr;4(1):1-32. [CrossRef]
    7. Scollon C, Prieto CK, Diener E. Experience sampling: promises and pitfalls, strength and weaknesses. In: Assessing well-being. Dordrecht: Springer; 2009:157-180.
    8. Myin-Germeys I, Klippel A, Steinhart H, Reininghaus U. Ecological momentary interventions in psychiatry. Curr Opin Psychiatry 2016 Jul;29(4):258-263. [CrossRef] [Medline]
    9. Bartels SL, van Knippenberg RJ, Dassen FC, Asaba E, Patomella A, Malinowsky C, et al. A narrative synthesis systematic review of digital self-monitoring interventions for middle-aged and older adults. Internet Interventions 2019 Dec;18:100283. [CrossRef]
    10. Barlow JH, Ellard DR, Hainsworth JM, Jones FR, Fisher A. A review of self-management interventions for panic disorders, phobias and obsessive-compulsive disorders. Acta Psychiatr Scand 2005 Apr;111(4):272-285. [CrossRef]
    11. Maher JP, Rebar AL, Dunton GF. Ecological Momentary Assessment Is a Feasible and Valid Methodological Tool to Measure Older Adults’ Physical Activity and Sedentary Behavior. Front. Psychol 2018 Aug 15;9. [CrossRef]
    12. Lenaert B, Colombi M, van Heugten C, Rasquin S, Kasanova Z, Ponds R. Exploring the feasibility and usability of the experience sampling method to examine the daily lives of patients with acquired brain injury. Neuropsychological Rehabilitation 2017 May 31;29(5):754-766. [CrossRef]
    13. Johnson EI, Sibon I, Renou P, Rouanet F, Allard M, Swendsen J. Feasibility and validity of computerized ambulatory monitoring in stroke patients. Neurology 2009 Nov 09;73(19):1579-1583. [CrossRef]
    14. Albert MS, DeKosky S, Dickson D, Dubois B, Feldman H, Fox N, et al. The Diagnosis of Mild Cognitive Impairment due to Alzheimer’s Disease: Recommendations from the National Institute on Aging-Alzheimer’s Association Workgroups on Diagnostic Guidelines for Alzheimer’s Disease. FOC 2013 Jan;11(1):96-106. [CrossRef]
    15. Parikh PK, Troyer AK, Maione AM, Murphy KJ. The Impact of Memory Change on Daily Life in Normal Aging and Mild Cognitive Impairment. GERONT 2015 Apr 09;56(5):877-885. [CrossRef]
    16. Rickenbach EH, Condeelis KL, Haley WE. Daily stressors and emotional reactivity in individuals with mild cognitive impairment and cognitively healthy controls. Psychology and Aging 2015 Jun;30(2):420-431. [CrossRef]
    17. Rickenbach EH, Almeida D, Seeman T, Lachman ME. Daily stress magnifies the association between cognitive decline and everyday memory problems: an integration of longitudinal and diary methods. Psychol Aging 2014 Dec;29(4):852-862 [FREE Full text] [CrossRef] [Medline]
    18. Malinowsky C, Almkvist O, Kottorp A, Nygård L. Ability to manage everyday technology: a comparison of persons with dementia or mild cognitive impairment and older adults without cognitive impairment. Disabil Rehabil Assist Technol 2010;5(6):462-469. [CrossRef] [Medline]
    19. Jekel K, Damian M, Wattmo C, Hausner L, Bullock R, Connelly P, et al. Mild cognitive impairment and deficits in instrumental activities of daily living: a systematic review. Alzheimers Res Ther 2015;7(1):17 [FREE Full text] [CrossRef] [Medline]
    20. van Os J, Delespaul P, Wigman J, Myin-Germeys I, Wichers M. Beyond DSM and ICD: introducing. World Psychiatry 2013 Jun;12(2):113-117 [FREE Full text] [CrossRef] [Medline]
    21. Maastricht University, smarteHealth, HealthHealth Foundation Limburg. PsyMate.   URL: http://www.psymate.eu/ [accessed 2020-09-10]
    22. Verhagen SJW, Hasmi L, Drukker M, van Os J, Delespaul PAEG. Use of the experience sampling method in the context of clinical trials: Table 1. Evid Based Mental Health 2016 Jul 21;19(3):86-89. [CrossRef]
    23. van Knippenberg RJM, de Vugt ME, Ponds R, Myin-Germeys I, van Twillert B, Verhey FRJ. Dealing with daily challenges in dementia (deal-id study): an experience sampling study to assess caregiver functioning in the flow of daily life. Int J Geriatr Psychiatry 2017 Sep;32(9):949-958. [CrossRef] [Medline]
    24. Verhagen S, Daniëls N, Bartels S, Tans S, Borkelmans K, de Vugt M, et al. Measuring within-day cognitive performance using the experience sampling method: A pilot study in a healthy population. PLoS One 2019;14(12):e0226409 [FREE Full text] [CrossRef] [Medline]
    25. van Os J, Verhagen S, Marsman A, Peeters F, Bak M, Marcelis M, et al. The experience sampling method as an mHealth tool to support self-monitoring, self-insight, and personalized health care in clinical practice. Depress Anxiety 2017 May 23;34(6):481-493. [CrossRef]
    26. van Knippenberg RJM, de Vugt ME, Ponds R, Myin-Germeys I, Verhey FRJ. An Experience Sampling Method Intervention for Dementia Caregivers: Results of a Randomized Controlled Trial. Am J Geriatr Psychiatry 2018 Dec;26(12):1231-1243. [CrossRef] [Medline]
    27. Bartels SL, Assander S, Patomella A, Jamnadas-Khoda J, Malinowsky C. Do you observe what I perceive? The relationship between two perspectives on the ability of people with cognitive impairments to use everyday technology. Aging & Mental Health 2019 May 06;24(8):1295-1305. [CrossRef]
    28. Folstein MF, Folstein S, McHugh PR. “Mini-mental state”. Journal of Psychiatric Research 1975 Nov;12(3):189-198. [CrossRef]
    29. Verhey FRJ, Rozendaal N, Ponds RWHM, Jolles J. Dementia, awareness and depression. Int. J. Geriat. Psychiatry 1993 Oct;8(10):851-856. [CrossRef]
    30. Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand 1983 Jun;67(6):361-370. [CrossRef]
    31. Bjelland I, Dahl A, Haug T, Neckelmann D. The validity of the Hospital Anxiety and Depression Scale. Journal of Psychosomatic Research 2002 Feb;52(2):69-77. [CrossRef]
    32. Cohen S, Kamarck T, Mermelstein R. A Global Measure of Perceived Stress. Journal of Health and Social Behavior 1983 Dec;24(4):385. [CrossRef]
    33. Cummings JL. The Neuropsychiatric Inventory: Assessing psychopathology in dementia patients. Neurology 1997 May 01;48(Issue 5, Supplement 6):10S-16S. [CrossRef]
    34. Kat K, De Jonghe JFM, Aalten P, Kalisvaart CJ, Dröes RM, Verhey FRJ. Neuropsychiatrische symptomen bij dementie: Psychometrische aspecten van de Nederlandse Neuropsychiatric Inventory (NPI). Tijdschrift voor Gerontologie en Geriatrie 2002;33(4):150-155.
    35. Jutten RJ, Peeters C, Leijdesdorff S, Visser P, Maier A, Terwee C, et al. Detecting functional decline from normal aging to dementia: Development and validation of a short version of the Amsterdam IADL Questionnaire. Alzheimers Dement (Amst) 2017;8:26-35 [FREE Full text] [CrossRef] [Medline]
    36. Malinowsky C, Nygård L, Kottorp A. Psychometric evaluation of a new assessment of the ability to manage technology in everyday life. Scandinavian Journal of Occupational Therapy 2009 Dec 14;18(1):26-35. [CrossRef]
    37. Delespaul PAEG. Assessing schizophrenia in daily life: the experience sampling method. Maastricht: UPM; 1995.
    38. Cain AE, Depp C, Jeste DV. Ecological momentary assessment in aging research: a critical review. J Psychiatr Res 2009 Jul;43(11):987-996 [FREE Full text] [CrossRef] [Medline]
    39. Eisele G, Vachon H, Lafit G, Kuppens P, Houben M, Myin-Germeys I, et al. The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population. https://doi.org/10.31234/osf.io/zf4nm 2020. [CrossRef]
    40. Petrovčič A, Taipale S, Rogelj A, Dolničar V. Design of Mobile Phones for Older Adults: An Empirical Analysis of Design Guidelines and Checklists for Feature Phones and Smartphones. International Journal of Human–Computer Interaction 2017 Sep 05;34(3):251-264. [CrossRef]
    41. Bandura A. Health promotion from the perspective of social cognitive theory. Psychology & Health 1998 Jul;13(4):623-649. [CrossRef]
    42. Pejovic V, Lathia N, Mascolo C, Musolesi M. Mobile-based experience sampling for behaviour research. In: Emotions and personality in personalized services. Cham: Springer; 2016:141.
    43. Kramer I, Simons C, Hartmann J, Menne-Lothmann C, Viechtbauer W, Peeters F, et al. A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry 2014 Feb;13(1):68-77 [FREE Full text] [CrossRef] [Medline]
    44. van Knippenberg RJM, de Vugt ME, Smeets C, Myin-Germeys I, Verhey F, Ponds RW. Dealing with daily challenges in dementia (deal-id study): process evaluation of the experience sampling method intervention 'Partner in Sight' for spousal caregivers of people with dementia. Aging Ment Health 2018 Sep;22(9):1199-1206. [CrossRef] [Medline]
    45. Hartmann JA, Wichers M, Menne-Lothmann C, Kramer I, Viechtbauer W, Peeters F, et al. Experience sampling-based personalized feedback and positive affect: a randomized controlled trial in depressed patients. PLoS One 2015;10(6):e0128095 [FREE Full text] [CrossRef] [Medline]
    46. Fredrickson BL. Positive emotions broaden and build. Advances in experimental social psychology 2013;47:1-53 [FREE Full text] [CrossRef]
    47. Folkman S. Positive psychological states and coping with severe stress. Social Science & Medicine 1997 Oct;45(8):1207-1221 [FREE Full text] [CrossRef]
    48. Ramsey AT, Wetherell JL, Depp C, Dixon D, Lenze E. Feasibility and Acceptability of Smartphone Assessment in Older Adults with Cognitive and Emotional Difficulties. Journal of Technology in Human Services 2016 May 06;34(2):209-223. [CrossRef]
    49. de Jager CA, Schrijnemaekers AMC, Honey TEM, Budge MM. Detection of MCI in the clinic: evaluation of the sensitivity and specificity of a computerised test battery, the Hopkins Verbal Learning Test and the MMSE. Age and Ageing 2009 May 19;38(4):455-460. [CrossRef]
    50. Ribeiro F, de Mendonça A, Guerreiro M. Mild cognitive impairment: deficits in cognitive domains other than memory. Dement Geriatr Cogn Disord 2006;21(5-6):284-290. [CrossRef] [Medline]
    51. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med 2004 Sep;256(3):183-194. [CrossRef]
    52. Daniëls NEM, Bartels S, Verhagen S, Van Knippenberg RJM, De Vugt ME, Delespaul PAEG. Digital assessment of working memory and processing speed in everyday life: Feasibility, validation, and lessons-learned. Internet Interv 2020 Mar;19:100300 [FREE Full text] [CrossRef] [Medline]
    53. Beaver J, Wilson KB, Schmitter-Edgecombe M. Characterising omission errors in everyday task completion and cognitive correlates in individuals with mild cognitive impairment and dementia. Neuropsychological Rehabilitation 2017 Jun 14;29(5):804-820. [CrossRef]
    54. Silvaggi F, Leonardi M, Tiraboschi P, Muscio C, Toppo C, Raggi A. Keeping People with Dementia or Mild Cognitive Impairment in Employment: A Literature Review on Its Determinants. Int J Environ Res Public Health 2020 Jan 29;17(3):a [FREE Full text] [CrossRef] [Medline]
    55. McCulloch S, Robertson D, Kirkpatrick P. Sustaining people with dementia or mild cognitive impairment in employment: A systematic review of qualitative evidence. British Journal of Occupational Therapy 2016 Nov 16;79(11):682-692. [CrossRef]
    56. Yates JA, Clare L, Woods RT. What is the Relationship between Health, Mood, and Mild Cognitive Impairment? JAD 2016 Dec 06;55(3):1183-1193. [CrossRef]
    57. Koyanagi A, Oh H, Vancampfort D, Carvalho A, Veronese N, Stubbs B, et al. Perceived Stress and Mild Cognitive Impairment among 32,715 Community-Dwelling Older Adults across Six Low- and Middle-Income Countries. Gerontology 2018 Sep 10;65(2):155-163. [CrossRef]
    58. Curran PJ, Bauer DJ. The Disaggregation of Within-Person and Between-Person Effects in Longitudinal Models of Change. Annu. Rev. Psychol 2011 Jan 10;62(1):583-619. [CrossRef]
    59. Rosenberg L, Kottorp A, Winblad B, Nygård L. Perceived difficulty in everyday technology use among older adults with or without cognitive deficits. Scand. J. of Occupational Therapy 2009:1-11. [CrossRef]
    60. Wagner N, Hassanein K, Head M. Computer use by older adults: A multi-disciplinary review. Computers in Human Behavior 2010 Sep;26(5):870-882. [CrossRef]
    61. Juengst SB, Graham KM, Pulantara IW, McCue M, Whyte EM, Dicianno BE, et al. Pilot feasibility of an mHealth system for conducting ecological momentary assessment of mood-related symptoms following traumatic brain injury. Brain Injury 2015 Aug 07;29(11):1351-1361. [CrossRef]


    Abbreviations

    Amsterdam IADL: Amsterdam instrumental activities of daily living
    ARS: activity-related stress
    azM: academisch ziekenhuis Maastricht
    ESM: experience sampling method
    GRAD: Guidelines for the Rating of Awareness Deficits
    HADS: Hospital and Anxiety Depression Scale
    MCI: mild cognitive impairment
    META: Management of Everyday Technology Assessment
    MMSE: Mini–Mental State Examination
    NA: negative affect
    NPI-Q: Neuropsychiatric Inventory Questionnaire
    PA: positive affect
    PSS: perceived stress scale
    UMC: Maastricht University Medical Center


    Edited by J Wang; submitted 04.05.20; peer-reviewed by E Broome, H Wright, W Eikelboom; comments to author 27.05.20; revised version received 16.07.20; accepted 28.07.20; published 16.10.20

    ©Sara Laureen Bartels, Rosalia J M van Knippenberg, Camilla Malinowsky, Frans R J Verhey, Marjolein E de Vugt. Originally published in JMIR Aging (http://aging.jmir.org), 16.10.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Aging, is properly cited. The complete bibliographic information, a link to the original publication on http://aging.jmir.org, as well as this copyright and license information must be included.