Participants were recruited from the memory clinic at the Maastricht University Medical Center (UMC) from June 2018 to January 2020. Inclusion criteria were (1) having a clinical diagnosis of MCI, according to Albert et al [14], (2) owning a smartphone with an Android or iOS operational system, (3) providing written informed consent, and (4) receiving written informed consent from a relevant other (ie, partner, family member, close friend) that was selected by the person with MCI and was recruited. Exclusion criteria were (1) insufficient abilities to participate in research (eg, a self-reported or relevant other–reported inability or lack of confidence to use a smartphone or to learn and remember the purpose of the study) and (2) severe health problems, such as a diagnosis of a somatic, psychiatric, or neurological disorder causing additional cognitive dysfunction. Both exclusion criteria were based on the clinical judgment of a psychologist or psychiatrist during the recruitment phase (eg, telephone conversations with a potential participant or relevant other).

The Medical Ethical Committee from the Maastricht academic hospital (azM) and Maastricht University approved the study (NL64310.068.17 / METC173055), and the protocol is registered on ToetsingOnline (64310). The authors comply with the Helsinki Declaration of 1975, as revised in 2008. All participants, including people with MCI and their relevant others, provided written informed consent before study participation.

The PsyMate smartphone app [21] is a cloud-based platform developed at Maastricht University and Maastricht UMC (Multimedia Appendix 1). It is a tool for repeated momentary assessments in daily life that has been extensively studied and refined in mental health care [22]. In this study, the PsyMate was programmed to prompt participants 8 times a day over 6 consecutive days with an auditory and visual signal (“beep”) to fill in a short momentary assessment. A high sampling frequency of 8-10 beeps per day was thought to provide sufficient insight into various daily contexts while not disrupting the flow of everyday life. The duration of 6 days was chosen to capture both weekdays and weekend days. This set-up was based on previous feasibility studies [23,24]. Beeps occurred unpredictably in semirandom time blocks of 112.5 minutes between 7:30 AM and 10:30 PM and were available to be filled in for 15 minutes after the beep. In total, 27 ESM items were included and could be answered on a 7-point Likert scale or in a multiple-choice set-up (Multimedia Appendix 2) assessing mood (eg, “I feel cheerful”), physical well-being (eg, “I feel tired”), subjective cognition (eg, “Since the last beep, I had memory problems”), and context (eg, “Where am I?”). Participants classified their responses individually, meaning that “work,” for example, could mean paid employment for one individual while another individual selected this option for gardening or doing chores. A morning and evening questionnaire was also part of the ESM, asking the participant to reflect on the previous night (eg, “I slept well”) and day (eg, “Generally, I felt tense today”). These questionnaires were not prompted via beeps but were available during the morning and evening, to be filled in self-reliantly, and this data was not included in this study (Multimedia Appendix 3). The development of this questionnaire was based on previous ESM studies [22,25,26]. Questions on subjective cognition were added after consulting with ESM experts and clinicians (ie, psychologists, psychiatrists, and neuropsychologists from the UMC).

Participants were approached via the Alzheimer Center Limburg research database, consisting of patients with cognitive impairments who had previously expressed interest in being contacted for research purposes and had been previously recruited through UMC or by their treating health care professional at the memory clinic. A member of the research team called potential participants, checked general eligibility, verbally explained the study, and sent out information sheets. Participants were called by phone 1 week later, and if willing to participate, a date for the orientation session was set. A standardized protocol was used: (1) an orientation session, (2) an ESM training session, (3) a 6-day ESM period, and (4) a debriefing session. Only the person with MCI participated in the ESM training, the ESM period, and the debriefing session, but both the person with MCI and their relevant other were present at the orientation session. Sessions took place either at the hospital or at the participant's home, depending on the participant's preference. Participants could drop out at any time without providing a reason.

Descriptive analyses were conducted to summarize the sociodemographic information and background questionnaire scores. The compliance rate of the ESM day questionnaires, responses to the debriefing questionnaires, and META scores of the human-technology interaction were also analyzed using descriptive statistics. For the usability demonstration, only participants who filled in at least 30% of the ESM assessments were included, as a sufficient amount of information needs to be available to describe daily patterns [37]. Thus, momentary ESM data from dropouts collected via the PsyMate were not included in this part of the analysis. The momentary data were demonstrated on a group level using mood, context, feelings of tiredness, and subjective cognition items, and analyzed with descriptive statistics to assess the usability in this population in general [12]. The variable positive affect (PA) consisted of the ESM items “I feel cheerful,” “I feel energetic,” “I feel relaxed,” “I feel satisfied,” and “I feel enthusiastic.” The negative affect (NA) included “I feel down,” “I feel insecure,” “I feel irritated,” “I feel lonely,” “I feel anxious,” and “I feel guilty.” To demonstrate elements of daily functioning, an activity-related stress (ARS) variable was generated using “I can do this well” (reversed), “This requires effort from me,” and “I would rather do something else.” The ESM data collected with the PsyMate has a multilevel structure with beeps (level 1) nested within participants (level 2). Average scores of PA, NA, and ARS were thus person-mean centered to take the within-person effect into account. Factor analyses were conducted to ensure sufficient internal validity (Cronbach α=.86, .84, and .68 for the PA, NA, and ARS, respectively). PA, NA, and ARS were average, on a person-mean level. On an exemplary level, the subjectively experienced cognitive problems of 3 participants were demonstrated with line graphs over the course of the ESM period. The 3 participants were selected without specific criteria but with the aim of showing variation, giving a general impression of the ESM data, and demonstrating how the data can be used in clinical practice prospectively. Daily functioning using ARS was also demonstrated on an individual level by using data from 3 participants exemplary. Stata statistical software (version 13.0; StataCorp) was used for statistical analyses, and Excel (version 16.16.19; Microsoft) was used to create graphic visualizations.

The data is stored at Maastricht University. Due to ethical and legal regulations, the data is only accessible for the MUMC+ research team. Sharing data with another research team needs to be approved by the Medical Research Ethics Committee azM/UM, or participants need to sign a new informed consent sheet.

A total of 152 people with MCI were approached to participate in this study; 21 people with MCI signed informed consent. The participant flow is illustrated in Figure 1. Their relevant others also agreed to participate; the relevant others of the study participants had a mean age of 63.3 (SD 8.9, range 47-78) years, 6 were men and 15 were women, and 19 were the partners of the participants while 1 was a sibling and 1 was a friend. Table 1 provides an overview of the characteristics of the total sample. Multimedia Appendix 4 shows the details of the study completers and dropouts.

In the study, 21 individuals started the ESM period, resulting in 673 beep records; 3 participants had problems using the ESM and did not complete the ESM period. These 3 dropouts had been eager to learn the app during the training session and their MMSE, other questionnaire scores, and general impression did not deviate outstandingly from the other participants. A statistical comparison between study completers and dropouts was not performed due to the small sample size.

The reasons for dropout were the following: Person A had problems using the right force pressing app buttons during the training session, forgot hearing aids repeatedly (according to partner) and thus did not react to the beeps, did not carry the smartphone along at all times, forgot the appointment, and seemed to generally deny cognitive problems. Person B expressed being very busy, only heard “some beeps” (no hearing problems, technical problems are unlikely according to IT specialist, reason unclear), and forgot the appointment for the debriefing session. Person C seemed generally nervous during the ESM training session (while expressing strong interest to participate), required very detailed and simple explanations of app use, and had forgotten instructions when contacted the following day. These 3 participants had not filled in the required 30% (16 beeps) to be included in the usability analysis, leading to a loss of 17 records (2.3%).

Of the 21 participants, 18 completed the ESM period and debriefing session, resulting in 656 valid beep records. On average, participants completed 38 beeps (SD 6.8; range 23-47) of the 48 beeps. The ESM compliance rate was 78.7%. Participants thought that the momentary questions reflected their experiences well (mean 4.83, SD 1.62) and that the PsyMate had little influence on their mood (mean 1.44, SD 1.15), activities (mean 1.61, SD 1.54), social interactions (mean 1.22, SD 0.73), or daily occupations (mean 1.39, SD 0.85). Filling in the momentary questions made participants marginally more aware of their activities (mean 2.17, SD 1.86) and moderately aware of their feelings (mean 3.56, SD 2.45) and memory (mean 4.56, SD 2.5). Of the 18 participants, 4 found increased awareness of their memory to be unpleasant, while 13 experienced it as pleasant or neutral. Table 2 provides detailed information on the general experience with the PsyMate and user-friendliness.

The META revealed that most performance steps involved in using the PsyMate did not cause any difficulties (Table 3). However, using the appropriate force, tempo, and precision caused, on average, some disturbances (mean 3.48, SD 0.51). With regard to the environmental characteristics influencing the use of the PsyMate app during the training session, the contextual influence (ie, the presence of researchers, which could be potentially stressful) was observed as not hindering smartphone use (mean 3.9, SD 0.3; range 3-4), while the technological design (ie, screen and button size) was observed as somewhat disturbing (mean 3.38, SD 0.3; range 2-4). The overall judgment of the participants' capacity to use the app was reflected in the capacity to recall necessary information as not disturbing (mean 3.86, SD 0.36; range 3-4), just like the capacity to pay attention and focus (mean 3.81, SD 0.40; range 3-4) and the capacity to manage stress (mean 3.76, SD 0.45; range 3-4). Of the 21 participants, most participants (12/21) had had smartphones for more than 10 years; 5 had used a smartphone for 3-9 years, 1 had used it for 1-2 years, and 2 had it for less than 1 year (1 participant could not indicate the duration). All 21 participants experienced the technology as very important and not replaceable; 18 used their smartphones daily and 2 used it at least weekly (for the remaining 1 participant, there is a missing value).

This study evaluates the feasibility and demonstrates the usability of a smartphone-based ESM in people with MCI. Several important findings emerged: (1) in study completers, the compliance rate was high and subjective ratings of the ESM procedure were positive; (2) the observable human-technology interaction between participants and the ESM app was generally unproblematic; (3) raising awareness for one’s own cognitive problems through ESM can be unpleasant for some individuals; and (4) cognitive issues (eg, forgetfulness) may lead to the inability to use the ESM.

Previous research found that the compliance rate, also referred to as adherence, use, or engagement, to technology-based self-monitoring such as ESM lies between 51-86% in middle-aged and older adults [9,38]. Therefore, the reported 78.7% of completed assessments in this study is a strong indication for the feasibility of ESM in a majority (18/21, 85%) of our MCI sample. A high sampling frequency, which was applied in this study with 8 beeps per day, is not thought to hinder ESM use, while the length of the questionnaires can increase burden [39]. The overall positive participants’ feedback on the procedure, including the length and frequency, supports the chosen ESM set-up. Additionally, the human-technology interactions were observed as overall unproblematic. Occasionally, a participant's inappropriate tempo, force, or precision of clicking on app buttons disturbed usage slightly. As older adults may benefit from large buttons and screens without scroll functions [40], it would be advisable to rotate the screen, increase button sizes, or provide a touch-pen to ease the app use even further. In addition to dexterity, older adults might also have hearing issues. In this study, one of the dropouts did not wear their hearing aids, which might have contributed to their inability to use the ESM and discontinuation of the trial.

According to the social cognition theory, self-monitoring can raise awareness for one’s own emotions or behaviors [41]. Repeated momentary assessments can use this increased awareness to promote behavioral changes towards healthy lifestyles [22,42] as well as improve mental well-being [43]. Within this study, there was no intention to change daily patterns, but nevertheless, participants became more aware of their memory abilities through repeated self-assessments. Of the 18 participants, 13 individuals experienced this as pleasant or neutral, but 4 reported this to be unpleasant. Similar side-effects of the ESM have previously been reported, and a suggestion could be to use positive formulations in the ESM questions [44]. For example, instead of asking about cognitive problems (eg, “Since the last beep, I had memory problems”), abilities could be targeted instead (eg, “I can remember well”). In this study, the experience of using ESM was discussed during the debriefing session, and in 1 case, a participant was advised to consult a health care professional for further treatment for cognition-related stress. In clinical settings, treating health care professionals may discuss experiences and increased awareness to develop coping strategies [45]. The individual profiles section highlights topics that may be discussed on an individual level, such as activities that elicit low stress (eg, relaxation) or the potential need for assistance or new coping strategies. Studies suggest that those feedback conversations could focus on positive emotions to increase resilience to stress [46] and stimulate goal-directed behavior [47]. This kind of feedback has shown to improve well-being, for instance, in an ESM-based intervention for carers of people with dementia [26]. The ultimate goal when using ESM is to support self-management through increase awareness for one's own abilities and orientate attention towards positive and meaningful aspects of daily life.

A small number of participants were unable to complete the experience sampling period. In older adults with undiagnosed subjective cognitive concerns, nonadherence to momentary assessments is thought to be greatly influenced by cognitive issues [48]. In this study, predicting dropouts using standardized instruments such as the MMSE was impossible. A systematic review reports that the averaged MMSE score in MCI samples seems to range from 23.1 to 28.7 [19], indicating a great variability of cognitive abilities in this population and that participants of this study potentially had relatively mild MCI. However, the MMSE has a limited discrimination between cognitively health adults and people with MCI, and other tests with a higher sensitivity (eg, Hopkins Verbal Learning Test) could have been used to determine study eligibility [49]. All participants were eager to participate, while no clear indication for exclusion could be identified. Follow-up phone calls were helpful to notice difficulties early. Dropouts seem to blame their inability to participate on the technology (eg, “It did not beep”). Admitting problems with technology might be easier than admitting other cognitive deficits, as even young and healthy individuals may occasionally face difficulties with technology. Further, reduced illness insight and cognitive deficits could have influenced the ability to use the ESM. The latter is supported by reports from participants and relevant others, stating that smartphones or hearing aids were forgotten, thus interfering with the ESM use. To prevent injustice in health care, all individuals with MCI motivated to use the ESM should be given a chance to do so, and frustration can be prevented through follow-ups, close guidance, and open communication.

Generally, the ESM group data revealed subjective problems with memory, concentration, and language in everyday life. This finding is in line with traditional neuropsychological assessments reporting a variety of cognitive deficits in MCI, of which memory is commonly most dominant [50]. A moderate level of fatigue has also been found in a healthy sample using the ESM [24] and may thus not be directly related to the cognitive deficits. To determine significant differences from healthy older adults, a control group is prospectively necessary. Furthermore, associations between daily fatigue, context, mood, and cognitive problems experienced by individuals may be studied using multilevel analysis [12].

On an individual level, cognitive fluctuations indicate trends of diversity both within and between subjects. The heterogeneity of the MCI group has been highlighted before [51], but this is one of the first studies to provide such a detailed insight into daily patterns using smartphone-based ESM. Next to the subjective evaluation of cognitive problems in everyday life, objective momentary cognition tasks can be added to this ESM app. The feasibility of 2 tasks has recently been tested in healthy individuals [52] and holds promise for future studies to describe a comprehensive picture of cognitive abilities. The ESM may also be useful to compare daily patterns of subjective or objective cognitive functioning in different neurological and psychiatric disorders.

Additionally, activity-related stress levels seem to vary between activities as working, for example, shows a trend for high levels of stress. This study is unfortunately not able to statistically explore activity patterns in people with MCI, but future research might follow up on this idea. Research shows that complex tasks are affected early on in the process of cognitive decline [53], and cognitive difficulties may decrease the ability of individuals with MCI to work [54]. Our understanding of necessary adjustments and ways to support working, particularly employment, when living with MCI are limited [55], but the insights gained from the participants in this study highlight the need to study working and employment as a potential stressor in this population further. High levels of anxiety and depression are commonly observed in MCI [56], as they were in this sample, and may also stand in relation to stress and cognitive deterioration [57]. As highlighted above, developing coping strategies and focusing on positive emotions might support daily well-being, including work-related stress. The ESM can be a useful tool to relate functional fluctuations with contexts and activities and thus understand patterns and networks in people with MCI, both on a within- and between-subject level [58].

Some critique regarding the ESM and study limitations need to be acknowledged. It is recommended not to overinterpret single items but rather to use momentary data as a starting point for a conversation about one's self-management and coping. Generally, many people (n=70) approached for the study had no smartphone or did not feel confident to participate in a smartphone-based study. This outcome indicates that there is a bias towards individuals with a higher technology familiarity to benefit from digital innovations in research and clinical work. Over the next decade, this bias might decrease, but researchers and clinicians need to be aware of this gap to not neglect individuals in need of support. Potentially, traditional paper-pencil diaries might be an alternative for people with MCI [16,17] that cannot or do not want to use smartphones. However, cognitive problems (eg, forgetting paper diary) or hearing problems (eg, not hearing the beeps from a prompting device) could still interfere. As learning and using a new technology is an intertwined process [59], and training is a key component for older adults to increase confidence and self-efficacy when using technology [60], prospective individuals with MCI who are not confident in their abilities to use a smartphone could receive training sessions and additional guidance. Unfortunately, this study is not able to determine if individuals with MCI would also be able to learn smartphone and ESM use together. Furthermore, the findings may be affected by a sex and education bias, as 76% (16/21) were men and only 10% (2/21) were low-educated. In addition, the etiology of MCI was not determined, resulting in an unspecified heterogeneity. As indicated by the MMSE, this MCI sample might have relatively mild cognitive problems, and a replication of our findings in a broader MCI sample might be necessary to increase the generalizability of the results. Detailed descriptive information about MCI subgroups could prospectively be added. The small sample size orientated on other feasibility studies [12,61] may limit the generalizability of the results, but the great number of assessments still result in a rich data set [22]. Finally, the study represents a specific group of people with MCI in possession of their own smartphones, and this recruitment criteria needs to be kept in mind when applying the ESM in future studies or clinical settings.

Technology-based ESM can be a useful addition to clinical questionnaires to reveal detailed moment-to-moment fluctuations, contextual patterns, and individual differences in subjectively experienced cognitive problems, affect, and activities. This feasibility study is a relevant step to better understand and support people with MCI in their everyday lives. Momentary data may prospectively be used to study individual and group-based patterns in this population and develop person-tailored self-management strategies.