This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Aging, is properly cited. The complete bibliographic information, a link to the original publication on http://aging.jmir.org, as well as this copyright and license information must be included.
Daily functioning of people with cognitive disorders such as mild cognitive impairment (MCI) is usually depicted by retrospective questionnaires, which can be memory-biased and neglect fluctuations over time or contexts.
This study examines the feasibility and usability of applying the experience sampling method (ESM) in people with MCI to provide a detailed and dynamic picture of behavioral, emotional, and cognitive patterns in everyday life.
For 6 consecutive days, 21 people with MCI used an ESM app on their smartphones. At 8 semi-random timepoints per day, participants filled in momentary questionnaires on mood, activities, social context, and subjective cognitive complaints. Feasibility was determined through self-reports and observable human-technology interactions. Usability was demonstrated on an individual and group level.
Of the 21 participants, 3 dropped out due to forgetting to carry their smartphones or forgetting the study instructions. In the remaining 18 individuals, the compliance rate was high, at 78.7%. Participants reported that momentary questions reflected their daily experiences well. Of the 18 participants, 13 (72%) experienced the increase in awareness of their own memory functions as pleasant or neutral.
Support was found for the general feasibility of smartphone-based experience sampling in people with MCI. However, many older adults with MCI are currently not in possession of smartphones, and study adherence seems challenging for a minority of individuals. Momentary data can increase the insights into daily patterns and may guide the person-tailored development of self-management strategies in clinical settings.
Clinical questionnaires are commonly retrospective in nature and are thus potentially affected by a memory bias and thought to have low ecological validity [
Momentary data collection, known as the experience sampling method (ESM) [
A recent review reported that technology-based self-monitoring such as the ESM is already applied in various populations, including people with depression, chronic pain, or other health issues, to study behaviors and promote health [
One group of individuals that might also benefit from this diary approach are people with mild cognitive impairment (MCI). By definition, MCI is not thought to impact daily functioning greatly [
To our knowledge, using the ESM in people with mild cognitive impairment (MCI) is rare. Daily or weekly paper-pencil diaries have been used to study momentary stressors and affect in MCI samples [
This study aims to determine the feasibility and usability of smartphone-based experience sampling in people with MCI. An ESM app was installed on participants' smartphones and programmed with a high sampling frequency to capture various intra-individual states (ie, mood, subjective cognitive problems) and situations (ie, activities, social context). Self-reports of using the ESM and observations of the direct human-technology interaction were conducted as part of the feasibility assessment. Human-technology interaction refers here to the person's ability to manage the ESM smartphone app, including specific performance skills, environmental characteristics, and individual capacities.
The usability of momentary data was studied on an individual and group level, focusing on subjective cognition, daily activities, and stress experienced in relation to those activities. Studying the data on a group level can provide valuable information on the daily functioning of the MCI population in general, while individual data can illustrate within-person fluctuations. This may result in helpful person-tailored insights that not only foster individualized therapy but also the diagnostic process [
Participants were recruited from the memory clinic at the Maastricht University Medical Center (UMC) from June 2018 to January 2020. Inclusion criteria were (1) having a clinical diagnosis of MCI, according to Albert et al [
The Medical Ethical Committee from the Maastricht academic hospital (azM) and Maastricht University approved the study (NL64310.068.17 / METC173055), and the protocol is registered on ToetsingOnline (64310). The authors comply with the Helsinki Declaration of 1975, as revised in 2008. All participants, including people with MCI and their relevant others, provided written informed consent before study participation.
The PsyMate smartphone app [
Participants were approached via the Alzheimer Center Limburg research database, consisting of patients with cognitive impairments who had previously expressed interest in being contacted for research purposes and had been previously recruited through UMC or by their treating health care professional at the memory clinic. A member of the research team called potential participants, checked general eligibility, verbally explained the study, and sent out information sheets. Participants were called by phone 1 week later, and if willing to participate, a date for the orientation session was set. A standardized protocol was used: (1) an orientation session, (2) an ESM training session, (3) a 6-day ESM period, and (4) a debriefing session. Only the person with MCI participated in the ESM training, the ESM period, and the debriefing session, but both the person with MCI and their relevant other were present at the orientation session. Sessions took place either at the hospital or at the participant's home, depending on the participant's preference. Participants could drop out at any time without providing a reason.
After the study procedure was explained once more and final questions were clarified, informed consent was signed by the person with MCI and their relevant other. Next, sociodemographic information was collected and questionnaires were filled in assessing characteristics of the person with MCI either with self- or proxy-reports. At the end of this session, a date for the ESM training session was set. The ESM training was not combined with the orientation session so as not to overburden participants (as filling in a range of questionnaires can potentially be intense, confronting, and tiring). Thereby, we hoped to prevent participants from forgetting the ESM-training instructions due to information overload.
During the 30-minute training session, the PsyMate app was installed on the participant's smartphone, and the participant was instructed on how to respond to beeps, operate the app, and interpret the momentary questions. An example ESM questionnaire was filled in to familiarize participants with the procedure. The management of the app was observed, guided by the Management of Everyday Technology Assessment (META), to get a detailed picture of the human-technology interaction and performance skills [
The PsyMate started sending beeps from the moment of installation; participants could respond on this day to train for filling in the momentary assessments, but they were instructed that the official 6-day ESM period would start the following day. On the second ESM day, a researcher called to check-in and solve potential technical problems or provide clarification.
This session took place 1 day after the last day of the ESM period. Participants were asked to report their general experiences using the app, and they received travel reimbursements and a small gift after participation but no financial reward.
Next to the sociodemographic information of the person with MCI (age, sex, education, living situation, years since first symptoms) and their relevant other (age, sex, relationship to person with MCI), reliable and valid instruments were filled in with the purpose of describing the sample. The Mini–Mental State Examination (MMSE) provided information on cognitive functioning [
The feasibility was determined through the compliance rate of the ESM assessments and was regarded as satisfactory when >70% of the momentary questionnaires were filled in [
The Management of Everyday Technology Assessment (META) [
Descriptive analyses were conducted to summarize the sociodemographic information and background questionnaire scores. The compliance rate of the ESM day questionnaires, responses to the debriefing questionnaires, and META scores of the human-technology interaction were also analyzed using descriptive statistics. For the usability demonstration, only participants who filled in at least 30% of the ESM assessments were included, as a sufficient amount of information needs to be available to describe daily patterns [
The data is stored at Maastricht University. Due to ethical and legal regulations, the data is only accessible for the MUMC+ research team. Sharing data with another research team needs to be approved by the Medical Research Ethics Committee azM/UM, or participants need to sign a new informed consent sheet.
A total of 152 people with MCI were approached to participate in this study; 21 people with MCI signed informed consent. The participant flow is illustrated in
Flow chart of recruited participants with mild cognitive impairment (MCI). ESM: experience sampling method.
Descriptive information about the participants with mild cognitive impairment (n=21).
Participant characteristics | Values | |
Age in years, mean (SD; range) | 66 (7.1; 48-79) | |
|
||
|
Men | 16 (76) |
|
Women | 5 (24) |
|
||
|
Low (<9 years) | 2 (10) |
|
Middle (9-10 years) | 11 (52) |
|
High (>10 years) | 8 (38) |
|
||
|
Retired | 14 (67) |
|
Working | 3 (14) |
|
Unemployed | 4 (19) |
|
||
|
With partner | 17 (81) |
|
With partner and children | 1 (5) |
|
Alone | 3 (14) |
Years since first symptoms, mean (SD; range) | 4.8 (4.0; 1-19) | |
Cognition (MMSEa), mean (SD; range) | 28 (1.26; 27-30) | |
|
3.4 (0.67; 2-4) | |
|
4: Intact, n (%) | 10 (48) |
|
3: Mildly disturbed, n (%) | 9 (43) |
|
2: Moderately disturbed, n (%) | 2 (10) |
|
1: Absent, n (%) | — |
Anxiety (HADS-Ac), mean (SD; range) | 11.8 (2.2; 6-15) | |
Depression (HADS-Dd), mean (SD; range) | 9.6 (1.4; 7-12) | |
Perceived stress (PSSe), mean (SD; range) | 19.1 (4.5; 9-28) | |
Neuropsychiatric symptoms (NPI-Qf), mean (SD; range) | 2.7 (2.1; 0-7) | |
Instrumental activities of daily living (IADLg), mean (SD; range) | 57.2 (7.3; 45.9-69.9) |
aMMSE: Mini–Mental State Evaluation; MMSE score range: 0-30, with higher scores indicating less cognitive difficulties.
bGRAD: Guidelines for the Rating of Awareness Deficits.
cHADS-A: Hospital and Anxiety Depression Scale–Anxiety; HADS scores range: 0-21 per scale (<7 noncases, 8-10 doubtful-cases, >11 definitive cases).
dHADS-D: Hospital and Anxiety Depression Scale–Depression; HADS scores range: 0-21 per scale (<7 noncases, 8-10 doubtful-cases, >11 definitive cases).
ePSS: Perceived Stress Scale; PSS scores range: 0-40, with higher scores indicating higher stress levels.
fNPI-Q: Neuropsychiatric Inventory Questionnaire; NPI-Q scores range: 0-36, with higher scores indicating a greater amount of neuropsychiatric behavior in the past month.
gIADL: Instrumental Activities of Daily Living; IADL
In the study, 21 individuals started the ESM period, resulting in 673 beep records; 3 participants had problems using the ESM and did not complete the ESM period. These 3 dropouts had been eager to learn the app during the training session and their MMSE, other questionnaire scores, and general impression did not deviate outstandingly from the other participants. A statistical comparison between study completers and dropouts was not performed due to the small sample size.
The reasons for dropout were the following: Person A had problems using the right force pressing app buttons during the training session, forgot hearing aids repeatedly (according to partner) and thus did not react to the beeps, did not carry the smartphone along at all times, forgot the appointment, and seemed to generally deny cognitive problems. Person B expressed being very busy, only heard “some beeps” (no hearing problems, technical problems are unlikely according to IT specialist, reason unclear), and forgot the appointment for the debriefing session. Person C seemed generally nervous during the ESM training session (while expressing strong interest to participate), required very detailed and simple explanations of app use, and had forgotten instructions when contacted the following day. These 3 participants had not filled in the required 30% (16 beeps) to be included in the usability analysis, leading to a loss of 17 records (2.3%).
Of the 21 participants, 18 completed the ESM period and debriefing session, resulting in 656 valid beep records. On average, participants completed 38 beeps (SD 6.8; range 23-47) of the 48 beeps. The ESM compliance rate was 78.7%. Participants thought that the momentary questions reflected their experiences well (mean 4.83, SD 1.62) and that the PsyMate had little influence on their mood (mean 1.44, SD 1.15), activities (mean 1.61, SD 1.54), social interactions (mean 1.22, SD 0.73), or daily occupations (mean 1.39, SD 0.85). Filling in the momentary questions made participants marginally more aware of their activities (mean 2.17, SD 1.86) and moderately aware of their feelings (mean 3.56, SD 2.45) and memory (mean 4.56, SD 2.5). Of the 18 participants, 4 found increased awareness of their memory to be unpleasant, while 13 experienced it as pleasant or neutral.
General PsyMate app and user-friendliness evaluation [n=18; dropouts (n=3) were not included because they did not participate in the debriefing session].
General PsyMate app and user-friendliness evaluation | Scores (1=“not at all” – 7=“very much”) | |||
|
||||
|
Was this a normal week? | 5.06 (1.51; 2-7) | ||
|
Did special events occur? | 2.22 (1.73; 1-4) | ||
|
Did the questions reflect your experiences well? | 4.83 (1.62; 2-7) | ||
|
Did the PsyMate app influence your mood? | 1.44 (1.15; 1-5) | ||
|
Did the PsyMate app influence your activities? | 1.61 (1.54; 1-7) | ||
|
Did the PsyMate app influence your social interactions? | 1.22 (0.73; 1-4) | ||
|
Did the PsyMate app hinder your daily occupations? | 1.39 (0.85; 1-4) | ||
|
Did you make mistakes when filling in the PsyMate app? | 2.17 (0.92; 1-4) | ||
|
|
3.56 (2.45; 1-7) | ||
|
|
If so, did you experience this as pleasant? na | 7 |
|
|
|
If so, did you experience this as neutral? na | 9 | |
|
|
If so, did you experience this as unpleasant? na | 1 | |
|
|
4.56 (2.50; 1-7) |
||
|
|
If so, did you experience this as pleasant? na,b | 6 | |
|
|
If so, did you experience this as neutral? na,b | 7 | |
|
|
If so, did you experience this as unpleasant?a,b | 4 | |
|
|
2.17 (1.86; 1-7) |
||
|
|
If so, did you experience this as pleasant? na,b | 3 | |
|
|
If so, did you experience this as neutral? na,b | 14 | |
|
|
If so, did you experience this as unpleasant? na,b | 0 | |
|
||||
|
Were you able to read the text on the screen well? | 6.06 (1.70; 1-7) | ||
|
Could you hear the beep well? | 6.44 (0.86; 4-7) | ||
|
Did you have problems using the PsyMate app? | 1.56 (1.65; 1-5) | ||
|
Were the verbal explanations regarding the PsyMate app clear? | 6.67 (0.60; 5-7) | ||
|
Were the written explanations regarding the PsyMate app clear? | 6.67 (0.60; 5-7) | ||
|
Were the questions from the PsyMate app unclear or difficult? | 2.28 (1.60; 1-7) | ||
|
Did you experience the use of the PsyMate app burdensome with regard to the number of beeps? | 1.44 (0.98; 1-5) | ||
|
Did you experience the use of the PsyMate app burdensome with regard to length of one beep? | 1.44 (0.62; 1-3) | ||
|
Did you experience the use of the PsyMate app burdensome with regard to the sound? | 2.33 ± 2.14 (1-7) | ||
|
Did technical problems hinder you from filling in the beeps?b | 1.88 1.09 (1-4) |
aQuestions were not answered on a 7-point Likert scale but categorically.
bMissing response (n=1).
The META revealed that most performance steps involved in using the PsyMate did not cause any difficulties (
Assessment of observable performance skills when using the PsyMate app during the experience sampling method (ESM) training session (n=18).
Performance skill | Observation scorea, mean (SD; range) |
Identify service and functionb | 3.90 (0.31; 3-4) |
Perform actions in logical sequence | 3.95 (0.22; 3-4) |
Manage series of number/lettersc | 4.0 |
Choose correct button or command | 3.76 (0.45; 3-4) |
Use appropriate force, tempo, and precision | 3.48 (0.51; 3-4) |
Identify information and respond adequately | 3.95 (0.22; 3-4) |
aObservation scores: range 1-4; 4=competent handling/management (ie, no deficits in this skill disturbs or hinders the person's use of the technology); 1=deficits in this skill hinder the person's use of the technology and/or the person is in need of assistance to perform the skill competently.
bn=1 missing, as skill was not observable.
cn=12 missing, as skill not observable.
In general, the 18 participants experienced a high level of PA (mean 4.95, SD 0.66; range 3.94-6.13), a low level of NA (mean 1.95, SD 0.93; range 1.07-3.92) and a low to moderate level of ARS (mean 2.73, SD 0.74; range 1.71-4.05). They felt moderately tired (mean 3.64, SD 1.39; range 1-6.29) and had low to moderate problems with their memory (mean 3.01, SD 1.11; range 1.34-5.29), language (mean 2.04, SD 1.15; range 1-5.21), and concentration (mean 2.85, SD 1.36; range 1.05-4.96). With regard to their contextual patterns, participants spent most of their time at home (72%) (other locations: transport [9%]; at family's/friend's [5%]; at work [5%]; somewhere else [5%]; public place [4%]) engaging in household (22%) or relaxing activities (29%) (other activities: eating/drinking [10%]; something else [10%]; work [8%]; nothing [6%]; traveling [4%]; in conversation [4%]; self-care [4%]; sport/physical activity [3%]), and were often in company of their partner (45%) or alone (31%) (other social company: family [8%]; colleagues [4%]; co-occupants [4%]; friends [3%]; acquaintances [3%]; strangers [2%]).
To illustrate the variability that can be studied using momentary data, several descriptive examples are presented, focusing on the subjectively experienced cognitive problems, daily activities, and activity-related stress in everyday life. These participants were selected without specific criteria but with the aim of visually illustrating fluctuations within subjects, variables, and days (
As shown in
Person 1: subjectively experienced cognitive problems.
Person 1: daily activities and levels of activity-related stress.
Daily activities | Daily time (%) | Activity-related stress levela |
Nothing | 24 | 3.14 |
Relaxing | 7 | 1.78 |
In conversation | 3 | 2 |
Something else | 50 | 2 |
Eating/drinking | 3 | 2 |
Work | 10 | 3.67 |
Travel | 3 | 2.07 |
aScale: 1 (not at all) – 7 (very much). This data stands in relation to the fluctuations of cognition (
As shown in
Person 2: subjectively experienced cognitive problems.
Person 2: daily activities and levels of activity-related stress.
Daily activities | Daily time (%) | Activity-related stress levela |
Household | 32 | 2.89 |
Relaxing | 39 | 2.69 |
In conversation | 2 | 3.33 |
Sports/physical activity | 5 | 2.33 |
Eating/drinking | 9 | 2.33 |
Work | 2 | 2.67 |
Self-care | 9 | 3.08 |
Traveling | 2 | 2 |
aScale: 1 (not at all) – 7 (very much). This data stands in relation to the fluctuations of cognition (
As shown in
Patient 3: subjectively experienced cognitive problems.
Person 3: daily activities and levels of activity-related stress.
Daily activities | Daily time (%) | Activity-related stress levela |
Household | 5 | 4.17 |
Relaxing | 48 | 3.3 |
Something else | 14 | 3.56 |
Sports/physical activity | 5 | 3.5 |
Eating/drinking | 2 | 4 |
Work | 7 | 4.67 |
Self-care | 7 | 3 |
Nothing | 12 | 5.33 |
aScale: 1 (not at all) – 7 (very much). This data stands in relation to the fluctuations of cognition (
This study evaluates the feasibility and demonstrates the usability of a smartphone-based ESM in people with MCI. Several important findings emerged: (1) in study completers, the compliance rate was high and subjective ratings of the ESM procedure were positive; (2) the observable human-technology interaction between participants and the ESM app was generally unproblematic; (3) raising awareness for one’s own cognitive problems through ESM can be unpleasant for some individuals; and (4) cognitive issues (eg, forgetfulness) may lead to the inability to use the ESM.
Previous research found that the compliance rate, also referred to as adherence, use, or engagement, to technology-based self-monitoring such as ESM lies between 51-86% in middle-aged and older adults [
According to the social cognition theory, self-monitoring can raise awareness for one’s own emotions or behaviors [
A small number of participants were unable to complete the experience sampling period. In older adults with undiagnosed subjective cognitive concerns, nonadherence to momentary assessments is thought to be greatly influenced by cognitive issues [
Generally, the ESM group data revealed subjective problems with memory, concentration, and language in everyday life. This finding is in line with traditional neuropsychological assessments reporting a variety of cognitive deficits in MCI, of which memory is commonly most dominant [
On an individual level, cognitive fluctuations indicate trends of diversity both within and between subjects. The heterogeneity of the MCI group has been highlighted before [
Additionally, activity-related stress levels seem to vary between activities as working, for example, shows a trend for high levels of stress. This study is unfortunately not able to statistically explore activity patterns in people with MCI, but future research might follow up on this idea. Research shows that complex tasks are affected early on in the process of cognitive decline [
Some critique regarding the ESM and study limitations need to be acknowledged. It is recommended not to overinterpret single items but rather to use momentary data as a starting point for a conversation about one's self-management and coping. Generally, many people (n=70) approached for the study had no smartphone or did not feel confident to participate in a smartphone-based study. This outcome indicates that there is a bias towards individuals with a higher technology familiarity to benefit from digital innovations in research and clinical work. Over the next decade, this bias might decrease, but researchers and clinicians need to be aware of this gap to not neglect individuals in need of support. Potentially, traditional paper-pencil diaries might be an alternative for people with MCI [
Technology-based ESM can be a useful addition to clinical questionnaires to reveal detailed moment-to-moment fluctuations, contextual patterns, and individual differences in subjectively experienced cognitive problems, affect, and activities. This feasibility study is a relevant step to better understand and support people with MCI in their everyday lives. Momentary data may prospectively be used to study individual and group-based patterns in this population and develop person-tailored self-management strategies.
Interface of the experience sampling method (ESM) smartphone app "PsyMate.".
Experience sampling method (ESM) item list of the day questionnaire.
Morning and evening questionnaires.
Descriptive information of the study completers (n=18) and dropouts (n=3).
Amsterdam instrumental activities of daily living
activity-related stress
academisch ziekenhuis Maastricht
experience sampling method
Guidelines for the Rating of Awareness Deficits
Hospital and Anxiety Depression Scale
mild cognitive impairment
Management of Everyday Technology Assessment
Mini–Mental State Examination
negative affect
Neuropsychiatric Inventory Questionnaire
positive affect
perceived stress scale
Maastricht University Medical Center
The authors thank all participants that took part in this study, as well as Roos Roberts, Astrid Quist, and Karel Borkelmans for their contribution to data collection. This research was carried out as part of the H2020 Marie Skłodowska-Curie Actions Innovative Training Network (ITN) action, H2020-MSCA-ITN-2015, under grant agreement number 676265.
None declared.