Between April and December 2024, we approached all consecutive participants attending the Healthy Brain Ageing clinic at the Brain and Mind Centre, University of Sydney, for participation. This clinic is a specialist memory and cognition clinic for older adults (aged 50 y or older) with new-onset cognitive or mood concerns. As previously described [18], the exclusion criteria for the clinic were: insufficient English to complete neuropsychological assessment, Mini-Mental State Examination (MMSE) score <20 [19], current substance dependence, intellectual disability, presence of nonaffective psychiatric disorder (eg, attention-deficit/hyperactivity disorder and schizophrenia), presence of neurological disease (eg, Parkinson and epilepsy), history of head injury with loss of consciousness >30 minutes, and history of stroke. Additional exclusion criteria for this study were the absence of a mobile phone and a diagnosis of dementia. Four participants were excluded for not having a mobile phone.

The Lifestyle for Brain Health (LIBRA) index is a composite measure of 12 modifiable dementia risk factors, developed through a systematic review and expert consensus using the Delphi method [42]. The LIBRA 2 index is an updated version that incorporates additional factors, including hearing impairment, social contact, and sleep disturbances [42]. In this study, we used the validated LIBRA2 index to calculate a composite risk score by summing the available factor scores for each participant, with a theoretical range of 0 to 100. Positive weights were assigned when risk factors were present or protective factors were absent [42] (Table 1), with higher scores indicating greater dementia risk. In our sample, data were available for 13 of the 15 LIBRA2 factors, including high physical activity, sleep disturbances, high alcohol consumption, hearing impairment, kidney disease, coronary heart disease, diabetes, high cholesterol, smoking, hypertension, obesity, depression, and high cognitive activity. Data for healthy diet and low social contact were not available. Following previously published methodology [43], when less than 4 factors were missing, scores were calculated using the available factors. Four participants had 4 or more factors missing; therefore, these participants were excluded from LIBRA 2 analyses.

We categorized participants into several groups. First, those who agreed versus those who declined to use the app. Second, those who completed all recall trials (full completers) versus those who did not complete 1 or both of the following: evening recall test and morning delayed recall test (partial completers). For additional exploratory data analysis, the median score for SDMC % retention was 92.40% (IQR 84.31-100.00). From this, we categorized participants as either low (n=18, <92.40) or high (n=18, ≥92.40) SDMC performers.

This study was approved by the University of Sydney Human Ethics Committee (2019/HE000271) and conducted in accordance with the World Medical Association Declaration of Helsinki. All participants provided informed consent before participating and were given the option to withdraw from the study at any time. The collected data were deidentified to ensure participant privacy and confidentiality. No compensation was provided to participants.

We conducted both quantitative data analysis and summarized user feedback on the Sleep Memories app. Quantitative data analysis was conducted using IBM SPSS Statistics (version 29) [44]. All continuous variables were inspected for normality using visual inspection (histograms), while the assumption of homogeneity of variance for these data was checked using the Levene test for equality of variances. For group comparisons, independent samples t tests were used for continuous variables that met assumptions of normality and homogeneity of variance. When the assumption of equal variances was violated, the Welch t test was applied. For nonnormally distributed variables, the nonparametric Mann-Whitney U test was used. Categorical variables were compared using chi-square tests. Prior to examining associations between SDMC performance and other variables (demographics, clinical factors, sleep behaviors, modifiable dementia risk factors, and global cognitive functioning), linear relationships were assessed visually using scatterplots. Following these assessments, the relationships between variables were analyzed using Pearson r for normally distributed variables and Spearman ρ for nonnormally distributed variables. We then compared the demographics, clinical factors, and subjective sleep-related factors between high and low SDMC performers as well as between participants who provided user feedback and those in the main sample.

Next, we summarized user feedback on the app. This analysis was based on open-ended questions at the end of each questionnaire section (see Multimedia Appendix 1), allowing participants to describe their experiences, usability, and challenges. The Likert-scale items were used only to guide discussion and provide context; no formal statistical analysis was conducted due to the small sample size (n=22), and participants’ responses were optional. We reported only the descriptive summary of selected items. The data analysis was conducted in an unbiased manner, with independent coding and theme identification and development carried out by 2 researchers (CCYT and SS). Following this, a meeting was held to compare and discuss their interpretations. Any discrepancies in coding or pattern identification and development were resolved through discussion, with a third researcher, AL, facilitating consensus when needed. To ensure impartiality, CCYT and SS were blinded to participants’ age, clinical diagnosis, and other personal information. Researchers identified patterns first; only then did they examine participants’ age, clinical diagnosis, and task completion to assess specific difficulties completing the tasks. The summarization of user feedback was conducted based on a four-step procedure: (1) becoming familiar with the data, (2) generating initial codes, (3) clearly defining and naming them, and finally, (4) producing writing that presents a coherent narrative supported by data extracts [45].

A total of 141 consecutive clinic attendees during the data collection period were invited to participate. Of these, 54% (n=76) agreed to participate. Of those who agreed, 89% (n=68) attempted the word-pairs task using the app. A subset (n=22) was contacted for postuse feedback (see Figure 3, which details the recruitment process, including numbers who agreed or declined, task completion, and participation in interviews). Additionally, 11 participants provided explicit reasons for nonparticipation during their clinic visit, including task too lengthy (n=1), lack of time (n=2), feeling overwhelmed by the task (n=2), difficulty downloading the app (n=2), struggling with technology (n=1), inability of the phone to scan the QR code (n=1), perceived lack of memory benefits (n=1), and forgetting to bring their phone (n=1).

Of the 68 participants who attempted to use the Sleep Memories app to complete the word-pairs task, 68% (n=46) completed the evening recall test, 69% (n=47) completed the morning delayed recall test, and 57% (n=39) completed both tests. Out of the 46 participants who completed the evening recall test, 15% (n=7) completed only the evening recall test and not the morning delayed recall test. Similarly, of the 47 participants who completed the morning delayed recall test, 17% (n=8) completed only the morning delayed recall test and not the evening recall test.

The feasibility of the Sleep Memories app was assessed by examining the completion rate of the tasks: evening learning trials 1 to 4 (58/68, 85%; 57/68, 84%; 60/68, 88%; 60/68, 88%; respectively), evening recall test (46/48, 68%), morning delayed recall test (47/68, 69%), morning multiple-choice test (45/68, 66%), and both evening and morning recall tests (39/68, 57%)—all tasks had completion rates above 50% (Table 2). As described in the Data Analysis section, after adjusting for outliers, we computed the mean SDMC score of 92.76 (SD 13.85). In addition, the chi-square test showed no significant differences in completion rates between SCD and MCI groups (Table 3).

Among those who completed both evening recall and morning delayed recall tests, on average, the interval between evening encoding and sleep onset was 98.49 (SD 56.55) minutes. In the morning, on average, the interval between wake-up time and morning delayed recall was 33.61 (SD 37.85) minutes.

First, we present a brief descriptive summary of selected Likert-scale items from the Sleep Memories App Feedback Questionnaire (see Multimedia Appendix 1). Overall, 82% (18/22) of participants agreed that the app instructions were easy to understand and that the task was straightforward. A majority (20/22, 91%) indicated a preference for completing the task using the app rather than in a sleep laboratory. Additionally, 86% (19/22) reported an overall positive experience with the app. Notably, none of the participants reported needing help from their family to complete the SDMC task, while only 14% (3/22) reported needing family assistance to navigate the app.

Next, qualitative feedback from participants described age-related physical and cognitive changes that influenced their experience with the app. More specifically, 2 participants commented that the app appeared more user-friendly for individuals who are younger or more computer-literate, rather than for older users. One participant noted that being from a generation familiar with computers, but not mobile phones, made completing the task on an unfamiliar device challenging. Another suggested that improvements could be made to better accommodate individuals with vision or dexterity impairments.

Participants generally provided positive feedback, describing the app as easy to use, user-friendly, and well-designed, with some finding it fun, interesting, and convenient to complete tasks at home. One participant even shared their positive experiences with others, and another described it as one of the best apps they had ever used. Less positive feedback included feelings of nervousness or stress during testing, concerns about performance, and perceptions that the app was less suitable or engaging for older users. Given that the app is designed for cognitive testing, some level of anxiety is expected.

In addition, participants reported several technical challenges related to the functionality of the Sleep Memories app and the clarity of the accompanying instructions. Participants also raised concerns regarding the app’s accessibility and suitability for older adults and individuals with additional needs. The most frequently mentioned issue was tasks being delivered too quickly, making it difficult to keep up. Nevertheless, they appreciated the voice input feature, making task completion easier (see Table 6 for more details).

There were no differences in SDMC across gender (t₃₄=−0.65, P=.52, Hedges g=0.23) and clinical diagnostic groups (t₃₄=0.55, P=.59, Hedges g=0.18). We also examined the associations between SDMC performance and several variables: demographics, clinical factors, sleep behaviors, modifiable dementia risk factors, and global cognitive functioning. We first examined these relationships using all available SDMC data points. As shown in Table 7, there was a significant and positive correlation between premorbid IQ (as measured by Wechsler Test of Adult Reading) and SDMC. Nevertheless, there were no statistically significant correlations between SDMC and other demographics, clinical factors, sleep behaviors, and modifiable dementia risk factors. Since this was an exploratory study with a small sample size, we did not correct for multiple comparisons.

Next, we conducted a sensitivity analysis to assess whether potential outliers on the evening recall task might influence the results. We excluded 3 evening recall data points that were SD 1.50 below the mean. These data points were excluded for memory consolidation-specific analyses (SDMC% retention) but retained for all other analyses. The SD 1.50 cutoff was selected as a pragmatic adaptation of common outlier detection rule [46]. Results with this exclusion are presented in Table 8, and findings were consistent both with and without the exclusion.

In terms of comparing the characteristics between low and high SDMC performers (Table 9), low performers had significantly less education and lower anxiety levels compared to high performers. There were no other differences between these 2 groups.

When comparing the characteristics of participants who provided user feedback (subsample) with those of the main sample, the subsample included a higher proportion of individuals with SCD than the main sample. Nevertheless, there was no difference between the groups in the proportion of full and partial completers. There were also no other differences between these 2 groups (Table 10).

We also reported task completion rates among participants who reported difficulty in using the app (Table 11).

This study aimed to evaluate the willingness to engage with and the feasibility of using the Sleep Memories app to assess SDMC in older adults with cognitive concerns and impairments within a brain health and memory clinic setting. It is important to note that this study focused on the feasibility of a digital delivery platform for the SDMC task rather than providing objective physiological sleep metrics, such as those obtained through actigraphy or polysomnography.

Overall, the app was deemed feasible (>50% completion) as determined a priori. Notably, our study involved greater logistical demands than prior app-based cognitive studies. Sleep-dependent memory protocols require tighter scheduling around sleep periods and may involve overnight laboratory stays in unfamiliar environments, which are not always feasible for older adults with cognitive impairment. Therefore, the 57% completion rate, while modest, indicates practical feasibility for real-world implementation. Willingness to participate was higher among those with SCD than MCI, those with higher global cognitive scores, and those with less difficulty waking up in the morning. Additionally, task completion was higher among those with more years of education and higher anxiety levels.

Interestingly, while similar completion rates in the SCD and MCI groups support the app’s feasibility, the higher initial refusal rate among individuals with MCI and lower cognitive scores suggests a barrier to participation. This aligns with prior work, which showed that, relative to those with more severe cognitive impairments, individuals with SCD are often motivated to participate in research studies due to concern about cognitive decline, despite normal objective performance [47]. Future research could adopt a dyadic approach by involving a care partner to assist with technical issues at home, or potentially offering on-call technical support, helping reduce barriers for those who might otherwise decline participation.

In this study, we recorded reasons for nonparticipation from a subset of participants, which suggested that nonparticipation was influenced by a combination of perceived task burden (eg, participant perceived the word-pairs task as lengthy), technical barriers (eg, phone unable to download the app), and motivational factors (eg, participant perceived the task as lacking memory benefit). These findings are in accordance with the Unified Theory of Acceptance and Use of Technology [48], where “effort expectancy” and “facilitating conditions” correspond to the perceived task burden and technical barriers observed in this study, and “performance expectancy” reflects participants’ motivation to engage with the app based on perceived cognitive benefits. Moreover, consistent with past research linking higher global cognitive functioning to greater health literacy [49], our results revealed that individuals agreeing to participate had more superior global cognition compared to those with cognitive decrements. This may be suggesting that participants with higher global cognitive functioning might be more motivated to engage in research perceived as supporting sleep and cognitive health. Difficulty with sleep also appears to affect willingness to participate in sleep-dependent memory tasks. Waking difficulties often linked to disrupted sleep or fatigue. This may reduce motivation or capacity to engage in research activities in the morning, especially those requiring structured participation such as completing tasks via a mobile app with strict timings (ie, 3 h before sleep and upon waking). Analyses of other sleep-related apps reveal similar issues. For instance, “SleepFit,” which aims to enhance sleep hygiene among paramedics [50], had a substantial dropout rate of 91.40%. The authors attributed this low engagement and retention to fatigue and disrupted sleep, particularly making it difficult for users to adhere to the app’s scheduled tasks.

The Sleep Memories app demonstrated completion rates comparable to other remote cognitive assessments in older populations. For example, a study of older adults at risk of cognitive decline and dementia (mean age 57.3, SD 5.3 y), using the smartphone-based iVitality app (featuring tests adapted from conventional neuropsychological assessments) reported completion rates of approximately 60% [36]. Similarly, an unsupervised app−based study in healthy older adults, assessing memory recall and object and scene discrimination, reported completion rates ranging from 54% to 69% [35]. Notably, the completion rate observed in this pilot study exceeded that reported for the Linus Health Core Cognitive Evaluation app in older adults (mean age 74.0, SD 6.0 y), which achieved a substantially lower rate of around 10% [37]. In this study, we found a strong initial task completion rate, particularly during the learning phase. However, the task completion rate declined during the evening recall and morning recall tests, suggesting that the completion of delayed recall tasks administered either 30 minutes later (evening recall task) or the following morning (morning delayed recall and multiple-choice tasks) may pose greater challenges in remote, unsupervised contexts. Re-engagement with the app after a delay may be influenced by contextual factors such as interruptions, competing demands, or diminished motivation to return to the task [51,52]. Notably, completion rates for all learning trials, evening, and morning tasks did not differ significantly between participants with SCD and MCI. This finding is consistent with a past study that similarly showed no significant differences in online cognitive test completion based on cognitive status [53]. Additionally, based on user feedback provided, aging—especially physical and cognitive changes—affects task completion. For instance, for some, a decline in fine motor skills appeared to hinder typing accurately on a mobile screen, especially on smaller devices. Slowed reaction times and decreased hand-eye coordination further contributed to difficulties with tasks requiring precise tapping. Additionally, reduced visual acuity common in later life [54] might also impact the ability to read small on-screen instructions and navigate through mobile apps. These sensory and motor limitations are often compounded by a general age-related unfamiliarity with digital interfaces, resulting in reduced confidence in using mobile apps. Importantly, our results showed no differences in age between full completers and partial completers, suggesting that it is not chronological age itself but the effects of aging such as changes in motor, visual, and cognitive abilities that influence task completion. Other than physical and cognitive changes due to aging, education and psychological factors also play a role in affecting the completion of app-based SDMC tasks, as we found full completers tended to have more years of education and higher anxiety levels compared to partial completers. Education fosters cognitive discipline and familiarity with structured tasks [55,56], potentially making these individuals more likely to persist through challenges. Meanwhile, elevated anxiety (when managed effectively) can enhance task vigilance, increase motivation to avoid failure, and drive individuals to complete tasks with greater urgency [57]. A similar pattern was observed when we compared the characteristics of low and high performers of SDMC tasks—high performers had more years of education and higher anxiety levels than low performers. Together, these findings suggest that education equips individuals with strategies to manage tasks, and that anxiety can provide motivation for completion and performance in SDMC tasks. When examining user feedback from full completer and partial completer groups, full completers generally reported more positive subjective experiences, noting that the app was easy to use, enjoyable, and fun. Two participants from the full completer group mentioned feeling nervous during the task, possibly due to the evaluative or confrontational nature of cognitive testing and expressed concerns about their task performance. Partial completers reported similar experiences, indicating little difference between full completers and partial completers in terms of subjective experiences. In terms of technical challenges, some full completers reported that the instructional sheet was unclear and that the task progressed too quickly. It is important to note that substantially slowing down task delivery may risk introducing ceiling effects, making the task overly easy and unnecessarily lengthy. In contrast, partial completers provided little feedback regarding the app or technical difficulties. This lack of feedback may reflect lower motivation, earlier disengagement before issues arose, or a simple disinclination to comment. One participant in the partial completer group (diagnosed with MCI) reported needing assistance from family members to navigate the app. Although this was an isolated case, and the task had originally been designed and validated for people with MCI within the sleep laboratory [5], this observation suggests that caregiver support may be necessary to facilitate engagement with the Sleep Memories app among some users with greater cognitive impairment.

Having considered factors influencing willingness to participate and the feasibility of the app based on task completion, it is also important to understand those affecting SDMC performance. Notably, cognitive reserve affects SDMC performance. We observed a positive correlation between premorbid IQ and SDMC performance, which aligns with past research showing that cognitive reserve is linked to better cognitive outcomes [58]. Our findings suggest that premorbid IQ influences task performance and retention, underscoring the importance of considering cognitive reserve in remote assessments.

Given that the Sleep Memories app was used in an unsupervised, remote capacity, we acknowledge that contextual and behavioral factors may contribute to variability in SDMC scores. However, we found that except for premorbid IQ, none of the demographic, clinical, sleep behavior, modifiable dementia risk factors, or cognitive variables were significantly associated with overnight memory consolidation. This suggests that while such factors may introduce variability, they do not systematically account for individual differences in SDMC performance.

This was the first study to demonstrate the feasibility of SDMC data collection delivered unsupervised through a chatbot and mobile app. While not a replacement for laboratory studies, the Sleep Memories app demonstrates potential as a complementary research tool that may support more accessible and cost-effective data collection.

The comparable task completion rates between participants with SCD and those with MCI indicate that the Sleep Memories app and the word-pairs task can be feasibly administered across these populations. Nevertheless, as with other digital cognitive tools, unsupervised SDMC testing presents challenges related to cognitive load and user engagement. In this study, we also highlighted key factors that may facilitate the adoption of the app, which could inform the development of personalized interventions aimed at optimizing healthy brain aging in the future.

Participant feedback emphasized the value of clear, concise instructions, and well-defined expectations regarding task timing and completion. To support usability among older adults, accessible design strategies are essential, such as larger icons, high-contrast text, intuitive audio cues, and customizable interface settings. Ultimately, refining and validating remote memory tools such as the Sleep Memories app across global cognitive and demographic profiles will be critical for advancing equitable, scalable cognitive health monitoring in clinical and community settings.

There are several limitations to this study. First, we did not record participants’ feedback. Future qualitative studies should consider audio-recording responses and conducting formal thematic analyses of verbatim data. Nonetheless, we identified several factors that influenced participation and completion rates. Second, we did not systematically assess the digital literacy of participants, which may have influenced their ability to use the app and interpret results. Third, as widely documented in the literature, the validity of remote cognitive testing must be carefully considered, given the potential for uncontrolled distractions and external assistance. Reassuringly, the observed scores in this study were comparable to those reported in similar lab-based SDMC studies involving comparable cohorts [5]. Nonetheless, incorporating additional validity safeguards and conducting larger-scale evaluations will be important to further mitigate these limitations. Fourth, the study did not directly validate against supervised lab-based SDMC assessments. However, complete equivalence should not be expected, given differences in delivery format and response conditions, and the observed scores were comparable to those reported in similar lab-based SDMC research in older and MCI samples [5]. Finally, sleep and wake times were self-reported, and no objective measures were collected to verify these estimates. Notwithstanding these limitations, the study has strengths worth mentioning. For instance, it demonstrates high ecological validity and that recruitment and SDMC task delivery via the Sleep Memories app are feasible for a memory clinic setting.

This study demonstrated the feasibility of using the Sleep Memories app to assess SDMC in older adults within a memory clinic context. These findings offer valuable guidance for the development of accessible digital tools that can effectively monitor cognitive health across varying levels of impairment.

The team is currently continuing to collect SDMC data using the app from a larger sample recruited through a brain health and memory clinic setting to support ongoing refinement and larger-scale validation studies. These efforts aim to further establish the feasibility, reliability, and clinical utility of the app, the characterization of neurobiological correlates, and sensitivity to interventions and to inform scalability for community-based implementation.