Mobile devices are becoming commonplace in patient-based research [1]. Their ability to capture sensor data and enable interaction with participants in both observational and interventional studies makes mobile devices a powerful tool to augment traditional data collection approaches [2]. For example, these devices passively record activity with an accelerometer and location via GPS sensors to track physical activity and mobility. This information could be useful in understanding patients’ symptoms in the free-living environment. Such knowledge would be ideal for patients with osteoarthritis who exhibit variable pain experiences that may also interact with their mood and fatigue levels [3,4]. Coupled with sensor-based mobility data, smart devices offer a rich portrait of the interplay between symptoms and mobility levels.

Osteoarthritis is a degenerative and progressive disease affecting approximately 250 million patients worldwide [5]. Pain experiences greatly differ between patients and are often irregular within the same patient [6]. The complexity of symptoms is partly due to the site (knee, hip, or hand), genetic predisposition, initial cause of damage (ie, injury), obesity status, level of inflammation, and environmental factors [5,7]. Traditionally, patients receive treatment after reporting pain complaints and a physical examination along with optional imaging (eg, radiographs) [8,9]. Physical activity patterns, mobility function, and symptoms are used by clinical practitioners to inform treatment decisions [8,10,11]. However, difficulty in retrospective assessment of complex experiences like pain and the recall bias of self-assessing activity patterns present obstacles for care management of patients with osteoarthritis [12]. As a result, there has been considerable interest in using smart mobile devices—phones and wearables—for ascertaining symptoms and objective activity measures for informing practitioners [13]. In 2019, approximately 30 to 40 apps were designed for logging pain symptoms, but only one-fifth of those apps engaged the patients for which they were designed [14]. Moreover, none were solely designed for a smartwatch interface. Mobile devices and smart wearables have the potential to better characterize symptoms in the free-living world, but involvement of end-users (eg, patients) are necessary for appropriate design and long-term adoption.

New tools are needed to collect symptoms, experiences, and patterns of mobility and activity in real time in the free-living environment. Ecological momentary assessment (EMA) is a method based on data collection originally developed by Larson and Csikszentmihalyi in 1983 [15] for the psychological assessment of what activities people engage in, how they feel, and what they are thinking during their daily lives. It was developed because people are poor at reconstructing psychological experiences after they have occurred [16,17]. Rather, EMA considers experiences in the moment in a real-world environment and is potentially more representative of reality [18]. EMAs were first collected using paper diaries, followed by dedicated electronic diaries [19]. Recently, however, smartphone and smartwatch apps are becoming a pervasive means of assessing medical symptoms [20,21]. Work by Murphy and Smith demonstrated that tracking activity patterns with daily EMA fatigue reports yielded insights into the manifestation of activity-induced fatigue in participants with knee or hip osteoarthritis [22]. Another recent report used a custom-designed smartwatch app to prompt older adults with knee osteoarthritis to report their pain 4 to 5 times per day for approximately 3 months. Results demonstrated that older adults wore the watch for 75% of the study duration and answered 50% to 60% of the twice-daily prompts to rate their pain. Despite some drawbacks, including battery drain and technical issues, participants generally thought the watch was convenient and acceptable [23]. Although this previous work is encouraging, additional research is clearly needed to document smartwatch satisfaction, usability, and compliance for knee osteoarthritis symptoms.

The large increase in mobile medical apps has prompted the US Food and Drug Administration (FDA) to release a guidance statement [24]. The FDA is clearly supportive of evaluating patient-reported outcomes [25]; however, the framework for regulating medical mobile apps is still in its infancy [24]. Moreover, FDA guidance documents state that any patient-based software should undergo evaluation for overall design, usability, and acceptability for use in clinical care and research settings [26]. In that regard, the objective of our study was to evaluate satisfaction, usability, and compliance using the real-time and online assessment and mobility monitoring (ROAMM) mobile app designed for smartwatches. This study builds on initial input from interviews about the ROAMM app interface and usability in both patients and practitioners [27,28]. We hypothesized that older adults with knee osteoarthritis would provide positive satisfaction and usability ratings while being compliant with wearing the smartwatch and answering EMA prompts over an approximately 2-week evaluation period.

Gerontechnology is a relatively new concept that aims to promote health and well-being through technology that considers older adults’ needs and preferences [45]. The ROAMM app was developed based on these guiding principles and was designed to capture information about gerontological symptoms in the free-living environment. To ensure the technology is appropriate for this population, our research team and others have conducted focus groups to gather feedback about gero-friendly visualization (eg, display size) and functionality [27,46-49]. In the next phase of this study, we evaluated the technology in a small target sample. In this context, the purpose of this study was to evaluate the ROAMM smartwatch app for usability, satisfaction, and compliance in a patient population of older adults with knee osteoarthritis. Subsequent paragraphs interpret the results within the framework of gerontechnology and compare the current results to the existing literature. Based on our exit questionnaire, a majority of participants positively rated the ROAMM app display and functionality (eg, rotating dial). About half of the participants felt the smartwatch was uncomfortable, but almost three-fourths were likely to participate in a long-term study asking them to wear the smartwatch. Additionally, EMA compliance rates reported here were similar to a recent meta-analysis that pooled data from 701 participants across 12 EMA studies [50]. The high EMA compliance rates also indicate that older adults were able to use the app in free-living conditions. Participants also responded that it was easy to enter information using the rotating bezel, the text was sufficiently large, and they were satisfied with charging the smartwatch and effectively charging it every night. These responses culminated in a high likelihood of participating in research asking them to wear the smartwatch in a 1-year study—a goal for research related to health monitoring. However, it should be noted that willingness to participate in a long-duration study might not transfer to long-term compliance. Overall, our results suggest that older adults with knee osteoarthritis were generally satisfied with the ROAMM app and smartwatch, but the next intervention requires improved comfort and wearability for planning long-term studies.

Battery drain was a consistent issue observed during the study. The ROAMM app collects sensor data simultaneously with EMA data. We previously reported that the battery is most susceptible to the GPS sensor, with approximately 1% battery drain per collected sample [29]. This drain is exponentially increased when all sensors are collected simultaneously and further affected when the screen is activated during EMA responses. In a similar study, investigators from the KOALAP (Knee Osteoarthritis, Linking Activity and Pain) study also struggled to ensure the smartwatch battery lasted during the day—about 15 hours. They also found that the lack of battery life significantly impacted engagement with the smartwatch [51]. Additional innovation is needed on battery technology, smart sensor triggering (eg, activate accelerometer during movement only, activate GPS outside a geofence), and energy efficiency to ensure that apps like ROAMM are capable of health monitoring for an entire waking day. Advances in sensor technology and EMA tools for health monitoring are only effective if sufficient compliance is demonstrated [52]. The compliance rates reached in this study were consistent with systematic reviews of EMA for assessing chronic pain in adults (eg, 83% [53] and 86% [54]). However, achieving good compliance is a multifactorial challenge, as it involves the type of behavioral coaching, perceived burden, demographics of the population, and the usability of the technology [55]. Regarding the demographics, older adults tend to have higher compliance (88%-90% at 75 years old) than younger adults (72%-74% at 25 years old) even in technology-based evaluations, as reported in a chronic pain study [50]. In fact, an EMA-based study in older African American adults reported over a 90% compliance rate when rating their activity and stress, four times per day, on a smartphone [56]. There was also some evidence that fewer questions yielded higher compliance. We observed that a single sleep quality question in the morning yielded the highest compliance. In prior work, microinteraction EMAs—where people are prompted with fast, glanceable questions that could be answered in a few seconds similar to ROAMM—were developed on smartwatches and compared to less-frequent EMA prompts on smartphones. Researchers found that although prompts on the smartwatch were eight times more frequent than those on the smartphone, participants were 35% more compliant to short microinteraction EMAs on the smartwatch [57]. Participants also responded to EMAs in less time and reported the EMAs to be less distracting on the smartwatch than on the smartphone [58]. Therefore, EMAs on a smartwatch might serve as an excellent approach for longitudinal studies, which was also conveyed by a majority of older adults in our study who were willing to participate in a 1-year research study.

Stratified analysis of compliance rates yielded important information for practice and for planning future research. In general, compliance was similar between participants with different opinions of the comfort and satisfaction with the function of the smartwatch and ROAMM app. Unexpectedly, compliance was similar among participants not likely to wear the smartwatch as their own personal watch and those who would not volunteer for a 1-year research study. Participants regularly wearing a wristwatch had significantly higher compliance than nonwearers. Furthermore, individuals who had difficulty reading the text on the watch had lower compliance than those who did not experience difficulties. In the focus group study, approximately 80% of the respondents reported the display text size was adequate [27]. In the current study the same results were found (24/28, 79%) and participants reported the text was large enough. To be more inclusive and generalize to the population as a whole, future studies will need to consider whether people regularly wear watches and ensure text size or fonts are optimized for compliance.

There are strengths and weaknesses of this study that will aid in conducting future research using smartwatch devices for monitoring health. One of the weaknesses is that this study was performed on a relatively small, homogenous sample of older adults with knee osteoarthritis. In particular, this was a well-educated sample, and the results may not be generalizable to individuals with lower levels of education. Furthermore, we did not employ a commonly used ”usability“ scale for assessing the ROAMM app, which makes comparisons to the literature difficult. At the time of data collection, existing scales were not appropriate for assessing both the software and hardware of wearable devices. Moreover, despite internal pilot testing, rapid battery drainage found during wear in the free-living environment remained to be an issue. These weaknesses are balanced with some strengths such as the thorough investigation of usability and user compliance following an extended use of the ROAMM app in real-world settings.

In conclusion, older adults with knee osteoarthritis positively rated and were generally satisfied with the ROAMM app on the Samsung smart watch. Battery life remains a concern and will need to be carefully considered in future studies. Compliance rates were generally high but were impacted by personal experiences wearing a watch and text readability. After using the ROAMM app for about 2 weeks, a majority of older adults were willing to participate in a 1-year study requiring them to wear the smartwatch. Overall, the results support new opportunities to monitor health symptoms while capturing objective sensor information from a smartwatch in older adults with knee osteoarthritis.