The types of technologies were selected based on feedback received from older adults and their caregivers during the needs analysis phase of the project. In the Italian pilot, the needs analysis led to the identification and deployment of two services: monitoring and socialization [29]. In this paper, we focus only on the socialization service.

The socialization service is based on Sentab technology (Sentab Estonia OÜ). The system was developed around a UI working on the web, Android, iOS, and Android TV. From a technical perspective, the Sentab backend solution was based on Enterprise Java on Jetty, open source RabbitMQ, and Redis dockers. Information was stored in a MySQL database on Ubuntu servers. The content delivery network was built on Amazon CloudFront and S3.

In the project, two UIs were used: an Android application installed on tablets (Apulia) and televisions (Tuscany) and an Android app for caregivers installed on their mobile phone. This final choice was made based on the feedback received from pilot managers as an outcome of the needs assessment. It is worth underscoring that the service was the same for both technologies—only the devices changed.

The older adults could interact with the tablet using the touch screen, whereas for the television, a separate remote control was provided and Sentab was accessed using the arrows and an “OK” button (Figure 1). For both technologies (ie, tablet and television), the Sentab technology has the following functionalities: (1) a video calling function, where older adults and their caregivers could video call through the UIs; (2) a stimulating game function, where older adults can access some cognitive games (eg, sudoku, picture memory) and monitor their improvement by checking the cognitive index calculated by Sentab; and (3) a stay-informed function, where older adults and IC can access news and information shared by hospital clinicians regarding best practices for maintaining a healthy and active lifestyle.

An overview of the socialization service can be found online [30]. By connecting to their UIs, caregivers could communicate with their relative (or “assigned” older adult) and share photos or news.

The experimental protocol was composed of the following phases: installation and training, testing of technology, and a reflection meeting. An overview of these phases is presented in Figure 2.

As mentioned in the System Service Description section, Tuscany and Apulia tested different devices. In Tuscany, the smart television was installed in the older adult’s home and then facilitators trained them in the use of the technology. On the other hand, in the Apulian pilot, facilitators trained the IC together with the older adult and the tablet was subsequently given to the older adult to take home and use freely. The participants could ask questions for clarification. At the end of the training, facilitators gave the participants the user manual along with their contact details.

As soon as the participants felt confident with the technology, the facilitators administered the questionnaires, providing baseline data (T0). Participants were then requested to freely use the technology and the service functionalities in their daily life. After 6 months of use (T6), they were requested to fill in the same questionnaire that was administered at T0. A description of the questionnaires is provided in the Questionnaires section.

To better understand the end users, we used an action research approach that suggests organizing reflection meetings to critically reflect on the “results of action” [32] and to underscore the lessons learned in terms of aspects that did or did not work well in the pilot. The reflection meeting was scheduled after the first 6 months of use to collect feedback and it involved meeting facilitators, facilitators, and FCs. The meeting facilitators were in charge of presenting the results, fostering discussion, and taking notes during the meeting.

The meeting was divided in three parts: presentation of the results following the first 6 months of use, discussion with the facilitators and FCs, and an online survey. After the initial presentation, the quantitative results were discussed to understand whether the facilitators and FCs expected or were surprised by these results and whether they could explain the results in a qualitative way. The discussion touched on the following: (1) the things that did and did not work properly; (2) the limitations of the technology; and (3) the findings of the study, including acceptance of the technology. After the plenary discussion, before closing the meeting, participants were asked to complete an online survey to give feedback on the main factors related to technology readiness and its acceptability; additionally, participants were asked to report what they considered to be a significant outcome from the perspective of the older adult, IC, and FC. The results of this meeting and survey were then aggregated to critically discuss the results of the questionnaires and to plan and suggest corrective actions for future tests.

In order to answer this study’s RQs, three cohorts of participants were recruited: older adults, IC, and FC (professionals in charge of their monitoring). A total of 105 people were recruited to be part of the study: 40 older adults, 40 IC, and 25 FC. The older adults were recruited and were randomized 1:1 into either the control or intervention group. The IC and FC were divided into control and intervention groups according to their older adult’s randomization.

The inclusion criteria for the older adults were aged ≥60 years, intact cognitive status (Mini-Mental State Examination score >24), and a frailty score from well to moderately frail; exclusion criteria were presence of severe cognitive impairment and other conditions that cause memory impairment or engagement difficulties. Participants in the control group were recruited with the same inclusion and exclusion criteria; this group did not interact with the technology and was only required to fill in the sociodemographic questionnaire. In this study, we will focus on the user experience of technology use, and thus only focus on the intervention group (older adults: n=20; IC: n=20; FC: n=13).

Participants were enrolled in two Italian regions (Apulia and Tuscany) that composed the Pharaon Italian Pilot. The recruitment strategies were different across the two pilots; in Apulia, the IC was recruited first, whereas in Tuscany, the older adult was recruited first. In Apulia, the participants were recruited at the Casa Sollievo della Sofferenza research hospital (San Giovanni Rotondo, Foggia, Italy) in different clinical units and at the University of the Third Age, and no participants were cognitively or medically compromised. In Tuscany, the participants were recruited among people that were already accessing home services provided by the Umana Persone S.r.l. (Grosseto, Italy) network of social cooperatives. The research was approved by the Comitato Etico Clinico, Azienda USL Toscana Sud Est on July 22, 2021 (Prot. 2021/000227), and by the Azienda USL Toscana Centro on October 18, 2022 (Prot. 2022/22131_spe). Furthermore, the two pilot sites used the technology either on a tablet or a smart television according to guidelines and feedback acquired in the needs analysis [29].

The ethical committee of the Casa Sollievo della Sofferenza research hospital approved the research on June 14th, 2021, with protocol number 89/CE. All participants read and signed the informed consent form before entering the study. The collected data were anonymized and no compensation was provided to participants.

Following the action research framework and to better discuss and reflect on the RQs, we also involved the facilitators, who have different professional backgrounds ranging from engineers to cooperative managers to health professionals, and whose main task was to ease the use of technology in all 3 cohorts and solve any problems that arose during the experimentation. Additionally, facilitators installed the technology and delivered the training to the intervention group and administered the questionnaires.

A total of 20 participants joined the reflection meeting: 5 participants from the University of Florence, who acted as meeting facilitators; 3 participants from the Apulian pilot site (Casa Sollievo della Sofferenza Research Hospital); and the remaining 12 participants were from the Tuscan pilot site (Umana Persone network of social cooperatives). Only 8 facilitators (3 from the Apulian pilot and 5 from the Tuscan pilot) agreed to fill out the final survey.

To gather information about the participants’ characteristics, sex, age, education, and living situation/environment, a semistructured interview was conducted at the beginning of the study. The Mini-Mental State Examination [33] was administered to assess the older adults’ cognitive status.

Usability was measured with the system usability scale (SUS) [34] (Italian version [35]), which is composed of 10 items on a 5-point Likert scale where 1 stands for “strongly disagree” and 5 stands for “strongly agree.” Items 2, 4, 6, 8, and 10 need to be reversed. The total SUS score is obtained by adding all the score contributions and multiplying the sum by 2.5. The SUS ranges from 0 to 100; scores lower than 68 are considered below average.

Acceptance was evaluated through the Almere model questionnaire (AMQ) [36], which is composed of 41 items on a 5-point agreement Likert scale. The AMQ is composed of 12 constructs created by combining specific items: anxiety (ANX), attitude toward technology (ATT), facilitating conditions, intention to use (ITU), perceived adaptiveness, perceived enjoyment (ENJ), perceived ease of use, perceived sociability, perceived usefulness (PU), social influence, social presence, and trust (TRUST). We investigated 6 of these 12 constructs: ANX, ATT, ITU, ENJ, PU, and TRUST. Note that ANX is reversed, therefore a higher ANX score translates to lower levels of anxiety.

To measure the quantity of perceived stress related to technology use (technostress), the Perceived Stress Scale [37] (Italian version [38] adapted as in [39]) was administered. The test comprises 10 items with a 0‐4 Likert scale with 0 meaning “never” and 4 being “very often.” Items 4, 5, 7, and 8 have reverse scoring. The total score is calculated as the sum of the single item contributions. A total score from 0-13 is considered low stress, 14-26 is moderate stress, and ≥27 is high stress.

At T6, participants were also asked to estimate how frequently they used technology by asking how many times they used it per day and per month.

A schematic overview of when and to whom the questionnaires were administered is presented in Table 1.

The demographic and cognitive characteristics as well as technostress at T0 and T6 for older adults and IC are presented in Table 2.

Participants on average used the technology 4 times per week. Overall, 85% (17/20) of the older adults lived in an urban area, and 60% (12/20) of the older adults lived with someone, of which 42% (5/12) lived with their IC. The older adults’ associated FC were predominantly women (69%, 9/13). The FC were of various professional backgrounds including psychologists, nurses, and care workers. No other demographic information was collected for FC.

For usability, at T0 and T6, there was no difference between those living alone and those living with someone. There was no significant change over time in SUS score in either living situation.

For acceptability, at T0, those living alone had significantly lower AMQ constructs than those living with someone (ANX P=.02, Cohen d=1.1; ATT P=.04, Cohen d=0.9; ITU P=.01, Cohen d=1.3; ENJ P=.03, Cohen d=1.0; PU P=.01, Cohen d=1.4; TRUST P=.01, Cohen d=1.3). After 6 months, the difference disappeared (see Table S2 in Multimedia Appendix 1 for mean and median values).

In older adults living alone, there was no change in AMQ constructs over time, whereas in those living with someone, ANX, ATT, ENJ, and PU declined over time (ANX P=.03, Cohen d=0.7; ATT P=.02, Cohen d=0.8; ENJ P=.03, Cohen d=0.7; PU P=.04, Cohen d=0.6; see Table S2 in Multimedia Appendix 1 for mean and median values).

The first research question aimed to investigate the role of time in technology perception intragroup and between cohorts of participants (RQ1). As for the acceptance evaluated intragroup, except for ATT, the differences reported in Figure 4 are just trends, as they are not statistically significant. It is worth mentioning that the older adult values for ITU (T0=3.88; T6=3.53) and ATT (T0=4.20; T6=3.65) were comparable to the intention to use and attitude measured in a related work (ITU=3.34; ATT=3.73) [44]. Moreover, values for ENJ, TRUST, and PU were similar to those of an older Chinese sample, demonstrating that the values we reported are aligned with the literature [45] even though the populations are geographically different. Note that we evaluated ITU and ATT with different models, but these constructs had comparable items. Additionally, in this study we observed a decrease of ATT for all 3 cohorts of participants over time. This result is aligned with the feedback collected during the reflection meeting; indeed, the facilitators emphasized how engagement is strongly interconnected with the personal intrinsic motivation that leads to long-term use of the technology. It was also noted that the FC or IC contacting the older adults through the technology (whether video calling or news sharing) helped in keeping the participant engaged. The higher value of ATT at T0 may be due to the older adults’ initial high expectations. Nevertheless, despite the decrease in ATT, the participants stated they used the system on average 4 times per week.

It is also worth noting that we observed a higher TRUST value for the FC after use compared to the other two cohorts. Higher trust is important because trust is strictly linked with the use of technology. These results are aligned with the feedback collected during the reflection session that highlighted some participants’ mistrust in technology use at the beginning of the study, during the training session.

Usability was rated differently between older adults and their caregivers, who on average rated the system higher through SUS. These results are aligned with a recent survey [10] that highlighted a different attitude and expectation toward technology according to age group. Indeed, the statistical difference between the older adults and IC group disappeared when including age and sex as controlling variables. Other than age and sex, there could be other factors that contributed to this result, such as the older adults’ digital competence. Furthermore, as described in the Technology Reliability and Ease of Use section, it emerged that in Tuscany, the system was perceived as more difficult to use by older adults, and they needed the caregivers’ support to use it. Nevertheless, despite the lower values, the usability for older adults increased over time, though not significantly. This suggests that older adults overcame their initial technical barrier and learned how to use the system, and this was independent of their living situation. Facilitators also noted that technical problems that occurred during the trial or problems surrounding the older adults’ experience of using the technology negatively impacted technology acceptance, as also observed by Peek et al [27].

In the Apulian pilot, there was a decrease in ANX, which translated into an unexpectedly higher anxiety perception (see Table S1 in Multimedia Appendix 1 for mean and median values). On the other hand, in the Tuscan pilot, there was no significant decrease or increase in acceptability. However, it is also worth noting that the recruitment of the older adults in the Tuscan cohort was performed among those using domiciliary care services and thus included older adults with higher frailty and lower Mini-Mental State Examination score compared to the older adults in the Apulian pilot site, which might influence perception of the technology. This was also brought up in the reflection meeting. The difference in user experience between the 2 pilots could be related to the different types of technology (television and tablet) selected at the beginning of the study or to the slightly different profile of the 2 cohorts of older adults.

RQ2 aimed to investigate the role of living situation on acceptance; we found that those living alone had significantly lower AMQ constructs compared to those living with someone. However, this difference disappeared at T6. It should also be pointed out that those living with someone had decreased ANX, ATT, and PU over time, indicating that the anxiety toward the system increased, and the usefulness and attitude toward technology lowered, suggesting that having someone helping them use the system may remove the perceived usefulness of the technology. This finding could suggest that living situation may be a barrier at the beginning of the study: people that are living alone may be more skeptical at the beginning, whereas people who are living with someone may have had higher expectations, yet the reality after 6 months of use was disappointing, which lowered the acceptability of the system.

Finally, this paper aimed to investigate how stress related to technology may affect the acceptance and usability of a certain technology (RQ3). The results highlighted a strong correlation between technostress and usability, anxiety, and trust in the older adults at T0 (Figure 5A) but all the correlations disappeared at T6. The results obtained at T0 are aligned with our previous findings [39], where we highlighted a link between perceived stress and related acceptance. However, these results may suggest that technostress could be a barrier only at the beginning of the study; as soon as the older adult becomes familiar with the technology, the link between perceived stress and related acceptance disappears. To mitigate the effects of stress at the beginning of the study, proper training sessions should be organized. These sessions ought to be devoted to successfully teaching participants how to use the technology and recall the functionality of the system after the training session, adapting the training to the participant’s cognitive and educational level. As remarked by facilitators, oftentimes older adults forgot how to use the devices and tended not to use them unless facilitators retaught them how to use the technologies. Indeed, well-conceived training is a key feature of success and critical for technology acceptance.

At T0, for the IC group, higher perceived stress was a barrier to trusting technology, and in the FC group, stress affected attitude and usefulness of technology, possibly because of fear of substitution or the worry that the technology may involve extra work for them [46-48]. Nevertheless, the correlations were not significant at T6 (Figure 5B,C). This highlights how technology-related stress for the proposed socialization service can be a barrier to technology acceptance; as caregivers can influence older adults’ perception of technology, it is crucial to include them in the study to get a multistakeholder perspective to support and reassure their older adults and promote a positive attitude toward technology.

The main limitations of this work are the sample sizes and the duration of the test phase (6 months), as well as the different recruitment strategies carried out at the 2 different sites. As for the sample size, we aim to increase the number of participants in the 2 pilots while tracking their cognitive abilities, thus evaluating the effect of this variable on the use of technology. Further studies should also be planned to extend the duration of the testing phase, given that our preliminary results at 6 months seemed to indicate that it was not long enough to get information regarding the impact of this service in real life. This is also of interest to policy makers who are coping with staff shortages and increasing health care expenses. Another encountered limitation was the different recruitment process between pilot sites and the lack of randomization for the technologies. These factors can all contribute to different results between samples; researchers should aim to standardize procedures as much as possible to obtain generalizable results. Future studies should also investigate the factors that may influence the low use of the technology and come up with countermeasures to encourage the use of technology, which can also have an impact on acceptability [27].

This paper investigates the role of time, living situation, and stress related to technology use on the usability and the acceptance of a socialization service. This paper presents the results collected after 6 months of use considering a multistakeholder perspective. In this study, we found that the older adults had higher stress and anxiety toward technology than the caregivers. Nevertheless, getting accustomed to technology over 6 months of use removed this initial barrier. It is also important to consider the living situation of the older adults as those living alone had lower acceptability than those living with someone, which could suggest an increased resistance to change. However, counterintuitively, the older adults living with someone had a decrease in enjoyment, usefulness, and attitudes toward technology, possibly because living with someone limits the need of the older adults to socialize with others. The reflection meeting with the facilitators qualitatively highlighted demographic barriers in the use of technology that should be further evaluated quantitatively.