Using Virtual Reality to Assess Spatial Navigation Ability in Individuals With Mild Cognitive Impairment and Older Adults: Cross-Sectional Study

Introduction

Spatial navigation impairment (SNI) is frequently encountered in people with Alzheimer disease (AD) and may appear in the early clinical stage of AD [1]. These patients usually compensate for their difficulties by using geographic or literal cues, taking fixed routes, or reducing their range of ambulation. As the disease advances, their ability to construct allocentric spatial representations and the translation between spatial reference frames becomes progressively impaired [2]. At this stage, they are no longer able to adopt alternatives when they are disoriented and may get lost [3,4]. Getting lost may result in going missing when efficient problem-solving strategies are absent or when timely help from pedestrians is unavailable. Although getting lost or going missing usually occurs in people with AD, it may also occur in those with neurodegenerative disorders due to non-AD pathology (MC Pai, unpublished data, April 2024). Under such conditions, the manifestations may be different from those of AD. People with Lewy body diseases, for example, may lose their way due to impaired egocentric spatial memory, executive dysfunction, or attentional deficit [5].

Impaired episodic memory is arguably a powerful indicator of hippocampal dysfunction and, hence, a valuable clinical feature for a diagnosis of AD. Notably, SNI or even getting lost may appear before episodic memory impairment in some patients with AD as one of the incipient symptoms [1]. In focal brain region models, the medial temporal lobe, especially the hippocampus, reflected the activity of allocentric navigation and episodic memory [6]. As the brain regions damaged by initial AD pathology overlap with the brain areas essential for human spatial navigation, SNI has been proposed as a marker for the diagnosis of AD [7-10]. Hence, the detection of SNI can not only prevent instances of getting lost or going missing but also identify early symptoms of clinical AD.

Currently, many tests have been developed to assess spatial navigation abilities. A real-world and large-scale setting is definitely more valid, direct, and accurate. However, unpredictable or unfavorable conditions may arise during testing, such as bad weather, physical discomfort, obstacles, or even accidents [11]. Nowadays, the various forms of virtual reality (VR) are mature and ready to assess spatial cognition in humans [12,13] (Table 1). For example, Tu et al [14] designed an indoor-environment VR test that was used to test spatial navigation abilities and episodic memory to distinguish AD from frontotemporal dementia, whereas Rekers and Finke [15] provided a novel design that reduced the confounding factors of fine motor skills to test the two abilities. However, many patients with AD lost their way in the outdoor environment, causing great life-threatening issues and social burden. Therefore, we hoped to design a large-scale virtual world to test SNI based on our previous model conducted in the real world.

As a result, the aims of this study were to test the feasibility of a VR setting to assess sense of location in people with mild cognitive impairment (MCI) and the power of VR to discriminate among groups with different clinical conditions. We hypothesized that the MCI group would have the poorest navigational performance compared to the cognitively unimpaired (CU) and subjective cognitive decline (SCD) groups, whereas the CU group would perform better than the SCD group.

Methods

Participants

We conducted a prospective study on individuals with cognitive impairment or concerns who regularly visited a special clinic in a university hospital. Participants included individuals with MCI and SCD and individuals who were CU. A diagnosis of MCI was made by a senior behavioral neurologist, MCP, based on patients’ clinical manifestations, the results of neuropsychological examinations, brain magnetic resonance imaging, and single-photon emission computed tomography [34]. Some participants received tests for AD biomarkers (Table 2). All participants with MCI met the Petersen criteria for MCI [35]: (1) memory concerns, preferably corroborated by an informant; (2) impaired memory function for their age and educational level; (3) preserved general cognitive function; (4) intact activities of daily living; and (5) absence of dementia. Similarly, a diagnosis of SCD was based on the chief concern of the patients and the scores on the Subjective Cognitive Decline Questionnaire (SCD-Q) [36].

Because of the novelty of using this device to quantify spatial disorientation and the absence of established normative benchmarks for key outcome measures such as linear deviations (LDs) and vector deviations (VDs), a priori sample size calculation was based on effect sizes reported in related literature [13]. Assuming a large effect size (Cohen f=0.40), a significance level of .05, and a statistical power of 80%, we estimated that a total sample size of 84 participants would be sufficient to detect statistically significant group differences using one-way ANOVA. To account for potential exclusions due to VR-related adverse effects or incomplete data, we planned for an initial recruitment target of 90 to 100 participants, anticipating an attrition rate of up to 20%.

Basic Measurements

Basic measurements included the Cognitive Abilities Screening Instrument [37], Clinical Dementia Rating Scale [38], Geriatric Depression Scale [39], SCD-Q [36], and Questionnaire on Everyday Navigational Ability (QuENA) [17]. The QuENA consists of patient (QuENA-P) and caregiver (QuENA-CG) versions. The definition of caregivers in this study was the spouses, children, or key care partners of the participants. We used perceived ease of use [40], the Computer Anxiety Scale [41], and perceived enjoyment [42,43] to assess domains from the technology acceptance model (TAM). Moreover, we inquired and recorded any symptoms of cybersickness, including nausea, oculomotor discomfort, and disorientation. Most of the questionnaires were administered before the VR test, whereas several questions regarding cybersickness and technological acceptance were administered after the test to find out the feasibility and practicality of this device.

Assessment of Spatial Navigation Abilities

As indicated in our previous research [13], we remodeled the Tzu-Chiang campus of National Cheng Kung University to adapt it to our VR devices due to its ecological validity and for further comparisons. The Pai-Jan virtual reality (PJVR) device set included a VIVE Pro Eye head-mounted display, joysticks, and sensors, which were contract-manufactured by HTC International Corporation, whereas the software was from SteamVR (Valve) and the Unity game engine (Unity Technologies). In the pretest phase, the participants were instructed to approach the designated punch-in points A, C, and E in the first round and D and B in the second round by manipulating joysticks, which allowed them to move forward and backward, punch in, and select the answers to the questions. The selection of punch-in points was based on visual blockage from the buildings in this model (Figure 1). The other scenario details, such as the exterior of the buildings, the texture of the asphalt road, and the addition of trees, cars, and other materials, strived to be the same as those of the real world. Two parameters, LD and VD, were tested in this study. LD was defined as a relative value (in meters) based on the coordinates established in the system determining the degree of difference between the location where the participant punched in and the correct punching point. VD was defined as the angular difference (in degrees) between 2 vectors originating from the starting point: one connecting the starting point to the target checkpoint (the correct location) and the other representing the participant’s pointing direction toward the presumed start point. LD was tested using the computer system, which calculated the relative distance between the standing point of the participants in the VR world and the correct punch-in point. We also asked the participants to point at the starting point (Chi-Mei building, the building with white exterior), and the computer determined the angular differences through the in-built system. The VR tests concluded after 2 rounds, and the remaining questionnaires mentioned previously were then administered. All the examinations were conducted in a classroom with great care for security and comfort to reduce other factors that may affect the results of the tests.

‎

Data Analysis and Statistics

All data were properly stored in an ethical manner. Continuous variables are presented as means and SDs, whereas categorical variables are summarized as counts and percentages. To compare demographic characteristics across the 3 groups, we first assessed the normality of the continuous variables. On the basis of the results, one-way ANOVA was applied to variables with a normal distribution, whereas the Kruskal-Wallis test was used for those that did not meet the normality assumption. For categorical variables, the chi-square test was used to examine group differences when expected cell counts were adequate. When the overall group comparison was statistically significant, post hoc pairwise comparisons were conducted using least significant difference to identify specific between-group differences. A correlation was analyzed between the total QuENA score and LD and VD. The main purpose of this study was not to concentrate on over- or underestimation judgments of distance or vector, so the VDs in the statistical analysis were processed in absolute values.

All statistical analyses were conducted using SPSS (version 17.0; IBM Corp), with a 2-sided significance level of α=.05.

Ethical Considerations

This study was approved by the ethical review board of National Cheng Kung University Hospital located in Tainan, Taiwan (B-BR-110-046). All participants gave their written informed consent to take part. The data were deidentified during analytic processes and were stored in an ethical and secure manner to prevent data leakage.

Results

A total of 128 participants provided informed consent. However, of these 128 participants, 15 (11.7%) were not tested using the PJVR device because 10 (67%) changed their minds and 5 (33%) had medical issues. Of the 113 participants who initiated the trials, 20 (17.7%) developed cybersickness and were unable to complete the PJVR test, including 4 (20%) who were CU, 7 (35%) with SCD, and 9 (45%) with MCI. Eventually, 93 participants finished the procedure, including 22 (24%) who were CU, 39 (42%) with SCD, and 32 (34%) with MCI. The demographic data are shown in Table 3.

Table 4 shows the results for the TAM domains and cybersickness. Regarding the TAM, the CU group expressed more enjoyment than the SCD group, whereas the MCI group manifested less perceived ease of use than the other 2 groups, which warrants further investigation. Regarding cybersickness, no difference was detected in any symptoms among the 3 groups. This is an important finding for future application.

Findings of the Core Test

The MCI group took more time to complete the test than the SCD or CU groups in the counterclockwise A-C-E sequence, but no difference was detected in the clockwise B-D sequence.

The mean LD of the CU, SCD, and MCI groups was 38.2 (SD 39.5), 50.4 (SD 40.7), and 100.4 (SD 46.2) meters, respectively (P<.001; Table 5). LD showed a large group effect (F_{2, 90}=17.87; P<.001; ω²=0.27; partial η²=0.28; f=0.63). In the same way, the MCI group yielded a greater VD than either the SCD or CU group (P=.02). VD showed a small to medium group effect (F_{2, 90}=4.33; P=.02; partial η²=0.088; ω²=0.067; Cohen f=0.31).

The LD of the PJVR device was correlated with the total scores on the QuENA-CG (P<.001), indicating good ecological validity. The correlation with the scores on the QuENA-P was not significant (P=.07). As expected, the MCI group yielded a larger LD and VD than the other groups in judging their current location and path integration, the functions of which are highly related to the entorhinal cortex [44].

Discussion

Feasibility

The PJVR device is theory-driven and carefully designed to detect deficits in spatial cognition in people with cerebral AD pathology [13,26]. The fact that 82.3% (93/113) of the participants completed the procedures with satisfactory technology acceptance supports its feasibility for use in older adults and individuals with MCI. Our recent research showed that people at the stage of all-cause MCI were at risk of getting lost or going missing. Hence, a test to detect SNI in this group is important. This study is part of the limited research that has used an immersive VR setting to assess spatial cognition in MCI in Asian countries, and the results may encourage more researchers to develop similar devices in this field.

Principal Findings

The CU and SCD groups shared similar features, mostly in demographics, cognitive assessment, and functions of sense of location and path integration. The SCD group was similar to the MCI group in SCD-Q and QuENA-P scores as these 2 questionnaires were self-reported by patients and the SCD group may have downplayed their spatial ability and exaggerated their cognitive impairment out of anxiety.

As expected, the MCI group spent more time completing the test as their spatial navigation ability was worse than that of the other 2 groups (Table 5). The correlation was high between the LD and the QuENA-CG score but not between the LD and the QuENA-P score (Figure 2). The QuENA-CG assesses explicit navigation behavior observed by others and shows good predictive accuracy for risk of getting lost [17,45]. Hence, it reflects the SNI of these patients and is useful for identifying people at risk of getting lost or going missing. However, the MCI group may have overestimated their navigational ability, and the greater the overestimation, the greater the likelihood of getting lost [17,45]. This may explain the poor correlation between the LD and the QuENA-P score. This was also true for the correlation between VD and the total scores on the QuENA-P and QuENA-CG.

‎

In terms of the comparison between the CU and SCD groups, both showed similar navigational abilities and mental functions, which were supported by the LD and VD results and the Cognitive Abilities Screening Instrument scores in our study. However, in the QuENA-P, the SCD group showed similar patterns of getting lost to those of patients with MCI in several aspects.

The absence of a difference in LD or VD between the CU and SCD groups did not support our hypothesis. One of the reasons is that SNI does not appear before the clinical stage of MCI. However, this was a cross-sectional study rather than a longitudinal one, so it was not possible to recognize the gradual deterioration of navigational abilities in patients with MCI who had been through the CU and SCD periods. Another possibility is that there was a difference in SNI between these 2 groups, but the current assessment was not sensitive enough to detect it. On the other hand, the SCD group showed mismatched results in comparison with patients with MCI in subjective getting lost experience and the standardized PJVR testing environment, which may be due to distinct severity of spatial disorientation.

Clinical Implications

Daily navigation in a familiar environment usually takes only a few minutes to half an hour. When traveling from one place to another, events that occur along the way may distract the navigator. Keeping the destination or sequential stops along the journey in mind is a challenge to those with poor spatial orientation and episodic memory. However, traditional neuropsychological assessments are not good at predicting the risk of getting lost or going missing for individuals at the stage of MCI or very mild AD dementia [3].

The QuENA examines navigation deficits in daily life explicitly and is superior to many similar questionnaires [17,46,47]. In this study, the correlations between the QuENA-CG total score and both LD and VD on the PJVR device were significant (Figures 2 and 3; for LD, P<.001; for VD, P=.03). On the basis of this, for those with poor performance on the PJVR device, the risk of getting lost or going missing is high when timely help from others is unavailable and their problem-solving ability is also impaired. The definition of poor performance on the PJVR device using LD or VD cutoff values needs further study to verify its external validity.

‎

The QuENA-CG assesses the risk of getting lost or going missing [45], and the PJVR device simulates real-world navigation scenarios. Thus, a poor performance on the latter indicates a higher risk of getting lost or going missing. However, different people may adopt different methods to overcome their disorientation and prevent themselves from going missing. Most older adults, for example, use an egocentric instead of an allocentric strategy to compensate for the impaired hippocampal function [47]. In this regard, the QuENA-CG score shows the risk of getting lost or going missing, and the PJVR device reveals evidence of SNI.

From a clinical viewpoint, patients are often aware of their deficits in spatial navigation but are unwilling to report the problem to others because of pride. Another possibility is that they cannot retain these disorientation or getting lost experiences in memory long enough to inform others. Gradually, difficulties in wayfinding increase until getting lost or going missing eventually occurs. Studies have revealed that baseline spatial navigation ability can predict clinical progression in patients in their midlife or with cognitive impairment [20,48-50]. Thus, we recommend a test or a questionnaire, such as the PJVR device or the QuENA, to assess spatial navigation ability in people with cognitive impairment, especially for those with suspected cerebral amyloid pathology. Moreover, many investigators have recommended SNI as a marker for diagnosing early AD [7-10,26]. Modification of the PJVR device to a simpler version is needed because the era of unsupervised digital cognitive assessment is imminent.

Comparison With Other Similar VR Settings

In comparison with other immersive VR settings used to evaluate spatial navigation ability, the PJVR device has several advantages. First, it was remodeled from the real-world Pai-Jan device, and head-to-head comparison is possible. During the PJVR test, participants wear VR headsets and orient themselves by turning their heads, and navigation is typically performed using a monitor and joystick. However, due to space limitations, participants used handheld controllers to move rather than walking physically. As a result, proprioceptive input such as joint position and actual horizontal displacement was not engaged, making the PJVR device relatively less immersive in this respect compared to the original real-world Pai-Jan device or other tasks conducted in smaller physical spaces where participants can walk, such as the triangle completion test [12] or Morris Water Maze [28] or setups such as the Virtual Environment Navigation Lab [29]. Second, as a VR spatial navigation test, the PJVR system features a testing environment modeled after a real university campus, enhancing its ecological validity compared to other VR-based navigation tasks such as the triangle completion test [12], Morris Water Maze [28], loop closure test [30], or Y-maze task [31]. These settings are computer-generated or indoors or involve nature or wild scenarios rather than human environments. As mentioned previously, an outdoor scenario is more akin to real-world environments where people with cognitive impairment would get lost or go missing.

Third, the PJVR device is a theory-driven design based on neuroscience evidence and safer than real-world outdoor tasks. The PJVR device assesses not only distance errors during navigation but also sense of direction (eg, the ability to return to the starting point) and navigational efficiency (measured via completion time). As participants must pass through multiple checkpoints in sequence, navigation errors can accumulate progressively. In other words, the PJVR device captures sequential errors that occur during long-route spatial navigation, which is an aspect that is rarely examined in other VR-based navigation tasks. Given the spatial complexity of the PJVR environment, executive functions may also be required for task completion. To reduce the cognitive demands related to direction selection and route planning, participants follow a predefined path. Notably, the design ensures that checkpoints are obstructed by buildings, preventing direct visual access. This reduces reliance on landmark or beacon cues and visuospatial memory, thereby emphasizing the use of allocentric spatial navigation strategies. Finally, similar to other VR settings, the PJVR device is a standardized, well-designed program, and the researchers could record parameters of interest precisely for analysis. More detailed comparisons can be found in a book to be published [51].

In summary, the PJVR device primarily evaluates allocentric spatial navigation ability, achieves a notable degree of ecological validity, and measures multiple dimensions of spatial navigation performance. Its unique design allows for the detection of more detailed navigational behaviors. With appropriate application, the PJVR device holds potential for advancing new discoveries or the development of novel hypotheses in spatial cognition and dementia research.

Limitations

In this study, participants with MCI were carefully selected based on clinical manifestation, neuropsychological tests, and neuroimages. Regarding the concept of early and late MCI, participants with late MCI might show worse performance in sense of location as assessed using the PJVR device. However, the number of participants in this study was insufficient to divide them into early and late MCI groups. Future studies with more participants and using amyloid positron emission tomography scans or fluid biomarkers to verify cerebral amyloid pathology are needed to answer this question. In addition, due to the innovation of our study, only a few studies could serve as the foundation, which might lead to insufficient explanatory power of the results. Nevertheless, this research provides a protocol for future studies to refer to, and the results presented in this paper could be verified by other researchers.

In conclusion, the PJVR device is feasible for assessing spatial navigation ability in people with MCI and can distinguish them from those who are CU or with SCD. Poor performers in the PJVR test have a higher risk of getting lost or going missing, and their care partners must take action to prevent unfavorable events.