Markerless Motion Capture to Quantify Functional Performance in Neurodegeneration: Systematic Review

Introduction

Markerless motion capture (MMC) technology uses sensors and advanced software algorithms to track and analyze human movement, without the attachment of physical markers to individuals or the use of external devices such as pressure sensors or wearables. There is growing use of MMC to provide highly accurate quantitative parameters of physical function including mobility [1,2], balance [3], upper extremity tasks [4], and activities of daily living (ADL) [5].

While 3D motion capture systems using markers are considered the gold standard for movement analysis, they have several limitations including their lack of portability, the need for trained staff, and the requirement for reflective markers to be placed precisely on participants’ bodies [6]. In contrast, the use of MMC provides several advantages: being easier to operate, requiring less space, and being more economical than traditional marker-based systems [7]. Importantly, their ability to capture movement unobtrusively is a key benefit for user compliance [8], particularly when working with individuals with cognitive impairments.

MMC is attractive for health care and research use, such as monitoring functional performance loss or improvement in neurodegenerative diseases. While traditional movement analyses are based on subjective clinical assessments, MMC can be used to generate objective and quantifiable digital biomarkers that can help detect a decline in functional performance by capturing movement unobtrusively [9]. Variations in these digital biomarkers could indicate underlying impairment and enable earlier support. The fact that MMC can be deployed in home environments may avoid unnecessary hospital visits for patients, as well as detect subtle changes in functional ability that may only be apparent in everyday home-based settings rather than within a clinic.

Several MMC devices can provide cost-effective assessments of functional performance in research and clinical settings. Broadly, the 2 main types of MMC camera hardware are depth cameras and standard red-green-blue (RGB) video cameras, used in single or multicamera systems. Commonly used and widely accessible depth cameras are the Kinect (Microsoft) devices, which use standard RGB color video as well as depth estimation by recording the distance between the camera and each pixel through the emission of structured light patterns [10]. Machine learning algorithms can be used to reconstruct 3D skeletal models in real-time from the RBG+depth (RBG-D) image. Alternatively, deep learning can be used with standard video cameras or mobile phone cameras to record limb location and orientation. This method uses deep neural networks trained from large datasets to estimate body segment position and orientation (pose) and motion tracking, without explicit depth sensing. It requires specific body segment positions known as the 6 degrees of freedom: 3 rotational (flexion or extension, abduction or adduction, and rotation about the longitudinal axis) and 3 translational (sagittal, frontal, and transverse) [11]. Both forms of MMC have shown promising use thus far.

A scoping review of single-camera MMC models used in health care highlighted the significant potential for use in clinical applications but also noted the need to improve their tracking accuracy [12]. A previous systematic review of MMC-based training devices used in neurological rehabilitation found that these devices improve motivation and enable better functional performance potentially due to the gaming element [13]. Another systematic review of MMC-based devices in rehabilitation found that balance training with the support of MMC resulted in better outcomes potentially due to more dynamic training conditions [14]. While those systematic reviews explored the use of MMC specifically in rehabilitation training, this review focuses on the technology-based evaluation of functional tasks. The recent increase in the number of studies involving MMC-based movement analysis in neurodegenerative diseases offers a strong rationale for this review. This trend includes the use of MMC to track gait decline [9], assess fall risks [15,16], detect disease traits [17], estimate disease severity [18], and detect cognitive impairment from gait features [19].

Neurodegenerative diseases such as dementia and Parkinson disease (PD) lead to declining functional performance. Detecting problems in everyday functional tasks in these patient groups can help provide early, timely, and clinically appropriate interventions that may help maintain independence, decrease caregiver burden, and potentially slow the rate of functional decline [20,21]. MMC can provide digitally measured functional performance data that could be used to enhance clinical decision-making and remote monitoring; identify risks such as falls; and better capture the impact of rehabilitative, pharmacological, and surgical interventions. Although MMC technology could offer the potential for detecting functional changes in neurodegenerative diseases, a model that is comparable to established gold-standard motion capture systems is essential for deployment in real-world applications. This study aimed to complete a systematic review of published literature on the use of MMC with full-body tracking for quantifying functional performance in people with dementia, mild cognitive impairment (MCI), and PD.

Methods

Study Design

The web-based Covidence (Veritas Health Innovation) software platform was used in this review, and the titles and abstracts were screened by 2 independent reviewers. The full text of the relevant studies was reviewed, and the quality of the studies was assessed by 2 independent reviewers. Data extraction was also performed by 2 independent reviewers, and any conflicts were resolved through discussion.

Search Strategy

The search strategy was designed to include all types of studies that used MMC with full-body tracking in individuals with dementia, MCI, or PD. To identify relevant studies, a combination of both the Medical Subject Headings thesaurus and free-text terms related to the 3 conditions and MMC technology were used. The search included publications from all years in the CINAHL, Embase, MEDLINE, and Scopus databases using the terms “Motion Capture,” “Motion Analysis,” “Movement Analysis,” and “Pose Estimation” in combination with “Dementia,” “Mild cognitive impairment,” and “Parkinson’s disease.” The details of the search activity can be found in Multimedia Appendix 1.

Inclusion and Exclusion Criteria

The inclusion criteria for the systematic review were as follows: (1) markerless optical motion capture; (2) full-body tracking; (3) involving participants with dementia, MCI, or PD; (4) original research; and (5) English language studies. Studies with the following characteristics were excluded: (1) motion capture with markers, inertial measurement units, body-worn sensors, or pressure sensors; (2) movement analysis of specific parts of the body or symptoms such as tremor and rigidity; (3) evaluating interventions such as exercises, deep brain stimulation, medication, rehabilitation protocol, dance, and gaming; and (4) pose estimation of videos found on the internet.

Data Extraction

The general information extracted from the studies included: the center and country where the study took place; study characteristics; funding sources; age, sex, and number of participants; number and duration of visits; study aims; inclusion and exclusion criteria; and the main disease condition evaluated. Methodological information extracted included technical details of the MMC system used; functional performance area evaluated, for example, gait or sit to stand; software used for feature extraction; and the method of analysis. The results information extracted included the following: statistically significant movement features, whether they were measured under single or dual task (motor or cognitive) conditions, whether compared to established gold standard models or a relevant clinical measure, and key outcomes including the level of accuracy obtained.

Results

Study Selection

The literature search yielded 1595 results; after removing duplicates, 1159 studies remained for title or abstract screening. Subsequently, 131 studies were identified for full-text screening, of which 26 studies met the inclusion criteria and were included in the review. A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart [22] outlining the selection process can be found in Figure 1.

‎

Quality Assessment

The 26 selected studies were assessed for quality using the Specialist Unit for Review Evidence questions to assist with the critical appraisal of the cross-sectional studies tool [23]. While no studies were excluded from the review based on this assessment, issues pertaining to quality were identified within several of the studies. Table 1 shows that all studies included in this review used appropriate outcome measures (n=26, 100%). Most studies clearly stated the study design (n=20, 77%); provided information on the study setting, location, and dates (n=20, 77%); described the results well (n=19, 73%); and provided participant characteristics (n=16, 62%). However, few studies reported if participants were fairly selected (n=10, 38%) or provided information on participant eligibility (n=2, 8%) and handling of missing data and control of potential biases (n=1, 4%).

Patient Groups

The 26 studies comprised 18 involving participants with PD, 6 involving participants with dementia, and 2 involving participants with MCI. Most (18/26, 69%) of the studies used Kinect sensors for MMC. All studies that included participants with dementia (n=6) used the Kinect sensor and were conducted in inpatient settings. The Kinect was used to quantify gait decline over 10 weeks [9], propose a prognostic model for fall risk [15], and demonstrate the association with clinical gait measures and future falls [16]. In inpatients with dementia and drug-induced Parkinsonism, the Kinect was used to capture Parkinsonian traits [17]; quantify Parkinsonian gait [38]; and along with pose estimation of recordings from a video camera, estimate Parkinsonian severity [18]. It was also used in the MCI studies reviewed (n=2), as a tool to detect MCI from gait features [19,39].

Of the 18 studies that included participants with PD, 10 (56%) reported the use of the Kinect sensor for analyzing gait, including its feasibility to extract relevant features [34,42], ability to detect PD [24,32,41,43], and ability to measure clinical disease severity [28,31,33]. Alternative MMC models that use image processing for pose estimation of videos from RGB cameras have also been used with participants with PD, demonstrating the feasibility of these models in quantifying gait impairment and disease severity [18,25-27,29,30,36,40].

Functional Performance Components

Most studies (24/26, 92%) evaluated aspects of gait, although there were significant variations in the features extracted and methods used for analysis, with some of them lacking statistical significance. Other functional performance components evaluated were sit to stand (n=5) and stepping in place (n=1). Table 2 shows functional performance components by study.

Feature Categories

Table 2 shows extraction feature categories by study. Spatiotemporal features of gait that were reported as having statistical significance included spatial parameters, such as step length (n=8), step width (n=5), and stride length (n=4), and temporal parameters, such as cadence (n=5), gait velocity (n=4), step time (n=4), stance duration (n=1), double support duration (n=1), stride time (n=1), turning time (n=1), turning speed (n=1), swing time (n=1), step velocity (n=1), and stride velocity (n=1). Other extracted feature categories included symmetry (n=9), stability (n=8), range of motion (n=2), and rhythmicity (n=1).

MMC Devices and Feature Extraction Methods

Table 3 shows that most studies (18/26, 69%) used Kinect depth cameras (4 used V1; 12 used V2; 1 used Kinect eMotion; and the latest version, the Azure Kinect, was used in 1 study), while the remainder used regular video or mobile phone cameras. Common camera positioning included frontal views (n=4), ceiling-mounted (n=4), and multiple cameras from different angles (n=3). However, camera position was not reported in 7 of the study papers. The majority of studies have developed their own custom programs (n=10, 38%) or have used open-source libraries (n=8, 31%) to identify bodies in frame and extract movements. The use of propriety software was less common (n=5).

Key Findings

Tables 4-6 summarize the key findings of the 26 studies that used MMC to study movement features in people with dementia, MCI, and PD. Stride length, cadence, gait stability, step length, arm swing, and number of steps were the primary features investigated in these studies. Notably, several studies [26,28,33,37] found that stride length and cadence are commonly affected in those with PD. Other studies [15,31,38] highlighted the potential of MMC for predicting fall risk and discriminating between individuals with PD and controls.

Most studies (20/26, 77%) used some form of clinical validation for the assessment of disease, and patients were referred to or assessed within a clinical research facility by a clinician. The most common clinical measures used were the Unified Parkinson’s Disease Rating Scale (UPDRS) for assessing Parkinsonism symptoms in those with PD and dementia and the Performance Oriented Mobility Assessment–gait and Performance Oriented Mobility Assessment–balance assessments for evaluating mobility characteristics. Many of the studies (23/26, 88%) used the MMC features to classify patients from control participants and to classify symptom severity (eg, UPDRS scores in PD) using various techniques, including support vector machines, random forest models, multivariate ordinal logistic regression, and adaptive neuro-fuzzy inference system classifiers. Several studies reported excellent classification accuracy, with some achieving 100% accuracy [26,27,41]. For instance, Seifallahi et al [39] achieved an accuracy of over 90% for differentiating between people with MCI and controls using an adaptive neuro-fuzzy inference system classifier. Khan et al [26] reported a 70.83% accuracy in predicting UPDRS-gait scores using a support vector machine model, with an area under the receiver operating characteristic curve of 80.88%.

Conversely, most studies included within this review (20/26, 77%) did not evaluate their MMC system or algorithms against an established gold-standard motion capture model, making it difficult to conclude whether their derived features for monitoring functional performance characteristics were comparable to an accepted measure of movement analysis. Some notable exceptions such as Cimolin et al [24] compared their Kinect setup to a Vicon system, which is an accepted and clinically validated method for assessing gait. Other studies used established and clinically validated spatiotemporal measures including the GAITRite system [34,40] and the Zeno Walkway system [37], although the study by Pedro et al [34] only had 2 participants. Li et al [29] had experts manually annotate videos, which, while subjective, proved effective for creating labels to train machine learning algorithms for task segmentation. They also used automated labeling to generate subtask segmentation, which could help automate larger-scale studies and clinical assessments.

MMC models showed moderate to strong positive correlations with Vicon [24], Zeno [37], and GAITRite [40]. However, some of the studies also identified limitations of MMC. For example, Pedro et al [34] found that Kinect cameras may overestimate step length variation in people with PD due to inherent smoothing, while Sabo et al [37] found that automated heel strike algorithms may struggle to identify short steps. Some studies [19,29] reported challenges with data processing and interpretation, highlighting the need for more standardized methods in this field.

Despite these limitations, the findings suggest that MMC is a promising tool for studying characteristics of functional performance in people with dementia, MCI, and PD. It is worth noting that specialized depth cameras may not be necessary for extracting suitable joint positions in camera space [37]. However, further research in this field is warranted to fully understand the potential of MMC.

Discussion

Principal Findings

This systematic review has shown that there is a paucity of studies exploring the use of MMC in people with dementia and models exploring the performance of ADL. Moreover, there is a lack of standardization in the used MMC models and clinical validation in real-world applications. The absence of standardization among the models used posed a significant challenge, precluding the possibility of conducting a meta-analysis to compare and synthesize study results.

The review findings suggest that there is more evidence of the use of MMC with full-body tracking in patients with PD (n=18) compared to those with dementia (n=6) and MCI (n=2). This demonstrates a bias toward movement disorders, where the motor symptoms are more prominent, and highlights a significant knowledge gap in the feasibility and effectiveness of using MMC models in quantifying functional performance in people with dementia and MCI. Moreover, the studies that included patients with dementia [9,15-18,38] were all conducted in inpatient dementia units, indicating a lack of research involving this patient group in real-world settings. This underscores the need for further investigation in this area.

While MMC models based on gait features extracted mainly from straight-line walking may provide useful preliminary data for model development, they have less scope in quantifying functional performance in a real-world context, particularly in people with cognitive impairment. In contrast, the evaluation of ADL tasks could potentially provide more comprehensive insights into real-world functional performance from routine daily activities. Previous research suggests that dual-task tests of mobility are more effective in detecting cognitive decline as well as predicting cognitive impairment and falls [44-46], potentially due to the increased cognitive demand on the individual. However, just 1 study included in this review [19] used dual tasks for the classification of MCI from control, and it was not reported how the completion of dual tasks impacted the results. Feature extraction of ADL tasks that require planning and organization could potentially facilitate the measurement of dual-task performance. Therefore, analysis of ADL tasks could help provide a more accurate assessment of neurodegenerative impairment.

The findings of this review suggest a lack of consensus on the most effective features used. Some spatiotemporal features of mobility such as step length are commonly used, but other features vary widely between studies, making it difficult to determine which are most effective. Additionally, some unique features such as vertical pelvic displacement [42] and D_body, the distance between nose and 2 ankles [30], have been identified in individual studies, but their effectiveness is unknown without further evaluation. Moreover, it is important to note that the effectiveness of several of these feature extraction models has not been tested in real-world settings which therefore requires further evaluation.

Several studies included in the review (n=10) reported machine learning classifier outcomes for identifying people living with dementia, MCI, or PD from control [19,25-27,31,32,35,39,41,43], whereas several others (n=8) reported models that computed clinical assessment scores [17,18,29,30,33,36,38,40]. Although these are useful outcomes, it is important to note that models that help detect gait impairment and predict falls (n=5) [9,15,16,28,42] could potentially be more useful in practical applications for assessing functional performance. It must also be noted that these models were all based on the Kinect cameras demonstrating the potential of RGB-D cameras for detecting and predicting functional impairment.

Accurate feature extraction and classification are crucial for improving the quality of MMC-based functional assessment [47]. The accuracy rates of MMC models reported in the reviewed studies ranged from 40% for a model predicting a clinical assessment score [18] to 100% for machine learning classification of PD from control [27]. Those numbers cannot be compared directly due to the different number of classes and the resulting chance level, as well as the task difficulty between classifying patients from control participants to rating symptoms. However, it is important to ensure that any clinical applications of these models are consistent and accurate because inaccurate predictions could potentially have consequences for patient care. Further validation and refinement of the models may therefore be necessary before they can be safely used in practical applications.

It is important to note that the accuracy of a model does not only depend on its ability to correctly identify a condition but also on its capacity to detect features of functional performance consistently in various real-world settings. Potential real-world applications include the detection of problems in functional performance in clinical settings and functional deterioration in home settings. The effectiveness of several feature extraction models reviewed in this study has not been tested in such settings, and therefore, the accuracy in practical applications remains unclear. Moreover, devices used in clinical applications must be subjected to a rigorous clinical validation process to ensure safety and efficacy before use on patients [48]. Many of the studies reviewed seem to have primarily focused on the technical aspects of the MMC models, such as feature extraction and analysis, with less focus on their clinical utility. Therefore, further MMC research should objectively evaluate the practical clinical and real-world mobility applications of this technology. If a standard MMC movement analysis protocol could be established, functional performance could be compared across diagnoses.

Additionally, the cross-sectional nature of most of the included studies may limit their ability to evaluate and track functional performance over time. Longitudinal studies would be necessary to assess the performance of these models for tracking functional changes caused by factors such as disease progression, infections, and treatment effects or recovery. Despite these limitations, the effectiveness of MMC models using the Kinect [24] and 2D pose estimation [37] in comparison to established gold-standard motion capture systems within experimental settings suggests they may be suitable for testing in real-world applications such as remote monitoring. However, further research is required to explore and address ethical and privacy considerations when deploying MMC devices that capture video and movement within people’s homes. Managing consent where patients lack mental capacity and safeguarding the privacy of patient data that is stored or shared with clinical teams will also need to be carefully addressed while deploying MMC in remote monitoring applications.

It is important to consider the overall quality of studies included in this review, as shown in Table 1, which summarizes key questions to consider when assessing quality. Most studies had a clear study design and focused research questions with appropriate measures of exposures and outcomes. However, only 1 of the studies reported if potential sources of bias from confounders such as musculoskeletal comorbidities, were controlled which could have significant implications for clinical applications. Moreover, few studies provided information on participant eligibility and whether they were selected fairly which could have implications for generalization of study results. The suboptimal quality observed in the included studies in key aspects such as bias control and participant selection suggests these MMC models need to be further evaluated potentially using more rigorous study designs before deployment in real-world applications. Studies that have attempted to create MMC models for fall prediction have primarily focused on retrospective analyses, for example, the number of falls in the past few months. While it is useful to examine historical patterns, future studies should aim to develop prospective studies. Testing the algorithms for MMC models in a prospective study would offer the capability to analyze more detailed information on fall events and contextual associated factors therefore making them more generalizable and valid for predicting falls.

The main findings of this review highlight the potential of MMC in assessing components of functional performance including gait and sit-to-stand characteristics in individuals with dementia, MCI, and PD. Notably, high classification accuracies in several studies demonstrate the potential for clinical applications, such as identifying, monitoring, and predicting outcomes in these populations. However, it is crucial to address the limitations and challenges, such as overestimation of step length variation and difficulty in identifying short steps, as well as the need for standardized methodologies and further research.

A segment of motion analysis research will likely continue to focus on simplified, discrete tasks executed within the controlled setting of a laboratory. However, advancements in technology are progressively enabling the expansion of movement analysis into real-world environments [49]. While the current body of literature predominantly centers on gait analysis, the potential applications of MMC extend far beyond this domain, particularly within the realm of ADL. The integration of knowledge gleaned from analyzing various types of functional tasks will empower clinicians to better assist individuals with neurodegeneration in enhancing their quality of life.

Limitations

It was not feasible to conduct a meta-analysis of the reviewed studies due to significant heterogeneity in the MMC models evaluated, the features extracted, and the analysis methods used. The use of search terms that are not specific to MMC such as motion capture and movement analysis may have introduced the possibility of inherent biases in the search results. However, the adoption of these broad search terms facilitated a more comprehensive screening of studies, encompassing a wider spectrum of the literature. Furthermore, it is important to acknowledge a potential constraint inherent in the search strategy, specifically about the inclusion criterion of full-body tracking MMC models. This led to the exclusion of studies that analyzed the movement of specific body parts. Another limitation of this review is that only a small number of studies met the inclusion criteria limiting the generalizability of this study’s results.

Conclusion

The findings of this review illustrate that the use of MMC technology with full-body tracking has the potential to quantify functional performance in people living with dementia, MCI, and PD. However, the lack of consistency in evaluating these models presents a challenge. Standardization of the extracted features and analysis methods may help overcome the heterogeneity of the evaluation process and propose a framework for assessing future models. The findings further suggest that MMC models based on both RGB-D and standard video cameras are viable options for analyzing movement, yielding similar outcomes. Nonetheless, RGB-D cameras have been favored in models intended to detect gait impairment and predict instances of falling.

It is worth noting that the majority of the reviewed studies evaluated aspects of gait, with no evidence of ADL tasks being analyzed. Future studies should incorporate ADL tasks, as this would be more representative of real-world scenarios, particularly for individuals with cognitive impairment. Moreover, longitudinal studies are required to develop models that could track functional impairment over time and potentially predict decline.

Although accuracy is an important factor to consider when evaluating MMC models for clinical applications, other factors such as comparability to established gold-standard motion capture models and capability for analyzing routine tasks and reproducibility in the natural environment are also important. Therefore, a more holistic approach to model development and evaluation with a clear focus on real-world clinical utility may be necessary to ensure that the models are suitable for use in practical applications.