A Step Forward Understanding Directional Limitations in Markerless Smartphone-based Gait Analysis

1. Introduction

In clinical settings, quick and accurate tests are essential for patient care. The Timed Up and Go test (TUG), which assesses mobility and balance, is a crucial example. This test was initially developed by Podsiadlo and Richardson in 1991 [1] on the original version of the test named the “Get-up and Go,” which aimed to clinically evaluate dynamic balance in elderly people as proposed by Mathias et al. [2]. This test was further successfully tested on patients with multiple disorders [3,4,5]. The TUG test highly correlates with age, socioeconomic status, and multiple comorbidities [6]. To obtain even more information from the TUG test, recent developments in movement analysis, like wearables and markerless motion capture systems, offer a fast way to gather detailed movement data [7,8]. These new technologies could improve how we conduct and interpret tests like TUG without slowing down the assessment process and potentially enhance our understanding and measurement of such tests without compromising evaluation time [9]. TUG test can be further processed into individual sub-phases, such as standing up from the chair, walking, turning around, walking back, and sitting down. In those sub-phases, parameters such as the speed and the quality of movement were previously evaluated [10,11,12,13,14,15,16,17].

Markerless motion caption that uses standard video to record movement without markers. Its progress is based on recent advancements in deep learning techniques that identify body segment positions and orientation [18]. The OpenCap [19] is a recent addition to the markerless motion capture technology field. It distinguished itself by its accessibility and ease of use for measuring kinematics in real-world settings [19]. Unique for requiring only two iPhones to operate, OpenCap simplifies capturing human movement, making it broadly applicable in various settings. While it is a relatively new technology and not necessarily a major advancement, its accessibility opens up possibilities for broader applications, particularly in environments where traditional, more complex systems are not feasible. OpenCap’s potential in clinical and research contexts, especially for assessments like the TUG test, is currently being explored. Studies focus on its capabilities and limitations in accurately capturing and analyzing gait and mobility [20,21,22], underlining its emerging role in efficient and practical movement analysis.

The TUG test encompasses a range of activities, including standing from a seated position, walking, turning, and sitting down. Accurate recording of these varied movements is imperative for applying markerless systems within clinical environments.

Our study aims to comprehensively compare the system’s precision in capturing gait from various orientations, specifically comparing data when walking towards (WF – walk forward) and away from the cameras (RW - return walk), critically evaluating kinematic and spatial-temporal parameters. This examination is vital to thoroughly assess the OpenCap system’s full potential, focusing on its application in rapid evaluation tests such as the TUG test, shedding light on how much its ease of use and accessibility correspond with its accuracy and dependability in practical clinical settings.

2. Materials and Methods

Participants

Our study involved 9 healthy participants and one with hip osteoarthritis (left side), including 6 men. The average age of the participants was 29.7±8.6 (Maximum: 51, Minimum: 21) years, weight: 74±13 kg (Maximum: 90 kg, Minimum: 49 kg), height 176.6±11.5 cm (Maximum: 190 cm, Minimum: 160 cm ), BMI: 23.5±2 (Maximum: 25.4, Minimum: 18.9). All participants gave their written consent to participate in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved within a larger project by the Ethics Committee of University Hospital Bratislava (Approval number: 07/2020).

Measurement Setup

Our experimental setup took place in the gait laboratory at the Orthopaedics Hospital Speising in Vienna. In addition to capturing the TUG test, we separately recorded walking movements towards (WF) and away from the iPhone cameras (RW), which we focus on in this study. 5-7 recordings were conducted for each participant for each walking direction. Our methodology involved a dual-system approach, utilizing marker-based and markerless camera systems. For the markerless system, we set up the OpenCap system according to the recommended guidelines [23] with two iPhone cameras, an iPhone 12 and an iPhone 14. These were positioned on a tripod at the height of 1.5 meters, angled downwards by 5°. The iPhones were arranged to ensure that the center of the TUG area was squarely in the middle of both cameras’ capture zones. Additionally, the cameras were angled at 35° toward the center of the walking area (Figure 1). The starting position for the WF was initiated from a stationary stance at a distance of around 5.7 meters from the camera. Conversely, in RW, the initiation point was set at 2.7 meters from the camera. The recording stopped when the subject exited the recording area of the OpenCap system. OpenCap recorded the videos with default settings utilizing the OpenPose estimation algorithm, with a resolution of 720x1280 pixels and a frame rate of 60Hz. OpenCap’s embedded musculoskeletal model was from Lai et al. [24] and Rajagopal et al. [25], with modified hip abductor muscle paths, according to Uhlrich et al. [19]. The OpenCap version used was from November to December 2023. An optoelectronic motion capture system comprising 17 cameras (VICON, Oxford, UK) was employed for the marker-based measurement. Modified marker sets, specifically the Cleveland Clinical Marker Set (for the lower extremity) and the PlugIn Gait Model (upper extremity) [26], incorporating a total of markers, were utilized. The recording frequency was set at 150 Hz. Within the Nexus software (Version 2.15, Vicon, Oxford, UK), markers underwent reconstruction, filtering (Woltring Filter, Mean Squared Error [MSE], smoothing at 15 units), and subsequent storage. Notably, a seasoned user manually designated the events of Initial Contact (IC) and toe-off. The marker trajectories were then used to run OpenSim’s [27] inverse-kinematic tool with a musculoskeletal model from Rajagopal et al. [25].

Data Analysis

Data processing and statistical analysis were done using Matlab R2022b. The OpenCap data were processed using OpenCap’s web application [19]. First, we adjusted the signal from markerless data to match the marker-based system’s frequency by interpolating the data to 150 Hz and applying a Butterworth low-pass filter of 10 Hz. Data synchronization was achieved by aligning the peak flexion of the right knee, followed by cross-correlation to adjust the entire signal. Every measurement was then individually checked and corrected if needed. After the synchronization, we cut the signal from either system to obtain the same gait cycles from both systems. From each walking recording, we eliminated the first step. We normalized all joint kinematic variables of both systems to match the 100% gait cycle (GC).

We employed the Root Mean Square Error (RMSE) and Statistical Parametric Mapping (spm1d [28]) analysis for each joint kinematic variable, walking direction, and utilized system. We then extracted the kinematic and spatial-temporal parameters, averaged across the participants for the left and right sides. We utilized the Spearman correlation test in spatial-temporal and kinematic parameters across the two systems to assert the correlation.

3. Results

3.1. Comparing Walking Directions in the Markerless System

The comparison of averaged RMSE for left and right legs using markerless and marker-based systems reveals significant differences in kinematic analysis (Table 1). The markerless system shows a grand mean RMSE of 4.48±2.85°, while the marker-based system has a notably lower mean of 0.98±0.79°. Differences between the two systems are particularly notable in pelvic obliquity (5.8±2.0° vs. 0.3±0.2°), hip flexion (4.6±2.9° vs. 1.7±1.4°), hip abduction (6.6±3.9° vs 0.63±0.47°), hip rotation (3.6±2.5° vs 1.02±0.72°), knee flexion (3.6±2.5° vs. 1.3±1.2°), and ankle flexion (8.5±4.6° vs. 0.8±0.82°).

The analysis further details the RMSE for WF and RW directions between the two systems, emphasizing greater discrepancies in RW (grand mean 6.56±4.64° vs 4.88±3.3°), especially in pelvic obliquity(5.3±3.2° vs 2.8 ±2.1°), hip abduction (6.1± 4.2 °vs 3.5±2.5°), and ankle flexion (11.9±6.4° vs 4.7±3.4°). Table A1 demonstrates notable discrepancies in spatial-temporal and kinematic parameters between directions in the markerless system, with most variables showing differences greater than 3°. In contrast, Table A2, detailing the marker-based system, reveals smaller variations, with differences under 2°. Additionally, the markerless system lacks significant correlation across 18 kinematic parameters for both directions.

This difference is confirmed by observed kinematic analysis between WF and RW in the markerless system, as shown in the SPM1d analysis in Figure 1 for the right side (Figure A1 for the left side). We do not observe such differences in the marker-based system (Figure A2 and Figure A3).

Figure 2. Kinematic comparison of the right GC between FW (blue line) and RW (red line) using the markerless system. Root Mean Square Error (RMSE) is shown as a red dotted line, and statistically significant differences, indicated by black bars beneath the respective kinematic signals, are determined by statistical parametric mapping (SPM1d) analysis at a significance level of p < 0.05.

Figure 2. Kinematic comparison of the right GC between FW (blue line) and RW (red line) using the markerless system. Root Mean Square Error (RMSE) is shown as a red dotted line, and statistically significant differences, indicated by black bars beneath the respective kinematic signals, are determined by statistical parametric mapping (SPM1d) analysis at a significance level of p < 0.05.

3.2. Comparing the Markerless against the Marker-Based System

The kinematic plot with SPM analysis and RMSE conducted between the markerless and marker-based systems in WF revealed notable distinctions in the gait cycle’s dynamics, which are visible in Figure 3. For the right GC, we observed disparities in different phases of GC in the pelvic list, hip abduction, hip rotation, knee flexion, and ankle flexion. On the left side in Figure A4, we see a similar trend.

Additionally, in Table 2, when performing Spearman correlation of calculated kinematic variables between the marker-based system and markerless system for WF, we have only found significant correlations for both sides in only 2 out of 23 kinematic extracted parameters: hip flexion range of motion and hip rotation at initial contact. Some kinematic parameters show significant correlation only on one side: mean hip rotation during the stance phase, knee flexion range of motion, maximal plantar flexion, and angle lift-off. When evaluating the spatial-temporal parameters, the Spearman correlation in Table 2 showed a significant association for both legs, namely stride length (r=1; p < 0.001), step length (r=0.98; p < 0.001), and gait speed (r=1; p < 0.001). We also see a significant correlation with step width (0.94; p<0.001).

Similar, although slightly worse, results are visible for the RW in Figure A5 and Figure A6, where the comparison reveals different propagation of kinematics in the pelvic list, hip abduction, knee flexion, and ankle flexion. Compared to WF, the differences in RW seem to be more exaggerated, which is confirmed in Table A3, where the difference in hip obliquity ROM > 9° (<2° in WF), hip abduction ROM > 7° (<3° in WF), knee flexion ROM > 6° (<3° in WF), knee flexion at IC > 6° (3° in WF). We do not see a significant correlation for any kinematic parameters. Spatial-temporal parameters show significant correlations, similar to WF.

Figure 3. Kinematic analysis comparing WF for the right GC using markerless (red line) and marker-based (blue line) systems, with the Root Mean Square Error (RMSE) shown as a red dotted line, and statistically significant differences, indicated by black bars beneath the respective kinematic signals, are determined by statistical parametric mapping (SPM1d) analysis at a significance level of p < 0.05.

Figure 3. Kinematic analysis comparing WF for the right GC using markerless (red line) and marker-based (blue line) systems, with the Root Mean Square Error (RMSE) shown as a red dotted line, and statistically significant differences, indicated by black bars beneath the respective kinematic signals, are determined by statistical parametric mapping (SPM1d) analysis at a significance level of p < 0.05.

3.3. Case section

In our comparative analysis of pelvis tracking data between marker-based and markerless systems, notable differences emerged in the standard deviations (STD) of the measurements. Specifically, the captured mean pelvic tilt in Table 2 displayed a larger standard deviation (7.2°) in the marker-based system, signifying a wider variability in pelvic movement. In contrast, data obtained from the markerless system demonstrated a smaller standard deviation in the mean pelvic tilt (2.4°), indicating more consistent pelvis movement data with less variability. While we see a wider range of pelvis tilt captured by the marker-based system (Figure 4), we do not observe the expected correlated values in the markerless system (the same is true for the left side). We will explore these findings through two case presentations.

3.3.1. Visible Lordosis

The first case illustrated in Figure 5 presents a participant with visible anterior pelvic tilt. The marker-based analysis, represented by the blue line in Figure 5, indicated a pelvic tilt of around 10 degrees visible over the whole gait cycle. Despite the apparent similarity in movement patterns, the markerless system fails to accurately capture anterior pelvic tilt, recording a value approximating 0°.

3.3.2. Pelvic Movement in a Participant with Hip Osteoarthritis

The second case presents the participant with advanced left hip osteoarthritis. His gait is not symmetrical, and he has reduced hip extension and compensatory movement in the pelvis. Looking at the pelvic movement from the markerless system in Figure 6, there is less anterior tilt of the pelvis at the end of the single support phase than in the marker-based data. The marker-based system shows less hip extension in the terminal stance phase. The markerless system presents a normal hip extension value. The ROM in the frontal plane in the pelvis and hip is greater with the markerless system, whereas the ROM is smaller in the transverse plane. From a clinical point of view, we would expect a reduced extension in the terminal stance phase for the participant, as seen by the marker-based system.

4. Discussion

The objective of the study was to assess and compare the outcomes of the markerless system while walking toward and away from the iPhone cameras. Specifically, we aimed to ensure uniformity in temporal-spatial parameters and kinematic results across both walking directions. The benchmark for this comparison was established using a marker-based system, where the kinematic data for both walking directions was anticipated to exhibit consistency. We conducted a comparative analysis of the directional movement results to gauge the concordance of data between the two systems.

The two walking directions in the TUG test, as shown by Bonnyaud et al. [10,11], do not present significant differences in the kinematic parameters for the hip, knee, and ankle in healthy and hemiparesis due to stroke. Due to the aim that we want to assess the accuracy of the markerless system to analyze the TUG test, the measurement system should be able to measure the same results between the FW and RW. First, we compared differences across all participants in the different walking directions. Overall, the RMSE between the two directions in the markerless system showed a mean difference of 4.47° across all the joints. Among those, we observed a clinically significant difference (>5 degrees) in pelvic obliquity: 5.75±3.26°, hip abduction: 6.6±3.92°, and ankle flexion: 8.51±4.59°. This high difference cannot be attributed to only different walking patterns of participants because the same measurement by the marker-based system recorded a grand mean RMSE of only 0,96°, and none of the joints presented an error higher than 2°. SPM analysis in the kinematics of joints highlighted these differences in the markerless system, showing significant differences in pelvic obliquity, hip abduction, hip flexion, knee flexion, and ankle flexion. When we compared the same gait cycles between the markerless and marker-based systems, we found lower RMSE values in WF, particularly in the pelvic obliquity (2.8° to 5.3° respectively), pelvic rotation ( 2.9° to 3.8° respectively), hip abduction (3.5° to 6.1° respectively), and ankle flexion (4.7° to 11.9° respectively). Max RMSE errors were also lower in WF. According to previous findings [18,29,30], there is a bias in pose estimation algorithms regarding the position and distances of the used cameras and currently available open-source training datasets that were never designed with biomechanical applications in mind. We can conclude that the errors observed in the markerless system related to directional differences are primarily due to less effective tracking when participants have their backs turned toward the iPhone cameras and walk away from the cameras.

We have further evaluated the markerless system compared to the marker-based system in WF’s kinematics and spatial-temporal parameters. The grand mean RMSE was 4.87, similar to the results from Horsak et al. [20], which was 4.61° without the subtalar joint (Horsak et al., 2023). The pelvis tilt and hip flexion in our measurements had slightly worse results. We recorded higher RMSE in the pelvic tilt at 8.94° (compared to 6.6°) and hip flexion at 5.4° (compared to 4.2°). However, the sole comparison of the RMSE does not tell us how the markerless system detects variability in different kinematic motions. We calculated a correlation between the markerless and the marker-based system for kinematic and spatial metrics. We found an excellent correlation in spatial-temporal metrics, especially measuring stride length, which was almost identical to the marker-based system (r=1; p<0.001) and a good correlation with the step width (r=0.94; p<0.001). When evaluating the kinematic parameters in Table 2, most of the parameters failed to record a significant correlation, meaning there could be a problem with how the kinematic curves change relative to individual participants.

These observations, notable the differences in the standard deviations (STD) between the markerless and marker-based system in the recording of the pelvic tilt, led us to examine the difference in the individual cases further. In the first case, a detailed examination of pelvic tilt revealed markerless’s problem detecting lordosis. We observed the markerless system’s tendency to report pelvic tilt measurements close to zero degrees, indicating a bias towards a neutral pelvic position. This standardization effect persisted even in case of significant lordosis, implying that the markerless system might not accurately represent deviations from neutral pelvic tilt. Given the coordinate definitions used in OpenSim calculations, while some underestimation compared to actual pelvic tilt is expected, a value around 0° suggests a considerable underrepresentation of actual pelvic tilt. The second case revealed markerless’s problem in capturing kinematic motion in a participant with advanced left hip osteoarthritis. This points to the markerless system’s lack of sensitivity in capturing nuanced gait dynamics, a critical aspect often necessary for clinical assessments.

It is crucial to acknowledge and address certain limitations, including small sample sizes, manual detection of gait events, different filtering, and the challenges in data synchronization. These factors could affect the precision of the analysis. Despite these limitations, our findings reveal robust differences. Addressing these issues would likely refine our results but not diminish the clear distinctions we have observed in the data.

5. Conclusions

In conclusion, while the OpenCap system offers promising accuracy in capturing spatial-temporal gait parameters such as step length and step width, its current limitations in providing comprehensive kinematic analysis cannot be overlooked. The comparison with the marker-based system has underscored these challenges, particularly in the variability based on walking direction and the challenges in the measurement of kinematics. For studies where pelvic movement is a key parameter, using markerless capture requires caution. WF and RW should be addressed when choosing the proposed camera setup for measuring the TUG test. Future research should focus on overcoming these limitations, potentially through improvement in pose estimation algorithms to enhance the reliability and applicability of the markerless systems in clinical gait analysis. Currently, marker-based systems have a clear advantage over markerless systems’ ability to consistently capture kinematic data.