Development of an Immersive Virtual Reality System during Robot Assisted Gait Training for Individuals with Neurological Diseases: Feasibility and Proof of Concept Study

1. Introduction

The recovery of functional gait is of great importance for neurological patients and is a key goal of their rehabilitation program [1]. In recent years, the field of neurorehabilitation has shown a growing interest in advanced technologies, like robotics and virtual reality (VR), due to their potential and multipurpose application to patients’ recovery pathways [1,2,3]. For example, robot-assisted gait training (RAGT) is a safe form of treadmill training that reduces physical demands on therapists and allows highly repetitive gait training early in the rehabilitation process [4,5]. In RAGT, depending on the applied technology, different training parameters, such as treadmill speed, amount of body weight support, or amount of support to leg motion, can be adjusted to create an optimized training intensity [6]. Notably, systematic reviews and meta-analyses have reported meaningful effects in different acute and chronic neurological conditions [7], including stroke [8,9], traumatic brain injury [10], spinal cord injury (SCI) [11], multiple sclerosis [12] and Parkinson’s Disease [13], corroborating the added value of integrating robotic technologies in standard rehabilitation programs across clinical populations. Feedback and gamification are used to enhance engagement and motivation of patients undergoing such training [14,15]. Currently, such “games” are usually based on simple tasks executed on a computer screen and thus partially disconnected from everyday gestures and movements, and therefore different from the natural experience of walking in an environment. In contrast, task-specific and context-specific training conditions have been proven to be key features for an optimal transfer of acquired motor skills to real life [16].

Researchers have shown that VR-based training could be an effective tool for both assessing and aiding in motor and cognitive rehabilitation [14,17,18]. VR is defined by three primary elements: immersion, sense of presence, and interactivity. Immersion measures how fully a virtual environment (VE) can deliver multisensory feedback coordinated with user interaction. This immersive experience, coupled with VE interactivity, shapes the user's perception, influencing their feeling of being present in the VE. Depending on how much they counter real-world sensations, VR systems are classified as either fully-immersive, semi-immersive, or non-immersive. Fully immersive VR tools include head-mounted displays (HMD) and cave automatic virtual environments (CAVE), while semi-immersive tools use large screens or projectors for moderate immersion, and non-immersive devices range from PCs with standard monitors to tablets. The application of various VEs applying different levels of immersion has allowed VR to offer diverse stimulation in the context of movement training and has improved opportunities for repetitive tasks, feedback provision, and sustained motivation, enhancing sensory inputs and facilitating motor learning [19,20]. Additionally, it has been suggested that VR promotes deeper engagement that can lead to synaptic changes at the cortical and subcortical levels, which are crucial for motor relearning [21].

A fundamental aspect that needs to be considered while developing a VE system, is the possibility of cybersickness which may to some degree be unavoidable in the use of VR [22]. When visual response processing for user input interaction is delayed, the signals from the eyes, the vestibular system of the inner ear and proprioceptive receptors to the brain conflict with each other, and motion sickness may be triggered [23]. Therefore, when designing immersive VR scenes and animations, the experience of cybersickness must be reduced according to the influencing factors of cybersickness [24]: (1) the user actively controls his/her viewpoint and adjusts it; (2) avoid or limit linear or angular acceleration or deceleration to reduce stimulation of the vestibular organs; (3) display visual indicators or movement tracks to users; and (4) dynamically blur the unimportant areas in the field of view.

While fully immersive VR presents a novel method to engage and motivate patients during RAGT, evidence of its feasibility and effectiveness remains limited [25,26]. To the best of our knowledge, among the few studies that have investigated the feasibility and acceptance of fully immersive VR system in neurological disorders, no study has focused on the use of an HMD in synchronization with a RAGT device. Thus, the primary aim of this study was to investigate the feasibility of an innovative, fully immersive VR system (applying an HMD) during RAGT in subjects suffering from neurological disorders. The secondary aim was to assess the acceptance of the approach in a clinical environment.

We first hypothesized that fully immersive VR during RAGT is feasible for subjects suffering from neurological disorders. Additionally, we expected that the innovative integration approach would find a positive acceptance among both participants and therapists.

2. Materials and Methods

2.1. Study Design and Setting

This feasibility study was conducted between January 2022 and July 2022. All study participants were recruited at the University Rehabilitation Institute, Republic of Slovenia.

The study was approved by the Slovenian National Medical Ethics Committee (0120-548/2021/3) and was carried out in accordance with tenets of the Helsinki Declaration. All participants gave their informed, written consent to participate in the study.

2.2. Participants and Recruitment

Subjects suffering from cerebral stroke or with spinal cord injury (SCI) were included in this study. All subjects received conventional treatment over a period of three weeks, three to five sessions weekly, and the course of this treatment they were invited to test the fully immersive VR system in single session combining it with RAGT. Before starting the session, participants were already familiar with the Lokomat training by having at least three sessions before using fully immersive VR system. Inclusion criteria were: (1) aged between 20 and 60 years; (2) ability to move lower extremities; (3) partial or impaired ability to walk with or without medical walking aids; (4) ability to participate in the study; (5) cognitive abilities to understand and follow simple verbal instructions. In addition, the Lokomat-specific inclusion criteria were also considered: (1) body weight <130 kg; (2) body height < 200 cm; (3) no other associated neurological dysfunction such as reduced sensation and contractures or cardiovascular diseases. Exclusion criteria were: (1) considerably reduced bone density (osteopenia or osteoporosis); (2) leg length difference 2 cm; (3) non-consolidated fractures; (4) restricted range of motion in lower extremities, (5) excessive spasticity, or contractures of lower extremities; (6) presence of disabling sensory alterations, including hallucinations; (7) concomitant medical and psychiatric illness possibly interfering with the robotic/VR training. These conditions were judged by the therapist supervising the study, for the individual patients. Fifteen participants were finally included (see Table 3).

2.3. Experimental Apparatus and Control

2.3.1. Robot Assisted Gait Training Device

The RAGT device used in this study was the LokomatPro V6 FreeD (Hocoma AG, Volketswil, Switzerland). The Lokomat is a treadmill-based robotic exoskeleton with actuated hip and knee joints and a dynamic body weight support system that supports the patient through a harness [27] (see Figure 1). The orthosis is programmed to follow a predefined gait pattern with an impedance control strategy, of which the rigidity/stiffness can be adjusted. In standard therapy, the gait pattern and the mechanical impedance of the robotic hip and knee joints (called “Guidance Force” in the Lokomat software) are manually adjusted by the therapist through a user interface. The Guidance Force is constant throughout the gait cycle.

2.3.2. Immersive Virtual Environment

The VR environment was developed with the Unity game engine Unity Technologies, San Francisco, US) and the plugin FMETP STREAM (Frozen Mist) that allows live video streaming and cross-platform network communication. The fully immersive VR was provided by the HMD Meta Quest 2 (Oculus by Facebook, Inc. - Meta Technologies, LLC) [28]. This device was selected in line with the argumentation in [29] where it is considered suitable for the context of neurorehabilitation.

The fully immersive VR environment consists of a light house park and two companions were created (see Figure 2).

The fully immersive VR system was integrated in the Lokomat Therapist Interface (Loko-control) in a similar style as other therapist control functions (see Figure 3a). The patient’s screen of the Lokomat streamed and displayed the same VE as displayed in the HMD in real time (see Figure 3b) so that the therapist can see what the patients sees and looks at, including the hands of the patients if in view. In addition, instructions or performance data were also displayed to the therapist.

The VR environment consists of an island with a lighthouse, trees, green grass, mountains, bridges and some landmarks in low-poly stylized graphics. Matching ambient sound of twittering birds and wind create a relaxing nature atmosphere. Multiple paths lead to different areas of the island, allowing exploration of the whole island, although movement is limited to the paths. After starting of all required applications, a wireless connection is established between the fully immersive VR headset and a router. The router connects to the Lokomat with a wired connection. Once the connection between the immersive VR headset and the Lokomat has been created in this way, the immersive VR headset and the Lokomat are exchanging measured signals, parameters and video streams. The patient is represented in virtual reality by an avatar human body, however instead of arms, just hand representations are visible and arms are missing from the shoulders down. The patient’s hand movements are tracked by the Meta Quest 2’s hand tracking capabilities and applied to the virtual hands. Orthosis leg angles measured by the Lokomat are sent to the fully immersive VR headset through the wireless network connection and mapped to the legs of the patient’s avatar. The presence of an avatar body with legs, and the first-person view that places the camera at the head position of the virtual avatar, lets the patient see the leg movements on the avatar when looking down. This creates the illusion that they walk in the virtual environment by themselves and what they see is their own legs and their movement. The avatar’s leg movements are used for forward motion of the avatar on the predefined paths in the virtual world. At junctions, the desired path can be chosen by looking at the signpost of the preferred direction. For this, the orientation of the immersive VR headset is used, there is no eye tracking. The mechanism of looking at something for selection is further used to collect objects along the path, which is one of the two possible tasks the therapist can choose for the patient. The other task uses the patient activity measurement calculated by the Lokomat using its force sensors, which measures the amount of effort that is contributed to the orthosis movement by the patient. For a given consistent high value of patient activity, the virtual environment changes its coloring to brighter, more saturated colors, adds flowers to the path in front of the avatar, and layers of music are added. With decreased patient activity over some time, those rewarding additions are getting slowly removed again. While exploring the island, the patient’s avatar can be accompanied by one or two companions. The choice can be made by the therapist in the Lokocontrol user interface, and they can choose to show no companion, both companions or either one of them. The companion options consist of a female human or a dog. The human companion’s leg movements are synchronized with the Lokomat’s orthosis movement and therefore also with the patient’s and the avatar’s leg movements (idea originating from [30]). To mitigate the risk of motion sickness associated with navigating the default curved paths, there is the possibility to traverse a straight path across the island. This is a setting that can be selected by therapists as an introductory option when introducing immersive virtual reality to new users or for individuals who are more susceptible to motion sickness.

Video S1: Integration of immersive virtual reality into gait rehabilitation robotics (Loko-VR).

2.4. Treatment Procedures

The participants tested the fully immersive VR system in a single session. During a Lokomat gait training session, the participants wore the HMD (see Figure 3) in order to immerse them in a VE, through the application as described in section 2.3 (see Figure 4). The Lokomat session lasted between 30 to 45 minutes with intermediate rests as tolerated by the patient and decided by the therapist in charge. Walking speed was progressively increased to a comfortable level. The sessions were provided by qualified therapists who operate the Lokomat system on a daily basis. The safety in the system was assured through established Lokomat product safety mechanisms, while wearing an HMD was supposed to minimally affect safety, while the sessions were closely observed by an additional technically educated observer, who could stop the device in case of observed risks or danger. Also, for the therapist an emergency stop button is always within reach, according to the design of the Lokomat for normal operation.

2.5. Assessment Procedures

2.5.1. Sociodemographic and Clinical Variables

A physician completed a report documenting the clinical features of the participants: type of neurological disorder (Cerebral Stroke or Spinal Cord Injury), affected side, onset of illness, age, height, weight, gender, Functional Ambulation Category (FAC) score [31] and Mini Mental State Examination (MMSE) score [32]. After the completed session, the participant and the therapist filled out the questionnaires informative on their acceptance of the session.

2.5.2. Feasibility and Session Adherence

The feasibility of the setting was assessed by collecting the following training parameters during the sessions: level of the Guidance Force (GF) (%left / %right) - a standard Lokomat setting on the amount of provided gait support, walking speed (km/h), body weight support (BWS) (% of body weight) and fully immersive VR session duration (min). The safety indicators were any occurrence of adverse events such as pain, motion sickness, dizziness, exertion, fatigue, and headaches as well as number of withdrawals (n). Common Terminology Criteria for Adverse Events (CTCAE) version 3.0 was used to assess the adverse effects of exercise intervention [33].

2.5.3. Acceptance

The therapists responsible for patient treatment made a first selection among their Lokomat patients on who they considered suitable to train with a HMD, considering the formulated inclusion criteria. We determined the prospective acceptability by the proportion of patients approached that accepted an offer to participate in the “fully immersive VR” session [28,34]. Reasons for declining the intervention were collected.

The acceptance was also investigated by means of questionnaires developed by the Authors based on the Technology Acceptance Model (TAM): one for the participants and one for the therapists. The first, patient questionnaire is composed by 18 closed-ended questions where the patients had to respond by means of a 5-point Likert scale (1: strongly Disagree; 5: Strongly Agree) [35] and 4 open-ended questions (see Table 1).

The second, therapist questionnaire is composed by 11 closed-ended questions where the therapists had to respond by means of a 5-point Likert scale (1: strongly Disagree; 5: Strongly Agree) [35] and 2 open-ended questions (see Table 2).

2.6. Data Collection and Management

Data were ordered and collected in an electronic sheet and stored securely in a database at the rehabilitation center. For the feasibility measures, descriptive statistics for the level of the Lokomat Guidance Force, walking speed, body weight support and fully immersive VR session duration were reported. For safety indicators, any occurrences of pain, fatigue, dizziness, or possible falls were reported. The number of dropouts was also documented. Questionnaire data were processed and the results were analyzed descriptively (mean value, standard deviation).

3. Results

In the following, we report the results of the feasibility study of the fully immersive, HMD-based VR system during RAGT that we performed in a cohort of 15 patients. Each patient was evaluated after performing a single session.

3.1. Participants

Fifteen subjects (months after diagnosis: 9.2±11.21, age: 54.3±11.97 years, height: 177.7±9.97 cm, weight: 84.8±16.49 kg, FAC: 3.5±1.46,) participated in a single session during their stay in the rehabilitation center. Five of the participants suffering from SCI and ten from stroke (seven with right body-side affected). Table 3 shows demographic and clinical features of the participants.

Although not monitored in the study, the involved therapists made the observation that around 20% subjects with stroke, and around 50% of subjects with SCI that were (or normally would be) included for Lokomat treatment, were by their responsible therapist considered to be suitable to use an HMD and invited to participate in this study. The difference between the stroke and SCI groups were related to occurrence of cognitive and perceptual limitations.

Table 4. shows the cognitive and visual status of the participants suffering from stroke which were included in the study. Participants suffering from spinal cord injuries did not have any cognitive problems therefore they did not perform the MMSE.

3.1. Feasibility and Session Adherence

As reported in Table 5, all fifteen participants performed the training session, according to personalized settings (GF: 96±7.37, walking speed: 1.40±0.19 BWS: 43.3±7.83 and fully immersive VR session duration: 20.9±6.76); one of the participants had to prematurely stop the session. No musculoskeletal events of CTCAE version 3.0 > Grade 2 occurred and no significant adverse events or side effects were reported during any session.

3.2. Acceptance

Fifteen of the invited participants accepted the intervention and fourteen concluded the fully immersive VR session during LokomatPro training, following the judgement of their therapist. One subject suffering from stroke presented concentration disorders, neglect and hemianopsia and was not able to properly cooperate with the VR game, and the session was prematurely interrupted. In this study, the results of the questionnaires are summarized in Table 6 and Table 7 , where median and mean scores are presented (according to the Likert scale; 1: Strongly Disagree; 5: Strongly Agree). Generally, these results demonstrated overall good acceptance among both participants and therapists.

Figure 5 represent boxplots of the participants’ replies, where the central mark (circle with dot) indicates the median score, and the left and right edges of the box indicate the 25th and 75th percentiles, respectively.

Open-ended replies provided by the participants after the session confirmed that the fully immersive VR session was considered acceptable (Table 8). All participants liked the virtual environment: “like the environment, nature, which relaxes”; “it felt positive”. Participants enjoyed to walk with a companion and expressed willingness to be more challenged: “to go uphill to mountain hiking”; “there could be some walking obstacles in the walking path”. A number of participants also provided valuable insights for future iterations of the fully immersive VR.

Open-ended replies provided by the therapists after the session also confirmed that the fully immersive VR session was considered acceptable and usable (Table 9). Therapist noted that the approach was easy to use, “the whole installation is fairly easy to use”, and participants were motivated “the patients had more motivation for training - especially those, who had several Lokomat trainings before participation”. Furthermore, therapists provided also valuable insights for future iterations of the fully immersive VR.

3.3. Further Observations

The involved therapists further reported that they always offered their patients the opportunity to take a break in between the single bouts of the ‘virtual walk’, that each lasted around 10 minutes to complete, but they always preferred to continue, usually up to performing three walks in a row.

Although this was anticipated by some therapists, none of the patients or other users complained about neck or back pain during or following the sessions as an effect of the use of the HMD.

4. Discussion

The results of this feasibility study suggest that using the innovative fully immersive, HMD-based VR system in combination with RAGT is feasible for subjects suffering from neurological disorders, with respect to safety, adherence and tolerance, within certain inclusion criteria. Furthermore, the very positive overall acceptance of the immersive VR system among both participants and therapists suggests that the immersive VR system has potential to be included as an optional feature in a regular LokomatPro session, to the benefit of the patients. Taken together, our results motivate further studies to explore and validate the clinical efficacy of such a combination, or extension of regular Lokomat training. As with all pilot studies with a limited sample size, these results should be interpreted with caution and may not be generalizable to the stroke and SCI population at large.

The novelty of this study is the implementation of an innovative fully immersive VR environment integrated and synchronized during a RAGT session. The successful application of a fully immersive VR system may lay the groundwork for additional treatment options for neurorehabilitation [18,36,37]. Fully immersive VR platforms, in particular HMDs, provide a richness of experience and credibility that cannot be achieved by semi-immersive VR modalities. They also enable the provision of treatment in alternative environments that aid relaxation, more closely approximate the real environment, or build safety awareness without jeopardizing patient safety. Fully immersive VR also provides the unique benefit of occluding the hospital or clinic environment, minimizing auditory and visual distractions while highlighting preferred stimuli (in this case, the affected extremity) [38]. It should be stressed that fully immersive VR has several requirements for motor and cognitive rehabilitation interventions: repetitive practice, feedback about performance, multimodal stimulation, and controlled, safe, and ecologically valid environments [39].

Over the past decade, technology-based exercise interventions have gained wide interest worldwide. VR, both immersive and non-immersive, has become a mature technology and it is currently experiencing an increasing adoption for clinical research, psychological interventions and cognitive studies [18,40,41,42,43]. Specifically, the use of VR as a physical therapy tool in neurorehabilitation was recently subject of analysis in a systematic review and the conclusion from the authors was that VR induced changes in neural plasticity for survivors of stroke and optimize functional recovery [44]. In addition, the authors highlighted that compared with regular computer monitors, participants in the immersive VR system reported more presence and a better learning experience.

4.1. Feasibility

The main objective of this study was to evaluate the clinical feasibility of a novel immersive VR integration approach during RAGT for individuals with neurological disorders. Participants demonstrated a 93% adherence to the protocol, surpassing results from other studies on VR feasibility [43,44]. It is important to note that therapists had pre-selected participants based on specific inclusion criteria.

No adverse effects were reported during the VR sessions; however, one participant discontinued due to cognitive impairments, affirming the safety of the system and intervention method. Many clinical trials have shied away from using head-mounted displays and immersive VR due to concerns over safety and cybersickness [25]. This study's fully immersive VR application was developed with these issues in mind, using a commercial HMD. Consumer-grade fully-immersive VR technologies have advanced considerably over the past five years [37]. These affordable technologies could be highly beneficial in real clinical environments [45,46]. Accessible hardware and open-source software enable the development of new VR applications. These factors help address previous challenges, particularly immersion levels and user interaction, including the risk of cybersickness. The study's findings indicate that innovative fully immersive VR can be safely combined with a robot-assisted gait device like the LokomatPro.

4.2. Acceptance

The secondary objective was to evaluate the acceptance of the approach within a clinical setting by both participants and therapists. The acceptability of immersive VR in neurorehabilitation is a critical issue that has not been extensively explored.

Overall, the fully immersive VR session during RAGT was widely accepted. Most participants found the intervention suitable for their needs and expressed a willingness to undergo it again in the future. Many participants described the fully immersive VR experience as positive and enjoyable. Participants reported high scores in terms of enjoyability, challenge, and effectiveness of the immersive VR experience. Common reasons for enjoying the experience included the sense of immersion, physical activity, and the novelty of the experience. In this way, immersive VR may address patients' unmet need to virtually "escape" the hospital environment and experience some degree of normalcy during training.

Therapists also found the fully immersive VR session acceptable and usable. They regarded the integration of the VR system during RAGT as a valid and reliable tool for enhancing participant engagement. These qualitative findings were further bolstered by most participants' endorsement that they would use VR again if given the opportunity. Additionally, the qualitative results identified areas for usability improvements. Participants provided feedback that could guide future iterations, such as incorporating more variations of immersive VR tasks to maintain motivation during RAGT.

4.3. Future Directions

This study demonstrated the feasibility and acceptance of a fully immersive VR, HMD-based system employed during RAGT among participants with neurological disorders and their therapists. Our feasibility study produced promising outcomes for utilizing VR in gait rehabilitation.

Properly designed VR applications enable active patient engagement in treatment, promoting a patient-centered approach. While VR cannot replace real experiences and task-specific training, it may enhance adherence to treatment. Emerging evidence indicates that combining physical and cognitive factors is crucial for recovery after neurological disorders [26,47,48,49]. For instance, VR has been shown to be a potentially valuable tool in motor assessment and rehabilitation during motor rehabilitation [50,51]. VR-based training offers increased opportunities for repetitive practice, feedback, and motivation for sustained practice, by applying visual, auditory, and tactile inputs, to enhance motor learning. However, potential risks, such as specific or long-term effects of this training modality on motor and cognitive aspects, need further examination and cannot be inferred from a short-term feasibility study.

There is a need for high-quality randomized controlled trials (RCTs) with larger sample sizes and standardized outcome measures for common functional outcomes to improve the generalizability of results and better understand the impact on treatment outcomes.

4.4. Limitations

While this study presents innovative and valuable findings, it is important to consider several limitations when interpreting the results.

Firstly, as a feasibility study, there was no control group to compare these findings against. Future research should focus on conducting randomized, controlled trials to evaluate the acceptance of immersive VR for individuals with neurological disorders, and compare it with alternative training modalities.

Secondly, the study relied solely on questionnaires to gauge participant and therapist acceptance. In subsequent studies, especially once these technologies are integrated into clinical practice, quantitative methods should be employed to track changes in acceptance over time.

Lastly, this paper primarily focuses on the feasibility of the approach without assessing its therapeutic effects. Moving forward, a rigorous randomized controlled trial should be designed to evaluate the efficacy of a fully immersive VR system during RAGT, paying close attention to the practical application details.

5. Conclusions

This research showed that a fully immersive, HMD-based VR system is both feasible and well-received by stroke and SCI patients and their therapists during RAGT. The study suggests that such a VR system could be a viable alternative to the screen-based training games as currently used in neurorehabilitation. It may be especially suitable for enhancing patient motivation and adherence to training, particularly if the application is enjoyable and not mentally taxing. However, despite these positive findings, whether the benefits of immersive VR outweigh the risks of using an HMD should be evaluated on a case-by-case basis, considering the therapist's judgement and the patient's personal preferences.