STORMS: Occupational Telerehabilitation in Multiple Sclerosis

1. Introduction

The compromise of cognitive functions affects up to 70% of the population diagnosed with Multiple Sclerosis (MS) and involves specific deficits in domains rather than a uniform global cognitive decline. Patients with MS (PWMS) may have difficulties in information processing speed, attention, learning and episodic memory, executive functions, and visuospatial skills. Cognitive impairment (CI) can also occur in the early stages of the disease, and about half of the individuals with disability report minimal or mild cognitive difficulties within the first years after diagnosis. CI has significant consequences for everyday life and is the main cause of occupational disability and difficulties in activities of daily living.

Neurorehabilitation is becoming a therapeutic option for patients with multiple sclerosis, and there is increasing use of technology to make therapeutic programs more accessible, convenient, and suitable for self-administration. Furthermore, after the COVID-19 pandemic emergency, healthcare professionals are increasingly integrating tele-rehabilitation techniques to allow patients to participate in rehabilitation programs remotely. This approach not only provides a safe alternative but can also increase flexibility and accessibility for patients who may have difficulty attending rehabilitation sessions in person. The findings from recent reviews suggest that rehabilitative exercise can be viewed as a significant complementary therapy for MS [2,3]. The results of recent reviews indicate that rehabilitative exercise can be considered an important complementary therapy for MS, inducing neuroprotective phenomena [4]. The use of new technologies such as Virtual Reality (VR) and Exergames has emerged as a reinforcement tool for the rehabilitative treatment of people with MS [5]. Systematic reviews on VR in rehabilitation across various neurological conditions, including MS [6], suggest that VR serves as a motivating and engaging rehabilitation method, potentially enhancing therapeutic compliance [8]. Additionally, by allowing for the selection of different exercises and levels of complexity, VR can adapt to the wide variability in patients’ conditions and disease progression [9].

The use of markerless sensors has recently emerged as the most easily feasible approach that avoids the use of wearable systems and can be easily used for the assessment of disabilities and as a support for home rehabilitation by the patient in full autonomy [10,11]. In this context, the Microsoft Kinect sensor has proven to be reliable and effective in telerehabilitation applications also for PWMS [12].

In our previous work [7], we extensively described the STORMS project, funded by Merck through the Digital Innovation Award in Multiple Sclerosis. Based on the use of the ReMoVES system [11], it is implemented with the aim of serving as a starting point for the development of digital telerehabilitation solutions to support patients with multiple sclerosis, thereby improving their quality of life.

At IRCSS Policlinico San Martino 34 patients have used so far the System ReMoVES as a support to rehabilitation. In addition 20 MS patients have followed the STORMS project so far.

The main originalities and strengths of this work are:

• Implementation of an IoMT (Internet of Medical Things) system for the assessment and support of both in-hospital and in-home rehabilitation in people with MS.
• Utilization and rapid adaptation of the markerless/contactless ReMoVES system to the new target user group.
• Ability to exercise, monitor, evaluate, and analyze both motor aspects (such as upper limb, lower limb, trunk movement, and balance control) and cognitive aspects (such as attention, memory, working memory, etc.).
• Provision of a personalized service tailored to the needs of each individual, with an assigned individual care plan.
• Ease of use, low cost, and integrability with other systems.
• Robustness and resilience regarding temporary telecommunication problems.
• Adaptive nonlinear filtering and segmentation of signals for data extraction, analysis, and visualization.

After a brief summary of the exergames extensively described in [7], the study carried out is introduced, and some significant results are provided to demonstrate the ability of the system in observing the patient activity at home and their evolution and eventual progression.

2. Materials and Methods

Following a concise overview of the exergames proposed for motor/cognitive rehabilitation activities, the paper proceeds to elaborate on the two studies discussed therein.

2.1. Motor/Cognitive Exergames

As described in [7], some new cognitive games have been developed during the STORMS project with the aim to treat some of the most common symptoms of multiple sclerosis such as:

• Coordination disorders;
• Balance problems, and dizziness;
• Vision disturbances which may also include impaired color vision;
• Cognitive disorders that incorporate problems with memory and learning;
• Difficulties in maintaining concentration;
• Difficulties in attention and computational problems;
• Inability to perform operations of a certain complexity;
• Problems in correctly perceiving the environment.

All exergames are based on voluntary limb movements or balance shifts. During execution the patient requires coordination and performs corrective reactions. Motor and visual coordinations are very important for the correct execution of assigned tasks. Direct and indirect measurements of symptoms such as numbness of the body and/or extremities or spasticity that may complicate movement can be obtained when games are played with movement of the pelvis or limbs.

In some exergames the patient is encouraged in the reaching task by the appearance of consecutive targets on the screen which must be moved with the movement of the arm. The more targets that are taken, the higher the game score. Such games aim to improve hand-eye coordination and spatial awareness. The Shelf-Cans activity entails placing a colored can on the shelf where similar cans are already positioned. Similarly, in the Owl-Nest game, players are tasked with placing targets of varying difficulty levels into the basket, thus stimulating attention mechanisms.

Other exergames promote trunk balance used to guide an object such as a car or a hot-air balloon and can be executed also when sitting in a wheelchair.

The Owl Nest, Supermarket, Numbers, and Business By Car exercises offer varying levels of difficulty, detailed in Table 1, ranging from easy to highly challenging. On the other hand, Shelf Cans and Hot Air activities each have a single level, serving as introductory exercises for patients to become accustomed to the system.

In all the exergames, except Business By Car, the total game time is 60 seconds. Business By Car has a longer duration (90 seconds) to ensure that the patient can reach the end of the game even if he sometimes makes a mistake along the path. This is in order to define the treatment plan based on the patients’ disability, aimed at selecting the most appropriate game and level to start and continue therapy.

3. Study Design

Within the STORMS project, a feasibility study has been carried out at San Martino hospital with several 20 patients affected by MS. After having learned how to use the system through some sessions carried out at the neurorehabilitation clinics, under medical supervision, some of the patients took the system at home to perform a telerehabilitation period.

Patients have been recruited according to the following inclusion and exclusion criteria:

3.1. Inclusion Criteria

• Confirmed diagnosis of Multiple Sclerosis (MS).
• Adults aged between 18 and 60 years.
• Expanded Disability Status Scale (EDSS) score ≤ 7.5.
• Below normal scores on at least two neuropsychological tests.

3.2. Exclusion Criteria

• Severe mood disorder.
• Steroid therapy within 2 months prior to the visit.
• Inability to maintain adequate visual fixation (e.g., nystagmus).
• Presence of post-chiasmatic perimetric defects.
• Photosensitive epilepsy.
• Poor compliance or insufficient motivation to follow the treatment regimen.

The neuropsychological battery consists of tests that are commonly used for cognitive assessment in individuals with disabilities. Specifically, the Brief International Cognitive Assessment for MS (BICAMS) and the Paced Auditory Serial Addition Task (PASAT) at 3-second and 2-second intervals have been used. BICAMS includes the Symbol Digit Modalities Test (SDMT), the California Verbal Learning Test II edition (CVLT-II), and the Revised Brief Visuo-Spatial Memory Test (BVMT-R). Patients were recruited if they scored below the 5th percentile for normative data adjusted for age, sex, and education in at least two of the aforementioned tests. Written informed consent was obtained from all participants before the study began.

All cognitive measures were administered at baseline (i.e., time T0), at the end of 10 exergame sessions (T1), and one month after the end of treatment (T2). Two of such studies are analyzed in the present paper.

In the former study, in the absence of a specific prescription, patients at home could perform the exercises they found most pleasant. However, this has led to greater use of activities that are easier or preferred, to the detriment of those that are more challenging or cognitively meaningful. As a matter of fact, patients outperformed basic exergames rather than improving their skills by leveling up.

In Section 4.1 the most significant results of the observation of a 49 year old patient (patient A) who practiced the ReMoVES system at home for almost a month will be presented.

For the second study, it was decided to take a different approach, prescribing a patient with a precise and personalized weekly treatment plan, with activities distributed from Monday to Thursday.

The patient analyzed in Section 4.2 (patient B) is a 56-years old woman and took the system home for 4 weeks. At the end of each week, a brief report detailing the outcomes of the activities was available to the medical professionists so that the therapy schedule could be improved and personalized for the next week. The results obtained on a weekly basis will be analyzed and the prescriptions will be compared with the actual sessions played by the patient.

Table 2 displays the cognitive assessment scores for Patient A and Patient B at T0, including MMSE, PASAT, SDMT, CVLT-II, and BVMT-R scores along with their respective cutoff values. Both the 3- and 2-second PASAT were administered. The cut-off takes into account the fact that both patients have less than 12 years of education.

Patient A is below the cutoff in PASAT, SDMT, and BVMT-R; Patient B is below the cutoff in SDMT and BVMT-R. PASAT tests failed (i.e., N).

4. Results

4.1. First Study

Patient A used the ReMoVES system at home for almost a month and his activity was observed throughout the period. Table 3 summarizes the number of sessions played for each exergame, specifying the chosen level. As one can notice, the patient played mainly at the first level of each activity, focusing more on the basic ones (Shelf Cans, Hot Air and Owl Nest).

The large number of HotAir sessions carried out demonstrate the patient’s interest and involvement in what is considered a pleasant activity. The analysis of the acquired data confirms the ease with which the patient is able to carry out this activity in which he almost always reaches maximum performance. In Figure 1 one can see that in most cases all the targets were caught correctly, apart from a few game sessions where the patient missed a few targets.

Referring to the supermarket exergame, the patient’s cognitive failures concern two types of errors that may occur when taking various objects. A simple error, the so-called "semantic error" occurs when taking a wrong object belonging to the same category as the one described (food or no-food) and is named a "semantic eror"; a more serious error occurs when the taken object belongs to a different semantic class than the requested object. From the Figure 2 it can be observed that the patient made some errors, even at the semantic level. The errors committed in sessions 1, 2 and 3 respectively are reported in Table 4 together with the objects in the to-do list. Such errors are most likely the reason why the patient played this game so little and only at the first level. The situation is similar for the exergame Business-by-car.

The sessions of the Shelf Cans and Owl Nest exercises are much more numerous and will be analyzed in more depth in the following sub-paragraphs.

4.1.1. Shelf Cans Analysis

The learning curve for the Shelf Cans activity reported in Figure 3 depicts the progression of gaming performance with increased experience, measured by the number of sessions. Gaming performance is quantified as a percentage increase relative to the first session. As expected, one can observe a low score in the initial sessions and a subsequent steady improvement during the later sessions.

To analyze patient performance in executing the Shelf-Cans exergame, various parameters are observed and plotted. Figure 4 provides an overview of the results, illustrating the angles between the optimal trajectories and the trajectories performed by patients, along with the time elapsed in moving the colored cans to their corresponding shelves. The graphs on the left depict the angles between the optimal trajectory and the trajectory executed by the patient. Meanwhile, the graphs on the right illustrate the time taken to complete the required movement. The results for the red, orange, and green trajectories are displayed from top to bottom, respectively.

A noticeable trend is the overall decrease in the time required for each session, indicating an enhanced speed of movement execution over time. Conversely, the angles between the optimal trajectory and the trajectory executed by the patient, serving as an indicator of movement precision, increased for the red can trajectory, while either remained constant or decreased for the trajectories of the orange and green cans, respectively.

In addition, a correlation between the angles (indicating precision of movement) and the times (reflecting speed of execution) can be observed. For the red can, there exists a negative correlation ($\rho =-0.48$), suggesting that as speed increased, accuracy decreased. In contrast, for the orange can, there is a low correlation ($\rho =-0.10$). Notably, for the green can, a positive correlation is evident ($\rho =0.53$), indicating the individual’s ability to maintain both speed and precision in their movements.

Regarding the analysis of the patient’s movement, Figure 5 shows the range of motion of the shoulder angles, in the frontal, sagittal and transverse planes, respectively, at each session. We note how, after the first sessions in which the movement was not carried out completely, the values stabilize on measures corresponding to large and correct movements.

Finally, a comparison between patient A and a group of healthy subjects is depicted in the box plots illustrating the angle between trajectories (see Figure 6). For the red and green trajectories, both the patient’s (in red) and the healthy group’s (in blue) box plots exhibit striking similarity, displaying a certain symmetry in the data distribution. In contrast, for the orange trajectory, the patient’s box plots include some outliers, representing the mistakes made in placing some cans on incorrect shelves.

4.1.2. Owl Nest Analysis

The patient performed 23 sessions of the Owl Nest task. He played with his right arm, except for one session played with his left (i.e. session 5), without significant differences. He didn’t play every day of the week, but he did more than one session a day. Match scoring was generally high, except for session 12, which was the last of three on the same day.

Figure 7. Targets taken at each session, Owl Nest, patient A. Orange sessions have been played with right arm; the Blue session with the left arm.

Figure 7. Targets taken at each session, Owl Nest, patient A. Orange sessions have been played with right arm; the Blue session with the left arm.

Viewing the trajectories of the hand moving across the screen during gameplay provides insight into the accuracy of the movement. Two extreme cases are shown in the Figure 8: the trajectories of session 12, in which the patient was visibly tired, and the trajectory of the last day where the patient demonstrated precise control of his movements.

4.2. Second Study

As described in Section 2, the second study incorporates the prescription provided to the patient making rehabilitation exercizes at home. Patient B used ReMoVES system at home for a 4 weeks period. Table 5 lists the exergames recommended for each week, along with the level and the minimum number of sessions suggested.

Figure 9 illustrates the patient’s adherence to the prescribed regimen.

Remote observation of the patient’s activity includes an assessment of adherence to the prescription and provides insights into the patient’s condition, including levels of difficulty in task execution, fatigue, and stress. The number of sessions performed each day can be visualized as shown in Figure 9, with separate plots for each of the four weeks. Sessions conducted on the first day of the week are represented in gray, those on the second day in blue, those on the third day in yellow, and those on the fourth day in orange. The prescribed number of sessions is highlighted with the red bar. The name of non-prescribed activities is written in red.

The graphs depicted in Figure 9 provide comprehensive insights for the medical team and should be interpreted alongside the analysis of execution parameters.

Although the patient did not strictly adhere to the prescription during the first week, she achieved commendable results in the assigned activities. Consequently, for the second week, therapists prescribed more advanced levels of the Owl Nest and introduced the new exergame of Business by Car. During the initial two weeks, the patient frequently engaged in the Owl Nest and Shelf Cans exergames without encountering notable difficulties. Consequently, these games were not prescribed in the subsequent two weeks. Instead, therapists introduced activities in which the patient seemed to face greater challenges, aiming to enhance her concentration and cognitive recovery.

However, it is evident over the course of the weeks that the patient deviated from the prescribed regimen and engaged in numerous advanced activities that were not part of the plan. Additionally, in weeks three and four, she did not participate in the prescribed Numbers exergame. Furthermore, there was a notable decrease in the number of sessions conducted during the final week.

A comparison between actual and prescribed gaming sessions highlighted a significant discrepancy. Various factors likely contributed to this gap, affecting the patient’s participation across different activities and prescriptions. The necessity to juggle multiple activities may have impacted her motivation and willingness to consistently engage in rehabilitation sessions. To enhance future interventions, it may be beneficial to adopt a more comprehensive and balanced approach tailored to individual needs and prescribed activities, which can benefit from the available observations. Addressing factors influencing participation can help ensure greater consistency in exercise adherence and facilitate the achievement of desired outcomes.

4.2.1. Business by Car Analysis

In the Business By Car exergame, the player navigates a car along a dynamically generated road. The car responds to lateral trunk movements, turning left or right accordingly. The car’s speed gradually increases until the player veers off the road, incurring a score penalty and resetting the speed. The player must memorize a list of destinations within a specified time frame, ranging from ten to twenty seconds depending on the selected difficulty level. Subsequently, the player drives the car along the path, selecting the correct route at intersections to reach the designated destinations. After completing the errands, multiple-choice questions appear on the screen, pertaining to the list of destinations or details observed in the game scenes or visited buildings. To answer, the player raises their arm and guides an on-screen hand to the answer button. The game offers three difficulty levels:

1. First Level: The player must remember four places to visit. The list of destinations is straightforward, without additional information. At intersections, a navigator guides the player to the correct route. If the player takes a wrong turn, a reminder message prompts them to pay closer attention at subsequent intersections. Two questions pertain solely to the list of destinations.

2. Second Level: The player must remember five places to visit. A detailed list of errands appears at the beginning of the game, providing context for the destinations. The navigator does not indicate the correct route, and only a warning message appears if the player deviates from the path. Three questions are posed at the end, two regarding the list of errands and one about a detail of a visited building.

3. Third Level: The player must remember six places to visit. No warning message appears for incorrect turns. At the end, four questions are presented, one related to the initial list of destinations and three concerning details observed in the visited buildings.

The graph displaying correct and incorrect paths per session provides a detailed assessment of the patient’s motor skills during gaming sessions. Correct paths indicate the patient’s ability to accurately follow the game’s predetermined path, while incorrect paths denote deviations or errors made. With this monitoring opportunity we can assess the patient’s attitude towards exercises and prescriptions; we obtain information on patient involvement and the appropriateness of difficulty levels.

The Business by Car game was prescribed to Patient B 12 times in total, at level 1 for the second and third weeks and at level two in the last week. From the analysis of the acquired data we can understand that the patient was stimulated to play and she totaled 22 sessions against the 12 prescribed.

Some fluctuation in the number of incorrect paths across sessions can be observed which may be indicative of various factors, such as the complexity of the path, level of concentration, or progressive adaptation to challenges. Sessions with a high number of incorrect paths may suggest instances where the patient encountered greater difficulties in following the predefined path, possibly due to increased gameplay complexity. Conversely, sessions with fewer incorrect paths may suggest a greater proficiency in managing the motor challenges posed by the game.

In addition, we can gain much more insight from the Figure 10. The patient, after playing level 1 correctly 5 times without errors, tried to play level 3 already in the 8th session, even though it was not prescribed. Since she failed 7 times out of 13 paths, the patient decided to go back to level 1. After two perfectly performed sessions at level 1, she tried level 2 in sessions 11 and 12. Since she encountered some errors, she used level 1 again for a while before switching to the prescribed level 2.

The graph depicted in Figure 11 illustrates the number of correct and incorrect answers per session, providing feedback on the patient’s cognitive abilities. Correct answers indicate the ability to provide accurate responses to questions posed at the end of the game, whereas incorrect answers signify errors or inaccuracies. The trend of correct and incorrect answers offers insight into the evolution of the patient’s cognitive abilities throughout the game program. Sessions with a high number of correct answers may indicate moments of enhanced understanding and concentration, while those with more incorrect answers could suggest instances where the patient struggled with processing information.

In general we can notice how the cognitive performances in the Business by Car exergame are reproducing a situation very similar to the motor performance along the played 22 sessions.

5. Cognitive Assessement

Table 6 presents the cognitive assessment scores for Patient A and Patient B at T1, at the end of 10 sessions. One can notice a general improvement, which is particularly significant for PASAT-3", and SDMT that is now above cutoff for both patients. Finally, one month after the end of treatment (i.e, T2), Table 7 shows that all indexes have been preserved or improved. BVMT-R scores show a significant increase from T0 and after T1 even though they are still below the cutoff, but not so far to reach it.

Finally, Figure 12 and Figure 13 compares all cognitive measures at baseline (T0), at the end of the 10 sessions (T1), and one month after the end of treatment (T2). The general improvement of all the indexes can be observed at one time.

6. Discussion and Conclusions

In the framework of STORMS project, the utilization of ReMoVES has been positively embraced by all patients, regardless of age, including elderly individuals typically less inclined towards emerging technologies. This has resulted in increased adherence to rehabilitation protocols, also enhancing the duration of treatment per session. Exergames can be regarded as virtual reality tools, representing an innovative approach to augment motor learning.

The "Challenge Point Hypothesis" [13] suggests the existence of an optimal difficulty level to maintain patient attention without inducing boredom or fatigue, which can lead to frustration and therapy abandonment. Incorporating gamification aspects can enhance treatment outcomes by creating a stimulating and engaging environment.

ReMoVES serves dual purposes as an assessment/measurement outcome tool and a rehabilitation instrument. It enables continuous measurement and monitoring of patient performance, facilitating a detailed functional analysis. This empowers physicians and rehabilitation professionals to identify and/or modify various rehabilitation strategies.

The user-friendly interface and cost-effectiveness of ReMoVES provide the opportunity to continue treatment at home, offering advantages in terms of cost and treatment effectiveness.