Com@Rehab: An Interactive and Personalised Rehabilitation Activity Based on Virtual Reality

1. Introduction

Coronavirus disease 19, commonly known as COVID-19, outbroke into one of the most severe global health threats of all time, being considered a pandemic in March 2020 by the World Health Organization (WHO) [1]. It is caused by the infection and spread of a severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that has affected the population of around 216 countries worldwide, causing more than 6 million deaths [2].

The disease is characterised by several symptoms with a wide range of severity depending on factors related to the individual, for example, age, pre-existing respiratory complications and other health issues, as well as the virulence of the SARS-CoV-2 [1].

In cases where the infection progresses, the developed symptoms may require urgent medical intervention and ICU admission in more severe cases. Once in the ICU, critically ill patients may require mechanical ventilation and be immobilised for long periods. Consequently, those who survive the critical illness experience significant disability and impairment in their cognitive and physical function that can persist for several years after the initial hospitalisation, leading to a loss of autonomy, reduced health-related quality of life, and higher 1-year mortality [3]. This generalised muscle weakness and loss of muscle mass are referred to as intensive care unit-acquired weakness (ICUAW).

Motor dysfunction is a common and serious complication of critical illness, which can be caused by critical illness polyneuropathy (CIP), critical illness myopathy (CIM), or a combination of both [4]. CIP is characterised by nerve damage that affects the muscles, particularly in the limbs, while CIM is a type of muscle weakness caused by inflammation and disruption of muscle fibres. It has been reported that approximately 26% to 65% of patients who receive mechanical ventilation may experience muscle weakness upon awakening in the ICU, and ICU-acquired weakness (ICUAW) has been diagnosed in up to 67% of patients who require long-term mechanical ventilation (≥10 days) [5]. Therefore, motor rehabilitation is crucial not only during the ICU stay but also after hospitalisation to facilitate the recovery of motor function, improve range of motion, restore coordination, and enable patients to regain their independence and resume a normal lifestyle.

An early and proven safe and viable rehabilitation approach is currently applied to ICU patients, involving mobilisation, intensive insulin therapy, and electrical muscle stimulation [6]. On the other hand, although rehabilitation has long been viewed as a crucial strategy for improving functional recovery in adult survivors of critical illness, there is currently limited evidence supporting post-ICU rehabilitation for post-critical illness patients [6]. Nonetheless, advancements in technology have led to the emergence of promising tools like virtual reality (VR) technology, which offers a substitute or complement for conventional rehabilitation approaches. The use of VR devices is supported by the fact that it allows intensive, assisted, repetitive, and task-oriented rehabilitation, which is crucial for achieving functional recovery, especially in severe motor deficits. Moreover, it provides patients with an increased amount of sensorimotor information. It empowers them to execute complex and lifelike tasks in a secure and regulated setting, akin to real-life circumstances [6].

Although the significance of technology has been steadily increasing over the years due to its integral role in transforming various aspects of society, ranging from communication to healthcare, there are still difficulties observed in patients’ interaction with technology. Moreover, concerning the rehabilitation process, therapists need to develop simple, clear, motivating and effective communication with the patient to reduce communication gaps felt by those involved.

Within this scope, Com@Rehab, a collaborative research project awarded by Prémio de Investigação Colaborativa Santander/NOVA 2019-2020, was created, bringing together several competencies, such as linguistics, communication and health literacy, motor rehabilitation, as well as virtual reality and game development, to facilitate health communication using technology.

Com@Rehab is a technological patient-centred VR activity idealised for individuals needing motor rehabilitation and with a certain loss of function due to critical illness hospitalisation, with the final aim of bringing back independence in daily living activities. This system aims to create a more effective, interactive, and humanised rehabilitation tool based on a VR activity and involving a Communication Module, therefore improving human-human and human-machine communication and contributing to technological literacy. The VR activity encompasses scenarios of real-life daily living activities, in particular showering, customised according to the patient’s clinical needs. The goal of this paper is to present the Com@Rehab concept and first prototype as an advance in Patient Rehabilitation and Technology through a description of the VR activity, its components, design, and user profile, as well as its versatility and future adaptability regarding different ages, pathologies, and rehabilitation contexts. In addition, to test and get feedback on the developed system and validate the player’s experience in a VR activity, informed consent was applied to a group of healthy individuals for them to participate in the research study. This was followed by a validation questionnaire, designed by the research team to collect sociodemographic data and assess three components: functional literacy, communication literacy, and motivational literacy [7,8].

2. Materials and Methods

Com@Rehab was designed and created by a multidisciplinary team gathering expertise in areas such as Linguistics, Technology, and Rehabilitation, whose purpose is to contribute to the rehabilitation of post-ICU patients in home environments. As mentioned, it is the result of interdisciplinary collaborative research that followed a co-creation methodology, integrating the research centre HEI_Lab, three organic units of NOVA - FCT, FCSH, NMS - in collaboration with the Centro de Medicina e Reabilitação de Alcoitão (CMRA) and the Value for Health CoLAB. The study was approved by the Commission on Ethics and Deontology of Scientific Research at the Escola de Psicologia e Ciências da Vida of Universidade Lusófona de Humanidades e Tecnologias in May 2022.

The Com@Rehab arises from the need to adapt the VR4NeuroPain^® solution to post-COVID patients and from the observation of the difficulties patients feel in interacting with technology, and in this context, therapists need to develop motivational communication with them. The VR4NeuroPain^® is an innovative interactive rehabilitation technology system, triply awarded (SantaCasa Challenge, Acredita Portugal, Hintt), which combines VR and a biosensor glove to rehabilitate patients in a hospital/home setting. To reduce the communication gaps felt by all involved, it is intended to integrate into the existing solution a communicative support module for post-COVID patients, designated MCDRehab. Com@Rehab integrates an interactive rehabilitation methodology focused on humanised communication between rehabilitation professionals (therapists) and patients through a virtual reality game. In this methodology, the patient plays a central role.

The following steps were taken in the development of Com@Rehab concerning the methodology employed and are shown in Figure 1:

• Identify the requirements of the serious game;
• Definition of the role of the actors within the communication for rehabilitation (patients, caregivers, therapists, etc.);
• Characterisation of the patient profile;
• Decision on which activity of daily living should be employed in the game;
• Definition and description of the VR activity tasks and dividing them into difficulty levels;
• Definition of the game design and game mechanics;
• Definition of effective patient-oriented types of communication (including verbal and non-verbal communication);
• Structuring the VR activity and interface considering the previous steps;
• Game development for VR rehabilitation for oculus Quest 2;
• Design of the questionnaire to collect data and evaluate health literacy components;
• Focus group test with non-patients (technology testing).

The defined requirements for the VR activity were firstly, being easy to use, which means the platform should be intuitive and quick to learn, especially since the main users are older adults, as well as physicians or therapists that professionally are not used to working with technology. Secondly, performing a serious and interactive patient-centred rehabilitation using different forms of communication, always with a motivating purpose. The VR activity should also be accompanied by a therapist and report the patient’s progress. Furthermore, the VR activity should be comprehensive and compatible with other types of diseases that also have motor repercussions besides COVID-19. A game scenario was developed to meet these requirements, where the innovation lies in the contributions of linguistics to structure the interactive component and increase communication effectiveness during the VR activity.

Regarding the patient profile, the inclusion criteria are: age above 65 years, post-ICU hospitalisation with overall decreased strength, i.e., muscle weakness and reduced range of motion. The individual shows no signs of ataxia, sensory deficits, or cognitive and/or perceptive impairments. Moreover, based on the International Classification of Functioning, Disability and Health (ICF), which is the WHO framework for evaluating health and disability at both individual and population levels, it was determined that the patient suffered from a “Moderate Problem”. ICF offers a generic qualifier with the negative scale, used to indicate the extent or magnitude of an impairment, where the moderate problem is defined as up to half of the time or half the scale of total difficulty, corresponding to a broad range of percentages related to the severity of the problem of 25% - 49% [9].

The ICF’s Activities and Participation domain guided the selection of suitable rehabilitation activities for the platform. Specifically, Self-Care activities were chosen, given their emphasis on taking care of oneself and personal health, as they are crucial for maintaining self-esteem and one’s image [9]. Within this chapter, Com@Rehab focuses on the subchapter of washing oneself, particularly in showering, where the patient is led, step by step, to reproduce the appropriate movements for autonomously washing a specific body area. The difficulty of washing body parts may vary depending on the individual’s condition and limitations. However, body parts that are harder to reach, require more range of motion or require more fine motor control may be more difficult to wash independently. Thus, the VR activity and the game design were structured so that body parts were arranged in varying difficulty levels, and progression to the next level depended on the preceding level’s successful completion. The choice of this setting is related to the patient’s occupational interests and the repercussions that post-COVID sequelae have on the performance of activities of daily living. In addition, in conventional therapy, the activities of daily living are trained, namely showering, in a clinical context to promote patient independence. In addition, before the training, the activities are graded to adapt to the capacity and dysfunctions of each patient.

For instance, the first difficulty level includes washing the belly and forearm, with voice instructions using clear and easily understandable language for the patient.

To develop the framework, it was necessary to outline the different types of communication underpinning the game development stage, subdivided into two, game-patient communication and patient-therapist-game interaction. On the one hand, the game-patient communication includes verbal communication, with voice-based and written instructions/messages containing domain-specific terms and expressions adapted to patient/caregiver understanding. On the other hand, non-verbal elements have also been included and play a key role, for example within the scope of VR, an animation of a 2D User Interface (UI) assistant performing the intended activities is used, and hence mimics, in case the player needs help to perform the activity; there are also sound effects, such as clapping, to congratulate the patient for the successful completion of a given task; the game contains visual indications of temperature and dirt when, for instance, a body part is being washed, which in turn, provides information on how long the patient might still take to complete that task; finally, emojis have also been incorporated to allow the patient to describe his/her emotions at different stages of the game.

Regarding the patient-therapist-game interaction, it exists in two forms: firstly, by showing motivational/emotional patient-oriented communication, which can be conveyed through written or visual means; for instance, the patient can interact with the game by selecting the body part he/she would like to wash which, in this case, will retrieve from the database a set of tasks pre-defined by the therapist according to that patient’s stage in the rehabilitation process, while the second form comprises therapist voice instructions for each rehabilitation exercise.

The interface was designed concerning the user’s characteristics and the narratives of the procedure and comprised two different screens. The main screen that shows the principal information is positioned on a level of a seated person, and the secondary screen shows auxiliary information below, usually the task’s status.

In cases where there are questions to the user, the options appear beside the main screen, and selection is made with the eye gaze.

The first challenge in the user interface and interaction was the clarity of the message, meaning the readability. A sans-serif font was used, and size and leading were considered to improve the readability in older adults [10].

The virtual environment in terms of space and colours was also designed with the impacts of ageing in sensory modalities, especially auditory, visual, and movement control (https://www.youtube.com/watch?v=6BbMis_tEMY) [11,12].

Figure 2. VR activity scenario with the animated character.

Figure 2. VR activity scenario with the animated character.

2.1. Prototype testing

The main goal of prototype testing was to evaluate the therapeutic experience through a self-assessment form for satisfaction and motivation. It was intended to collect the user’s opinions on their experience, namely the main difficulties encountered in using the VR activity and suggestions for improving the interaction. A usability test was conducted with a group of 33 individuals without associated pathologies to evaluate the activity’s usability from a technological point of view.

The tasks asked were divided into two different activities in a bathroom originally designed in a virtual environment: washing the belly and washing the forearm. These two activities correspond to the first level of the VR activity and have a maximum duration of 10 minutes. The activities’ options were chosen through the individual’s gaze direction to simplify the selection. For both activities, yellow balls are placed in the region to be washed to help perform the necessary path and successfully complete the activity. In contrast, in the case of the belly, it is first necessary to calibrate the location where the player intends to wash it. All activities have three repetitions, according to the rehabilitation process. If the player has difficulty performing the movements, a 2D animated character with animations of the movements to be performed in each of the activities is available on the player’s right-hand side. All player options, in addition to the number of times the player looks at the character for help, are recorded in the database after each session for further analysis. After completing the two activities, the VR session ends.

The questionnaire consisted of three groups of questions, the first related to functional literacy, the second about communicational literacy, and the third was about motivational literacy. Each of them was composed of 6-8 questions, which were rated by participants on a scale of 1 (totally disagree) to 5 (totally agree) according to their level of agreement. The functional literacy questions concern the clarity of the elements, for example, if the time is enough to read the instructions or if using gaze to choose the options is easy. Meanwhile, communicational literacy questions focus on understanding words, voice instructions, and non-verbal communication effects. Finally, motivational literacy, as the name suggests, has to do with the motivation felt by the subject concerning different aspects of the activity.

Once the questionnaire was finished, the test was concluded, with a total duration of approximately 30 minutes.

In regards to the questionnaires, interviews, and game metrics encompassed in this study, each individual was identified by a code. All of the data enters a database associated with the participant’s respective code, and the identity of each participant and his/her code association will be included in a single document accessible only to the investigators responsible for the Com@Rehab project, which will be destroyed upon completion of the study.

2.2. Experimental Protocol

The experimental protocol followed a series of steps demonstrated in Figure 3 that will be explained in the next paragraph. At the beginning of the experience, it was given to the participant a statement of informed consent, which included the objectives and relevant information about the experience, as well as the guarantee of anonymity of the recorded information and the contact information of the responsible researchers. Besides that, it was reinforced that the participation was voluntary and that the withdrawal from the experiment was available at any time. Once signed, the participant was asked to complete a sociodemographic questionnaire about age, genre, nationality, and experience with video games and VR, among others.

After that, and before starting the experiment, the VR glasses were placed on the participant, who was in a seated position, and the handling of the controllers was explained. After completing the activities, the VR glasses were removed, and a questionnaire about the game experience and its usability was conducted. In this type of activity, considering that the participant only had to perform movements of the upper limbs and head, the risk of cybersickness was very low. However, in case it occurs or if the participant feels any discomfort, he/she has the possibility to abandon the experience at any time.

3. Results

The present analysis aimed to investigate the effectiveness of the VR technology integrated with a communication module in motor rehabilitation. As referred to earlier, we collected sociodemographic data from participants, including age, education level, and previous experience with VR technology. Additionally, we evaluated participants’ functional, communicational, and motivational literacy understanding after the intervention. In this results section, we present a detailed analysis of these outcomes to shed light on the future improvements of the project. It should be noted that the following focuses on the results of both the functional and motivational literacy sections of the questionnaire, while the investigation of communicative literacy is addressed in a separate article currently under preparation. Overall, our results suggest that integrating VR technology and a communication module holds promise for improving patient-centred rehabilitation in individuals with motor impairments.

3.1. Sociodemographic Questionnaire

The sociodemographic questionnaire was completed by 33 participants, with an average age of 36 years (standard deviation (SD) = 15.9) and an age range of 21 to 71 years. A pie chart that depicts the participant’s distribution by age is shown in Figure 4. The participants’ education level is represented in Figure 5, where the majority of participants (approximately 76%) had completed at least some college education, while 24% had a high school diploma or equivalent, and 33% had a PhD.

Regarding previous experience with VR technology, as can be seen in Figure 6, 76% of participants reported having used VR devices before, of which 27% used them occasionally or regularly.

We also conducted statistical analyses on their prior experience with VR based on age (Figure 7) and educational level (Figure 8). The results revealed a significant difference in the level of VR experience across different age groups, with older patients reporting less prior experience with VR than younger patients. However, there was no significant correlation between educational level and prior experience with VR. These findings can inform the development of future VR-based rehabilitation programs that consider the varying levels of prior experience with VR among different patient populations, focusing on the needs of older patients who may require additional support in using VR technology.

These findings suggest that our sample is relatively diverse in terms of age, education level, and familiarity with VR technology, which will be important to consider in our subsequent analyses.

3.2. Participant state

The graph represented in Figure 9 displays the number of selected emojis representing various emotions before and after the rehabilitation activities. The x-axis represents different emotions, including anxiety, demotivation, tiredness, excitement, happiness, and motivation. The y-axis represents the number of times each emotion was selected. The graph shows a noticeable shift in the distribution of emotions after the rehabilitation, with an increase in positive emotions and a decrease in negative ones. This suggests that the rehabilitation process had a positive impact on the patients’ emotional well-being.

3.3. Functional Literacy Questionnaire

The functional literacy part of the questionnaire was composed of 8 questions related to the function of the game, more specifically, if the types of communication used were adequate and helpful. Through Figure 10, it is possible to conclude that there were two questions whose response was not as positive as expected, namely regarding the choice of options through the gaze (question 5) and the use of the animated character or UI assistant as an animation to mimic the intended movements and therefore clarify and help accomplish the tasks (question 8), both questions related to the technological aspect of the activity. Considering the divergence among respondents regarding the use of the animated character, it is imperative to enhance its purpose and provide a clearer understanding of its intended function, thereby addressing an existing gap in the game. Respondents generally provided positive feedback to the remaining questions, indicating that the literacy techniques were proficiently implemented in the functional aspect of the game.

3.4. Motivational Literacy Questionnaire

The motivational literacy questionnaire, which consisted of 6 questions regarding the motivation of participants when playing the game and their responses to verbal and non-verbal communication, received the least amount of “totally agree” responses out of the three literacy questionnaires. As represented in Figure 11, the most positive response was the “neither agree nor disagree” option, indicating a lack of strong agreement. However, it should be noted that the usability tests were completed by healthy participants who did not require the motor rehabilitation process, and therefore, the less positive responses to questions 5 and 6 should not be interpreted as negative feedback.

4. Discussion

One of the strengths of the project lies in the interdisciplinarity and complementarity of the theoretical and methodological approaches underlying the different scientific areas involved. The various contributions are interrelated, aiming at designing and implementing an interactive and collaborative solution that, while highly technological, allows for the maintenance and strengthening of the human aspect, betting on more effective communication among all those involved in the rehabilitation process.

Com@Rehab aims to achieve the following results:

• Enhanced management of therapeutic activities and optimization of personalised rehabilitation through a virtual reality (VR) system tailored to the specific needs of each patient.
• Dynamic adaptation of the rehabilitation program based on continuous monitoring of patient progress and evolution.
• Reduction of burden on the healthcare system by enabling the use of the program in clinical and home environments, facilitating earlier patient discharge.
• Wide availability of the solution, including the communication module, to the entire community involved in the rehabilitation process.
• Creation and accessibility of tutorial videos in European Portuguese, Brazilian Portuguese, and English, targeting patients and caregivers, providing clear instructions on system usage.
• Evaluation of patient satisfaction through an experience report form to collect feedback and drive continuous improvements in the interaction system.

By achieving these objectives, Com@Rehab aims to revolutionise therapeutic interventions, offering a technologically advanced platform that promotes effective communication and improved rehabilitation outcomes for patients.

After conducting in-depth analysis of the laboratory and usability test results, we will proceed with optimising the initial level of activity. Once this is accomplished, the project will advance to subsequent phases, focusing on the development of the remaining levels and conducting validation with the target population.

Figure 12. Simplified step by step methodology ahead.

Figure 12. Simplified step by step methodology ahead.

5. Conclusions

In conclusion, this solution creates a more effective, interactive, humanised rehabilitation approach based on a VR activity. The Com@Rehab Communication Module improves human-human and human-machine communication while contributing to health literacy.

Regarding the upcoming steps, our plan involves conducting validation tests with end users and extending the framework to support additional languages, including sign language, and other activities of daily living. In order to maximise the communication components of patient rehabilitation technology, it is crucial to go deeper into the issues found, address the participants’ concerns, and explore creative solutions. Furthermore, once we have successfully demonstrated the proof of concept using COVID-19 as a starting point, we intend to shift our focus towards exploring other pathologies or medical specialties. This collaboration has been an incredibly enriching experience, and we are eager to expand it further. Our primary objective is always to address the needs and well-being of the patients, while also emphasising their active participation and engagement, which is a pivotal aspect of 21st-century healthcare.