Human-In-The-Loop Optimization Of Knee Exoskeleton Assistance For Minimizing User’s Metabolic And Muscular Effort

1. Introduction

Musculoskeletal disorders currently stand as the most prevalent work-related health problem among European workers. A 2019 report commissioned by the European Agency for Safety and Health at Work disclosed that 60% of workers with work-related health issues reported suffering from a musculoskeletal disorder, and 30% experienced muscular pains in the lower limbs [1]. Activities such as carrying or lifting heavy loads and working in uncomfortable positions, such as standing or squatting, pose a higher risk for lower limb musculoskeletal disorders [1,2,3]. Furthermore, it has been estimated that 63% of workers who carry or lift heavy loads exhibit musculoskeletal disorders [1].

Lower limb exoskeletons are starting to gain notoriety among industry employers as devices able to reduce musculoskeletal stress or metabolic cost by augmenting, assisting, or reinforcing the workers’ musculoskeletal abilities [4,5,6]. Promising results indicate that active lower limb exoskeletons can reduce the acute physical stress and strain of workers during tasks like walking and carrying heavy loads [6,7]. However, the adoption of these devices is still limited due to usability concerns and the lack of personalized assistance. Typically, an exoskeleton’s assistance is adjusted by performing an offline optimization, and the inter-subject variability in users’ motor needs and responses to the devices is disregarded [8,9]. This raises the need for employing adaptive control strategies that adjust the assistance to each user individually and in real-time.

In the last few years, human-in-the-loop (HITL) controls have been implemented in exoskeletons to automatically adjust the control parameters (such as the assistive torque) according to the physiological measurements of each user [9]. Overall, the HITL controls available in the literature have proven to be successful in optimizing the users’ metabolic cost [8,9,10,11], muscle activity [12,13], and gait speed [14]. For instance, Zhang et al. [9] developed a HITL control for a tethered ankle exoskeleton and obtained an average metabolic cost reduction of 24.2% and 14% compared to walking in a zero-torque condition and without the device, respectively. Ding et al. [15] and Xu et al. [12] developed a HITL control for hip exoskeletons capable of reducing the participant’s metabolic cost and muscle activity by 17.4% and 13%, respectively, when compared to walking without the device. Furthermore, Slade et al. [10] optimized the assistance of a wearable ankle exoskeleton, obtaining a 17% reduction in the users’ metabolic cost and a 9% increase in their selected speed, compared to walking without the exoskeleton. Han et al. [13] reduced the muscle activity of one participant wearing a tethered ankle exoskeleton by 31.6%, 26.2%, and 10.7% for normal walking, uphill walking, and loaded walking with 5kg, respectively, compared to walking with a zero-torque condition. Therefore, HITL controls have the potential to be used in industrial exoskeletons to lower workers’ metabolic cost while carrying and lifting heavy loads.

Indirect calorimetry is the standard metabolic cost estimation method [16]. It requires the use of a respirometer device to measure the flow of oxygen and carbon dioxide through a person’s respiratory system [16]. Despite obtaining reliable metabolic cost estimations, this method requires expensive equipment [17], takes a long time (ranging from 2 to 3 minutes) to obtain a steady-state metabolic cost [18], and generates a noisy signal [19]. Furthermore, the obstructive design of the respirometer device makes it impractical for workers to wear it in real industrial environments [20].

An alternative to indirect calorimetry is the use of machine- [16,20,21] or deep-learning [22,23] regression models that estimate the metabolic cost based on data acquired from wearable and non-intrusive sensors. For instance, Slade et al. [20] used a linear regressor that received inertial measurements from the shank and thigh as inputs, obtaining a mean absolute percentage error (MAPE) of 13.7%. Lucena et al. [17] developed a hierarchical linear regressor, with more inputs - the wrist, ankle, and hip acceleration, and the heart rate – and obtained a root-mean-squared error (RMSE) of 0.613 kcal/min. Ingraham et al. [16] also employed a linear regressor while studying the use of various input signals, namely the acceleration, heart rate, breath rate, electrodermal activity, skin, and ambient temperatures. Despite being considered a simple model, the linear regressor of Ingraham et al. [16] obtained an RMSE of 1.03 W/kg. More complex machine-learning models have been used, such as a support vector machine which presented an RMSE of 0.78 kcal/min only relying on the foot pressure and acceleration as inputs [21]. Lopes et al. [22] developed a convolutional neural network that estimated the metabolic cost based on the feet, legs, pelvis, and torso acceleration, the legs’ muscle activity, and the heart rate, reaching an RMSE of 0.36 W/kg. Overall, the linear regressors or some variation of these were the most used regression models [16,17,20]. Additionally, the input signals that have resulted in better metabolic cost estimation are: (i) the waist, wrist, and ankle acceleration; (ii) the muscle activity; (iii) the heart rate; (iv) the breath frequency; and (v) the minute ventilation [16,17].

To the authors’ best knowledge, the recent work of Slade et al. [20] is the only one that performed a real-time experiment using a regression model to estimate the metabolic cost, reaching a MAPE of 23%. Additionally, several regression models for metabolic cost estimation have been developed in the literature, however, these models have not been integrated into HITL controls yet. This work proposes a HITL control designed for an industrial exoskeleton, intending to assist workers in reducing their physical effort while carrying and lifting heavy loads. In this manuscript, our goal is to present a HITL control capable of performing an online and user-specific optimization of an exoskeleton’s assistive torque. This optimization aims to reduce the physical effort experienced by a knee exoskeleton user during walking. The users’ physical effort is evaluated by their metabolic cost and their interaction torque with the exoskeleton to provide a more comprehensive understanding of the human effort. This assessment of physical effort is in line with the definition of physical activity, which is any human movement resulting from a muscular force that causes an energy expenditure [24]. As far as we are aware, this is the first study that combines both a metabolic parameter and a muscle activity parameter in the cost function of a HITL control. Furthermore, this work innovates by using an inertial data-based regression model to estimate, in real-time, the metabolic cost, instead of relying on the indirect calorimetry method, as previously employed in the literature on HITL controls [9,11,19]. Our work contributes to scientific knowledge by demonstrating that regression models can accurately estimate physiological signals, namely the metabolic cost, in real-time; thus, opening a new avenue for HITL control application in industrial exoskeletons.

2. Materials and Methods

This section presents the development and validation of the HITL control designed for assisted working. Figure 1 depicts an overview of all stages of this control. The user is equipped with a smart assistive device named SmartOs. It includes an active knee exoskeleton and a team-owned wearable inertial sensor system [25]. The chest, wrist, waist, and ankle acceleration and the user’s body mass index (BMI) are used as inputs of an Exponential Gaussian Process Regressor (EGPR) model to estimate the metabolic cost. The user’s physical effort information (i.e., estimated metabolic cost and human-exoskeleton interaction torque) is subsequently sent to the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) optimization algorithm. It iteratively tests various combinations of control parameters to find the flexion and extension torque peaks that minimize the cost function (i.e., the user’s physical effort). Lastly, a torque-tracking control is used to generate the torque profile for each set of control parameters and ensure that the exoskeleton actuator’s torque replicates the desired pattern. A detailed description of each stage is presented below.

2.1. SmartOs System

SmartOs is a modular, smart, and fully wearable assistive exoskeleton for the lower limbs, namely the knee and/or ankle joints of the H2 exoskeleton (Technaid, Spain). It includes a Central Controller Unit (CCU, UDOO X86 with an Intel® Celeron N3160 up to 2.24 GHz processor) responsible for managing the communication between all system modules, and for running motion analysis tools and the high-level controllers at 100 Hz. The CCU interfaces with two development boards with lower computational capabilities (STM32F4-Discovery board, STMicroelectronics, Switzerland). These boards consist of: (i) the Low-Level board, which manages the mid- and low-level controllers at 1 kHz and interfaces with the active knee and ankle exoskeletons (H2 exoskeleton, Technaid, Spain) through a Control Area Network protocol; and (ii) the Wearable Motion LAB board, handling real-time data acquisition of team-developed sensor systems (such as the InertialLab system [25]). Further details are presented in [26].

In the scope of this study, we used the knee exoskeleton for the right lower limb (as illustrated in Figure 1). It consists of an electrical actuator (EC60 100W Flat Brushless (Maxon, Germany)) coupled to a gearbox (CSD-20-160-2A-GR (Harmonic Drive, Japan)) with a gear ratio of 160:1, capable of providing a nominal torque of 20 Nm in the sagittal plane. The device is also equipped with various sensors: (i) a potentiometer that measures the knee’s angle; (ii) strain gauges that measure the user-exoskeleton interaction torque; and (iii) Hall effect sensors that assess the motor’s torque. The power supply system consists of a lithium iron phosphate battery (LifePO4) with a voltage of 22.4 V and a capacity of 12 Ah. Moreover, we used four inertial measurement units (IMUs) from a team-owned wearable inertial sensor system [25] to measure the 3D acceleration signals of the chest, right wrist, left waist, and right ankle at 100 Hz. Figure 1 presents the on-body IMUs’ locations in blue.

2.2. Metabolic Cost Estimation

The metabolic cost is estimated in real-time by a machine-learning regression model (blue block of Figure 1), namely an EGPR. From our previous benchmark analysis [27], we verified that the EGPR was the best-performing model when compared to other machine and deep learning regression models such as a boosted decision tree, a bagged decision tree, a support vector machine, and a convolutional neural network [23]. The EGPR model, with an optimized Sigma of 0.15, was trained with data from a publicly available dataset [16]. The data selected from the dataset included the acceleration measurements at the chest, right wrist, left waist, and right ankle, and the BMIs of 10 participants. These input signals were selected since they presented better results than the electrodermal activity and the acceleration derivative and vector norm in a previous study [27]. Additionally, only the data from the relevant industrial activities, namely the standing, sitting, and walking activities, was selected. The ground-truth metabolic cost was estimated by indirect calorimetry.

We implemented the trained regression model in the SmartOs’ CCU board (C++, Ubuntu mate) to estimate the metabolic cost in real-time every 10 seconds. This time window guarantees at least two respirations and can be considered adequate to estimate the instantaneous metabolic cost [27]. For this purpose, the system executes the following steps in real-time: i) measuring the 3D acceleration signals of the four mentioned on-body locations; ii) filtering these signals with a 4^th order Butterworth low-pass filter of 20Hz [23]; iii) reorganizing the data into 10-second windows [28]; iv) creating the feature vector by computing the mean absolute deviation (MAD) of each variable [29]; (v) normalizing each feature using the z-score method; and (vi) using the previously trained EGPR regressor to estimate the metabolic cost based on the input features. Figure 2 presents the sequence of all the pre-processing methods, which were established based on the results of our previous study [27].

2.3. HITL Optimization

The HITL optimizer is responsible for adjusting two exoskeleton’s control parameters over various iterations to find the set of parameters that minimize the user’s physical effort (the cost function). This effort is measured by a weighted sum of the user metabolic cost estimated by the EGPR model and the human-exoskeleton interaction torque, with equal weights for both parameters. The interaction torque has a direct relation to the user’s participation and muscle effort since the muscular activity decreases as the users reduce their interaction torque. Combining physiological measurements that regard both the metabolic and the muscular effort provides a more comprehensive perception of the user’s needs; however, it was not proposed in previous human-in-the-loop controls.

For this optimization problem, we implemented a CMA-ES optimizer into the Low-Level board of SmartOs (green block of Figure 1). The CMA-ES is a derivative-free search algorithm, commonly used for black-box scenarios, such as HITL controls in lower-limb exoskeletons [9,11]. It stochastically samples several candidates over several generations and learns by adapting the covariance matrix of a multivariate normal distribution in real-time [30]. We computed the initial parameters of the algorithm by following the strategy proposed by Hansen et al. [30]. Table 1 presents these initial parameters.

An iteration ($\lambda$) refers to each tested set of control parameters. The optimizer performs a certain number of iterations before updating its internal parameters (called a generation). The CMA-ES optimization process is finished after performing 20 generations (each with 6 iterations). Thus, the optimization lasts for 20 minutes, similarly to previous studies [10,15,31], as each iteration takes 10 seconds to complete (the time required by the EGPR to estimate the metabolic cost).

The control parameters adapted in real-time by the CMA-ES are the torque magnitudes of the flexion and extension peaks of the knee torque profile. Figure 3 presents the knee torque profile, which represents the torque trajectory that the exoskeleton must perform over one gait cycle. The profile is composed of three sections. While the right leg is fully stretched, the torque is null. A first positive torque curve causes the knee flexion motion and a following negative curve executes the knee extension motion.

Figure 3 presents the possible torque profiles tested by the CMA-ES algorithm. The optimizer was set to test flexion torque peaks between 15 Nm and 20 Nm and extension torque peaks between -15 Nm and -20 Nm. The torque profile is initialized with values of 17.5 Nm and -17.5 Nm for the flexion and extension peaks, respectively, intermediate values to the range of tested control parameters. These torques are values allowed by the SmartOs DC motor and can guarantee a comfortable gait for every user, something previously tested and empirically evaluated with various persons. Subsequently, the CMA-ES explores various combinations of these two parameters in each iteration until the maximum number of generations is reached, identifying the optimal profile, i.e., the torque profile that minimizes the SmartOs user’s physical effort.

2.4. Torque-Tracking Control

We developed a torque-tracking control as the low-level of the HITL control and implemented it in the Low-level Board of SmartOs (red block of Figure 1). The torque-tracking control aims to adjust the exoskeleton actuator’s torque based on the reference torque profile, as defined by the peak flexion and extension torques. The reference torque profile is adjusted in real-time and individually for each subject during the HITL optimization. We used a natural cubic spline interpolator algorithm to generate the reference torque profile based on the flexion and extension torque peaks optimized by the CMA-ES. First, the points that define the torque profile are identified. Then, the spline algorithm creates a piece-wise polynomial cubic curve that intersects the desired points by obtaining three arrays of constants (a, b, and c), based on the approach introduced in [32]. By estimating the gait cycle percentage (P), it is possible to determine the desired torque at each gait stage ($T\left(P\right)$) by using Equation (1) [32],

$$T\left(P\right)=t_i+a_i(P-p_i)+b_i{(P-p_i)}^2+c_i{(P-p_i)}^3$$

(1)

where $i\in \{1,2,\dots ,n\}$ and $p_i\le P<p_{i+1}$, n is the number of points that define the torque profile, and p and t are the arrays defining these points coordinates, i.e., the gait cycle percentage and torque value, respectively. In this case, the torque profile was composed of 13 points ($n=13$), required to correctly shape the curve according to our needs.

Every 10 milliseconds (the mid-level control period), a gait cycle phase estimator is used to estimate the gait cycle phase, defined in percentage (P), based on the gait speed [33]. Then, this value is used to compute the desired torque by applying Equation (1). Lastly, the desired torque is sent to a proportional–integral–derivative (PID) torque controller, a closed-loop controller that minimizes the difference between the desired torque and the real motor torque, measured by the Hall effect sensors. Every millisecond, the PID controller transmits a command to the DC motor to force its torque to follow the reference pattern. The PID proportional, integral, and differential gains were optimized with the Ziegler–Nichols method and their values were 135, 1.5, and 1.5, respectively.

3. Experimental Validation

The presented solution was experimentally validated according to two protocols with healthy volunteers. The first experiment enabled us to assess the performance of the EGPR model in estimating the participants’ metabolic cost, in real-time, in comparison to the indirect calorimetry method. The second experimental protocol intended to analyze the efficacy of the HITL control in reducing the exoskeleton users’ physical effort, comparatively to other control methods.

All participants were informed of the study’s objectives and methodology and received an informed consent form, which they read and signed. Both protocols were approved by the University of Minho Research in Life and Health Sciences Ethics Committee.

3.0.1. Participants

The EGPR performance for metabolic cost estimation was tested during a real-time experimental protocol with 5 volunteers (3 males and 2 females). The participants were healthy individuals without clinical history or evidence of motor and cognitive impairments. They presented ages between 22 and 29 years old (25 ± 2.9 yr.), body masses ranging from 65 kg to 99 kg (77.8 ± 13.4 kg), and BMIs between 24.0 $kg/m^2$ and 29.2 $kg/m^2$ (26.8 ± 2.0 $kg/m^2$).

3.0.2. Instrumentation and Protocol

We instrumented the participants with four IMUs from the InertialLab system to measure the chest, wrist, waist, and ankle 3D acceleration (as shown in Figure 4). Further, the participants wore a respirometer device (K5, Cosmed, Italy) to estimate the ground-truth metabolic cost by indirect calorimetry. The participants did not wear the exoskeleton during this experiment but were equipped with a backpack accommodating the SmartOs control system, which executed the EGPR model every 10 seconds.

The participants performed three activities: (i) standing; (ii) treadmill walking at 1.5 km/h, 2.0 km/h, and 3.0 km/h, and (iii) sitting in a regular chair without arms. Each task was performed for 10 minutes, without rest breaks, and the three walking speeds’ order was random for each participant. This procedure replicates the experimental protocol performed by Ingraham et al. [16], whose data was used to train our regression model. Figure 4 presents one participant performing the experimental protocol while instrumented with the used equipment.

3.0.3. Model Evaluation

The data collected during the protocol was composed of the metabolic cost estimated by the SmartOs system, through the regression model, and the respirometer mask, through indirect calorimetry, which provided the ground-truth. Similarly to the procedure followed during the EGPR training, the ground-truth was directly computed by the unprocessed breath-by-breath data obtained by the respirometer. We evaluated the EGPR model’s performance by computing the RMSE and MAPE between the EGPR estimation and the ground-truth. These metrics were computed by using Equations (2) and (3), where N is the number of estimations, $y\left(i\right)$ is the $i^{th}$ ground-truth metabolic cost, and $\widehat{y}\left(i\right)$ the $i^{th}$ metabolic cost estimated by the EGPR model. Furthermore, we also analyzed the concordance between the two signals with Bland-Altman plots. The computational load of the regression model within the SmartOs architecture was also studied in bench tests.

$$RMSE=\sqrt{\frac{{\sum }_{i=1}^N{(y\left(i\right)-\widehat{y}\left(i\right))}^2}{N}}$$

(2)

$$MAPE=\frac{1}{N}\sum _{i=1}^N\left|\frac{y\left(i\right)-\widehat{y}\left(i\right)}{y\left(i\right)}\right|$$

(3)

3.1. HITL Control

3.1.1. Participants

We carried out an experimental validation to demonstrate the proof-of-concept of the proposed HITL control with one healthy participant. The participant was a 23-year-old female, with a body mass of 65 kg and a BMI of 24.5 $kg/m^2$, a value within the range of BMIs of the participants of Ingraham’s study [16]. She had no previous experience in using the knee exoskeleton.

3.1.2. Instrumentation and Protocol

During the experiment, the participant used the SmartOs knee exoskeleton on her right leg. She also wore the four IMUs required by the EGPR model to estimate the metabolic cost, placed on the chest, right wrist, left waist, and right ankle. Figure 5 depicts the participant performing the experimental protocol.

The participant walked on a treadmill at 1.5 km/h. The protocol was composed of three phases: (i) the acclimatization to the device; (ii) the online optimization of the torque profile performed by the CMA-ES optimizer for a maximum of 20 minutes; and (iii) the comparison of the HITL control (i.e., optimal torque-tracking control) with other exoskeleton control strategies. The participant rested for 10 minutes between each phase to guarantee that no phase would affect a subsequent one [11].

Figure 6 presents the activities performed by the participant during each phase. Firstly, the participant conducted an acclimatization phase that lasted until she felt comfortable while walking with the device. In this phase, the participant walked with the device in three different control strategies: (i) zero-torque control, where the exoskeleton functions in a passive transparent mode [34]; (ii) position-tracking control, a closed-loop control used to manipulate the actuator to follow a fixed reference joint angle trajectory [34]; and (iii) torque-tracking control with a fixed reference torque profile with flexion and extension torque peaks of 20 Nm and -20 Nm, respectively. Additionally, the participant rested between each mode until she felt capable of continuing the experiment.

In the protocol’s second phase, the participant walked with the exoskeleton until the CMA-ES optimizer found the optimal torque profile, i.e., the torque that minimized her physical effort. During this phase, the optimizer generated new control parameters (torque peaks) every iteration (10 seconds), over 120 iterations. These were sent to the torque-tracking control that computed a torque profile for each set of torque peaks and manipulated the knee exoskeleton to follow the desired torque trajectory. At each iteration, the CMA-ES analyzed the user’s response to the set of control parameters and determined the next values of peak torques to be tested. The process stopped when the peak torques that minimized her physical effort were found. After optimizing the assistance to the participant the CMA-ES was not again executed, and the HITL control applied the optimal peak extension and flexion torques to the torque profile, which was used by the torque-tracking control to manipulate the exoskeleton.

The third phase aimed to evaluate the efficacy of the optimal torque profile in reducing the user’s effort, and to benchmark the HITL control with commonly applied control strategies. The participant started the third phase by performing a trial without the exoskeleton. Then, she walked with the exoskeleton in three assistive strategies: (i) zero-torque control; (ii) torque-tracking control with the fixed reference torque profile with maximum extension and flexion peaks (20 Nm and -20 Nm, respectively); and (iii) HITL control, i.e., torque-tracking control with the optimal torque profile obtained by the CMA-ES optimizer in the second phase. This experiment was based on the protocols performed in the literature [15,31,35].

3.1.3. Control Evaluation

To evaluate the HITL control effectiveness, we compared the user’s metabolic cost and interaction torque during the four tested conditions and analyzed if using the HITL control led to an increase or decrease in the user’s physical effort. Boxplots were generated to display the variation of the participant’s metabolic cost and interaction torque during all conditions. Additionally, we also analyzed the variation of the participant’s physical effort and the two control parameters over the CMA-ES iterations.

4. Results

This section presents the results obtained during the experimental validation of both the metabolic cost estimation and the HITL control.

4.1. Metabolic Cost Estimation

Figure 7 depicts the evolution of the estimated metabolic cost by the EGPR model and indirect calorimetry during the experiment for one representative participant. We observe that, in general, the EGPR model slightly underestimates the metabolic cost in comparison to the indirect calorimetry method. This finding was consistent across all participants. Results also demonstrate that the EGPR model presents a less noisy signal than the indirect calorimetry.

We compared the metabolic cost estimation provided by the EGPR model and the indirect calorimetry method, for the 5 participants that participated in the experimental validation, by computing the RMSE and MAPE between the two signals. Figure 8 presents the average RMSE and MAPE for each motor activity, as well as the lowest and highest values of these errors across the 5 participants. On average, the RMSE and MAPE values were 0.66 W/kg and 26%, respectively, across all activities and participants.

Figure 9 presents Bland-Altman plots of the data for each activity, comparing the estimations made by the EGPR model (“Estimation”) and the indirect calorimetry method (“Target”). Each plot shows the agreement between the two metabolic cost estimations, identifying estimation biases, i.e., the average value of the difference between the ground-truth and the EGPR estimation. The bias increased with the exigence of the exercise, being, therefore, higher for walking at 3 km/h, and lower for the sitting activity. Figure 9 also shows that the difference between the two estimations was less dispersed for the static activities and walking at 3 km/h, a speed closer to a natural gait.

Additionally, we assessed the computational load of the pipeline of the regression model implemented in the CCU of SmartOs (Figure 2). The CCU took, on average, 0.23 ms to read and save the acceleration data, which was measured every 10 ms. Furthermore, to process the data and estimate the metabolic cost every 10 seconds, the model took 2.6 and 9.9 ms, respectively. Therefore, all computational processes were executed within the set requirements.

4.2. HITL Control

The results from the experimental validation of the HITL control strategy were analyzed to assess the behavior of the optimization algorithm during the HITL optimization phase and the performance of the optimal torque-tracking control when compared to other control strategies.

Figure 10 presents the progression of the real-time torque optimization with the CMA-ES algorithm. It shows the variation of the flexion and extension torque peak values (i.e., the control parameters) and the cost function value (i.e., the user’s physical effort) over the 120 iterations. After going through the optimization process, which lasted 20 minutes, the CMA-ES considered as the optimal control parameters the flexion and extension torque magnitudes of 18 Nm and -17.9 Nm, respectively.

Figure 11 depicts the boxplots of the user’s estimated metabolic cost variation throughout the four tested conditions in the third phase of the experimental protocol. Furthermore, Figure 11 presents the percentual reduction of the metabolic cost between the zero-torque control and the HITL control (i.e., the optimal torque control). The metabolic cost was higher with the zero-torque and the fixed torque controls, and lower with the optimized HITL control. The HITL strategy enabled a 7.3% reduction in metabolic cost compared to the zero-torque control.

Figure 12 presents the median of the module of the interaction torque during three different control strategies at the third phase of the experimental validation. Additionally, it is displayed the reduction of the interaction torque when comparing the zero-torque control and HITL control (i.e., the optimal torque control). The interaction torque measured when the participant experienced the optimized HITL control was similar to the fixed torque control, and 32.3% lower than the zero-torque control.

5. Discussion

The main goal of this work was the development of a novel HITL control for a wearable knee exoskeleton, capable of finding a user-specific optimal assistance that minimizes the users’ physical effort while they walk with the exoskeleton. The control presented in this study innovated by adapting the exoskeleton assistance considering two indicators of physical effort, the metabolic cost and the interaction torque representing the user’s metabolic and muscular effort, respectively. Additionally, this was the first HITL control strategy that relied on the metabolic cost estimated by a machine-learning model; thus, fostering a more practically assistive approach than indirect calorimetry for real use in industry. It should be noted that walking was the activity that required a higher metabolic effort from the participants, roughly 2 to 3 times higher when compared to the sitting and standing activities. Therefore, due to these results, this work focused on assisting exoskeleton users when walking.

5.1. Metabolic Cost Estimation

Similarly to this work, various previous studies have developed machine- and deep-learning regression models to estimate the metabolic cost based on data from inertial sensors. Strath et al. [36] used the wrist, hip, and ankle acceleration to estimate the metabolic cost with an RMSE of 0.73 kcal/min compared to the indirect calorimetry method, while Slade et al. [20] used as inputs the shank and thigh acceleration and obtained a MAPE of 13.7%. Other studies fused the acceleration data with additional inputs, such as: (i) the foot pressure (RMSE of 0.78 kcal/min) [21]; (ii) the heart rate (RMSE of 0.613 kcal/min) [17]; (iii) the heart rate and muscle activity (RMSE of 0.36 W/kg) [22]; and (iv) the electrocardiogram (RMSE of 0.71 kcal/min) [23]. The results obtained across the literature are highly variable, as they are dependent on the inputs used, the type of regression models, and the activities performed. Despite that, there is evidence that the RMSE is typically higher for more demanding exercises [20,21]. This is supported by our findings, as the average RMSE of the model was lower for the static tasks and increased with the walking speed, as seen in Figure 8. The higher RMSE for walking activity may be explained by the higher intra- and inter-subject variability in more demanding exercises. The Bland-Altman plots depicted in Figure 9 also showed an increase in the difference between the EGPR estimation and the ground-truth with the physical exigence of the task.

5.2. HITL Control

5.2.1. HITL Optimization

CMA-ES optimizers have been broadly used across the literature to adapt the assistance of lower limb exoskeletons in real-time [9,10,11,13,14]. The time that takes to fully optimize an exoskeleton’s assistance is directly dependent on the number of control parameters adapted but is also dependent on the termination conditions enforced to the CMA-ES algorithm. The literature studies that used a CMA-ES optimizer took from 12 minutes/parameter [13] to 36 minutes/parameter [14]. Regarding the control parameters optimized by the literature, these studies mainly focused on two [8,10,13] or four peaks [9,11,14] of the torque profile of an ankle exoskeleton.

Considering the literature outcomes, we also used a CMA-ES optimizer. The behavior of our CMA-ES algorithm during the optimization showed that the values of the control parameters (the flexion and extension torque peaks) fluctuated significantly at the beginning of the optimization process, and converged to their optimal final values over time, as the optimizer got smarter and closer to find the optimal solution. The value of the objective function (the user’s physical effort) presented a similar progression, getting closer to its minimum as the CMA-ES was learning. The algorithm took a total of 20 minutes to optimize the two control parameters of a knee exoskeleton, requiring less than 12 minutes/parameter to find the best exoskeleton assistance.

Additionally, the optimal flexion and extension peaks had approximately equal absolute values (≈ 18 Nm). This optimal value corresponds to an intermediate value between the maximum and minimum possible peaks (20 Nm and 15 Nm, respectively). Higher torque peaks result in faster and sudden movements, which increases the user’s metabolic cost, whereas lower torque peaks lead to higher interaction torque between the user and the exoskeleton. Therefore, the optimizer settled for flexion and extension peaks not too high nor too low. Despite that, it is worth noting that this optimal torque profile is a user-oriented control parameter.

5.2.2. Controls Comparison

In this study, we developed a HITL control for a knee exoskeleton that aims to adapt its walking assistance, in real-time, for each user, providing them with a personalized torque-tracking control. This is possible by employing an optimization algorithm that finds the control parameters that minimize the users’ physical effort, while they walk on a treadmill with the device. HITL controls have been increasingly studied in the literature, especially on lower limb exoskeletons for gait assistance. Regarding the assistive effects, single-joint exoskeletons with HITL controls were able to reduce the metabolic cost of their users when compared to walking without the devices, reaching reductions that range from 7% [8] to 17.4% [15]. Similarly, our study verified that the user’s metabolic cost was 5.9% lower when using the exoskeleton in optimal torque control than when walking without the exoskeleton. These outcomes suggest that the exoskeletons brings positive and augmentative assistive advantages even for healthy users while performing a common activity, such as leveled walking.

Compared to walking with an exoskeleton in a zero-torque control, the state-of-the-art of HITL controls achieved metabolic cost reductions of 17.4% [15] and 24.2% [11] when using a single-joint ankle exoskeleton and 48% when using a hip-knee-joint exoskeleton [37]. Our results showed that the user achieved the lowest metabolic cost when walking with the exoskeleton set for the user-specific optimal torque control, suggesting that the HITL strategy was effective in minimizing this physiological parameter. Compared to wearing the exoskeleton in a zero-torque control, the HITL control reduced the user’s median metabolic cost by 7.3%. This effect is according to the findings of Zhang et al. [9] for slow walking, who showed that the percentual reduction in metabolic cost depends on the gait speed, reporting 3%, 33%, and 39% for slow (2.7 km/h), normal (4.5 km/h ), and fast walking (6.3 km/h), respectively, when comparing a HITL control with walking in a zero-torque mode [9]. Moreover, the HITL control also led to a reduction of 32.3% in the user-exoskeleton median interaction torque when compared to the zero-torque control; thus, it appeared that the control was able to minimize the user’s muscular effort. Overall, these findings highlight the assistive advantages of active HITL assistance over passive assistance (zero-torque control), regarding the users’ metabolic and muscular effort. When comparing a torque-tracking control with torque peaks fixed to the maximum assistive values of the exoskeleton with the optimal peak values found by the HITL optimization, we demonstrated that the HITL control led to a 7.8% reduction in the user’s metabolic cost while maintaining similar user-exoskeleton interaction torque values. This showed that the fixed maximum torque-tracking control, despite enabling low interaction torques, resulted in accelerated movements which increased the user’s metabolic cost. Thus, the conducted use-case validation highlights the relevance of setting user-specific control parameters in line with the current paradigm of personalized assistance, rather than using fixed references across subjects.

The work presented here served as a proof-of-concept of a HITL control that optimizes both the user’s metabolic cost estimated by a machine-learning regression model and the human-robot interaction torque, a metric related to the user’s muscle activity. The results favor the future implementation of HITL control in active exoskeletons over zero-torque and non-user-specific assistance. However, it is necessary to conduct further experiments with a broader group of volunteers performing more physically demanding activities, such as load carrying and lifting.

6. Conclusions

This study presents a HITL control that appears to reduce the users’ physical effort while walking with a knee exoskeleton, measured by their interaction torque with the device and metabolic cost. The proposed strategy goes beyond the state of the art by estimating the user’s metabolic cost with a machine-learning regression model, based on data obtained from a minimal number of input signals measured by wearable accelerometers and the user’s BMI. The regression model achieved less noisy metabolic cost estimates and enabled the reduction of the time required to estimate the steady-state metabolic cost from 3 minutes to 10 seconds, compared to the indirect calorimetry method.

Furthermore, to the authors’ best knowledge, this was the first HITL control that minimized simultaneously a metabolic and muscular physiological parameter. The results suggested that walking with the developed HITL control led to a reduction in the user’s metabolic cost when compared to walking without the device and with the device in a zero-torque and non-user-specific torque control. The user’s muscular effort also decreased when assisted by the torque-tracking control with non- and user-specific parameters, compared to the zero-torque control.