Prediction and Fitting of Nonlinear Dynamic Grip Force of the Human Upper Limb Based on Surface Electromyographic Signals

1. Introduction

Surface Electromyographic (sEMG) signals generated by muscle contractions are widely used for the recognition of human movement intention. These signals are recorded and analyzed to determine the functional states of peripheral nerves, neurons, neuromuscular junctions, and the muscles themselves. When a muscle contracts, neurons transmit electrical signals to muscle fibers through neuromuscular junctions. These signals can be recorded by sensors and used to identify different actions or gestures. sEMG signals carry control information about the neuromuscular-skeletal system; they not only reflect muscle contraction intensity and activation but also contain dynamic information about limb movements. Due to their ease of collection, non-invasiveness, richness, and naturalness, sEMG signals are highly suitable as input sources for various human-computer interaction control systems. In recent years, this area has garnered widespread academic attention, with ongoing research and development leading to significant advances and preliminary applications in fields such as rehabilitation medicine, prosthetics and exoskeleton control, human-machine interaction (HMI), and health monitoring and prevention [1,2,3,4].

However, the use of sEMG signals to predict muscle force still needs to be further investigated. This is because the prediction of muscle force is indispensable if safe and smooth HMI system is to be achieve. For instance, the development of smart rehabilitation devices is increasingly driven by the need for precise force control. Although various commercial prostheses are available for users with limb disabilities, many of these prostheses offer limited movement types and have relatively low levels of sophistication. Most can only recognize specific actions and do not control the force applied by the prosthetic hand. In addition to smart prosthetics, many human-computer interaction devices using sEMG signals as a control input focus primarily on action recognition, often overlooking force control. During gripping, human hands not only perform specific actions but also apply varying forces, such as grip strength or pinch force. Simple action recognition alone is insufficient for achieving natural and functional prosthetic control. This can affect user experience and even safety. For example, in prosthetic hands, applying excessive force may deform or damage objects, while insufficient force might cause the prosthetic hand to drop objects. Thus, precise force control using sEMG signals is crucial [5].

Establishing a mapping relationship between dynamic grip force and electromyographic signals holds significant long-term potential. Dexterous hands are essential tools in daily life and work, playing a crucial role in various activities. Electromyographically controlled prosthetic hands are a key focus in current commercial development [6,7,8]. For prosthetics to effectively replace human hands, they must simulate the posture and force application of real hands. In many real-world scenarios, continuous dynamic grip force control is required, which necessitates a mapping between EMG signals and dynamically changing grip force. Integrating this mapping into smart prosthetics would enable them to mimic human-like dynamic grip control.

Many researchers are dedicated to developing methods for estimating muscle force based on sEMG signals. Kim et al. proposed a method for estimating fixed static forces based on sEMG signals [9]. This method estimates a fixed grip force value in scenarios with a constant force output. Wu et al. investigated the relationship between electromyographic signals and muscle force, demonstrating a good linear correlation between the two and confirming that force changes can be inferred from electromyographic signals [10]. In their work, for force decoding, the authors chose the most direct and simplest method of linear grip force control, analyzing the rate of linear change in grip force per unit time based on electromyographic signals and establishing a mapping relationship between linear grip force and electromyographic signals. While these studies extensively explored instantaneous, static, and linear grip forces, there is still a lack of specific investigation into continuously varying dynamic grip forces. Al-Timemy et al. also noted that changes in force can result in a 60% reduction in the accuracy of the electromyographic control system [11]. Subsequently, they investigated the robustness control of hand prostheses under varying force levels and found that changes in force may have a greater impact on the robustness of prosthetic control [12].

Therefore, one of the most important goals of force decoding from EMG signals is to estimate the required grip force in real life, in real time, based on sEMG signals, and to accurately reflect this force in prosthetics or other devices. However, the use of sEMG signals as a source of force control signals for smart prosthetics faces two problems. Firstly, it is necessary to utilise as few channels as possible for force analysis to meet the requirements of practical applications. Second, dynamic force estimation rather than static calibration needs to be performed using sEMG signals. Due to the limitations of experimental requirements and algorithms, the above two aspects need to have further research.

The aim of this paper is to find a suitable algorithm to establish the mapping relationship between the electromyographic signals of forearm muscle groups and dynamic grip force. And the experiment must be conducted in a dynamic force environment to ensure that the experimental conditions meet the requirements of practical applications. The experimental results verify the effectiveness of the proposed scheme.

2. Mechanism Analysis of sEMG Signals

2.1. Forearm Muscle Group Distribution

sEMG signals are generated by muscle contractions, with different muscle groups playing dominant roles depending on the type of movement and force applied. The strength of force in the forearm is closely related to the extent of muscle contraction. During gripping, the force exerted by different fingers varies, and each muscle has a specific function. Multiple muscles work together to complete a movement pattern. The forearm muscle distribution is complex, with different movement patterns potentially involving the same muscle groups or varying muscle groups for the same pattern. The closer a muscle is to the hand, the more its contractions are related to hand movement patterns [13]. Therefore, understanding the structure and distribution of forearm muscles is crucial for selecting target muscle groups for signal acquisition in experiments [14], leading to better models for analyzing movement conditions.

The forearm muscles, located around the radius and ulna, are divided into anterior and posterior groups, totaling over 20 muscles. Most of these are long muscles with long tendons; the muscle belly is proximal, while the tendons are distal, making the upper part of the forearm thicker and the lower part thinner. The anterior group mainly includes muscles for wrist flexion, finger flexion, and forearm pronation, located on the front and medial side of the forearm. The posterior group includes muscles for wrist extension, finger extension, and forearm supination, located on the back and lateral side of the forearm.

2.2. Noise Source Analysis

The sEMG signals are closely related to neuromuscular physiological activities and are bioelectrical signals that reflect muscle strength in different movement patterns. These signals are extremely weak and highly sensitive to noise, making them susceptible to various types of interference during acquisition. Therefore, analyzing and distinguishing noise sources in sEMG signals is crucial for selecting appropriate preprocessing methods to improve the signal-to-noise ratio. The main sources of noise in sEMG signals include:

• Baseline Drift and Motion Artifacts [15].

When collecting sEMG signals, electrodes must be fixed near the target muscle. During limb movement, changes in muscle state cause surface fluctuations in the arm, leading to relative movement between the muscle, skin, and electrodes. This relative movement can create motion artifacts in the electromyographic signals and result in baseline drift [16].

Factors such as cable jitter, loose connections, electrode displacement due to sensor looseness, high skin impedance due to poor skin cleanliness, and reduced electrode conductivity from sweat can all contribute to baseline drift or motion artifacts. This issue is more pronounced with metal dry electrodes due to their tendency to slide relative to the skin.

2.

System Noise

Electronic components in signal acquisition devices inherently produce noise. This noise is related to the performance and manufacturing process of the components, as well as environmental factors such as temperature and humidity, and the usage duration of the components. System noise, which is random and inevitable, stems from the inherent performance of electronic components.

3. Materials and Methods

3.1. Experiment Design for Subjects

In the signal acquisition experiment, we have developed a system that can simultaneously collect sEMG and grip force signals. A three-channel sEMG signal acquisition device was used, which supports wireless data transmission. The sensors had a sampling frequency of 1000 Hz, and the collected signals were filtered between 10-1000 Hz. Additionally, the grip force was measured using an S-type grip force sensor with a sampling frequency of 10 Hz.

Hand and wrist movements depend on the coordinated action of multiple muscles, and different hand movements involve different muscle groups. To enhance the accuracy and representativeness of action recognition, the study selected three specific muscles for sEMG signal collection. These muscles are: Brachioradialis (Channel 1), Extensor Carpi Ulnaris (Channel 2), Flexor Carpi Ulnaris (Channel 3).

Figure 1. Electrode placement.

Figure 1. Electrode placement.

Thirteen healthy subjects were recruited in this stud. The subjects were instructed to follow prompts from the experimenter. They first had three sets of hydrogel wet electrodes attached to the right forearm for measuring sEMG signals from the targeted muscles. The right hand was then positioned in a relaxed grip on the load cell sensor in preparation for detection. Under the experimenter's direction, subjects were required to gradually increase their grip strength to firmly grasp the load cell sensor from a relaxed state. Each grip was held for 5 seconds, with a 5-second interval between each grip, and the process was repeated three times.

3.2. Data Preprocessing

3.2.1. Outlier Removal and Filtering

To remove outliers, which can result from device interference producing data far above or below normal values, a threshold method was employed to filter out and retain only the valid data.

The sEMG is a microelectrical signal with an amplitude in the μV range, characterized by non-linearity and non-stationarity. The signal primarily falls within the frequency band below 500 Hz, with major frequency components concentrated between 10-300 Hz and core energy around 20-150 Hz [17]. Due to these characteristics, sEMG signals are highly susceptible to noise interference, which significantly impacts subsequent signal processing and gesture recognition. Additionally, during signal acquisition, 50 Hz power line interference can introduce 50 Hz noise.

The experiment uses a Butterworth filter for signal processing. First, a bandpass filter with a range of 10–500 Hz is applied to filter out high-frequency and low-frequency noise. Then, a 50 Hz notch filter is used to remove power line interference.

3.2.2. Wavelet Denoising

Wavelet denoising is a signal processing method based on wavelet transform, known for its excellent time-frequency localization properties and suitability for handling non-stationary signals. After wavelet decomposition, the wavelet coefficient magnitudes of the signal are generally larger than those of the noise. Thus, coefficients with larger magnitudes are typically signal-dominant, while those with smaller magnitudes are largely noise [18]. Therefore, wavelet transform effectively decorrelates the data by concentrating the signal's energy in a few large wavelet coefficients, whereas the noise energy is distributed throughout the wavelet domain. By applying a thresholding method, most noise coefficients are reduced to zero—coefficients below the threshold are considered interference noise and removed—followed by wavelet reconstruction to achieve denoising.

3.2.3. Removing Motion Artifacts

Baseline drift in sEMG signals, as explained in noise analysis, generally manifests as a very low-frequency curve superimposed on the original signal, causing slow and slight fluctuations. This baseline drift can distort FFT analysis, correlation analysis, and power spectral density analysis, leading to spikes in low frequencies and even obscuring the main frequency components, thus affecting accuracy [19].

Collecting the baseline drift in sEMG, the least squares method can be used to minimize the sum of squared errors and find the best function fit. This trend component, representing baseline drift or other signal trends, can then be fitted and subtracted from the original signal, resulting in a new signal with baseline drift removed, thereby improving signal processing accuracy and effectiveness.

3.3. Data Fitting and Regression

3.3.1. Neural Network Time Series Fitting

Neural networks, also known as artificial neural networks (ANNs), consist of interconnected nodes or artificial neurons arranged in layers, transmitting and processing data through weighted connections.

Neural network time series fitting is a method that uses neural network models to predict or analyze time series data. The core idea is to incorporate the time dimension of the data into the neural network, allowing it to model directly over time.

The Nonlinear Auto Regressive with eXogenous inputs (NARX) neural network was applied in this study, predicting future values of time series muscle force signals (response signal F(t)) based on past values of both the response signal and another time series of electromyographic signals (predictive signal E(t)). The NARX neural network is a specific type of recurrent neural network designed for time series data, learning the time dependencies between inputs and outputs to predict future output values [20].

The NARX neural network consists of an input layer, hidden layers, and an output layer, utilizing the backpropagation algorithm for training and continuously optimizing the network's weights and parameters by minimizing prediction error.

The defining equation for the NARX neural network is:

y(t)=f(u(t),u(t-1),u(t-2),...,u(t-Du),y(t-1),y(t-2),...,y(t-Dy))

(1)

In this equation, y(t) and u(t) represent the output and input of the network at time t, respectively. Du is the maximum order of input delays, and Dy is the maximum order of output delays. Therefore, u(t-Du), …, u(t-1) are the historical inputs relative to time t, and y(t-Dy), …, y(t-1) are the historical outputs relative to time t. The function f is the nonlinear function fitted by the network.

The structure diagram of NARX model is shown in Figure 2. The S(t) and F(t) represent the sEMG and force signals, respectively. he time delays in the NARX neural network refer to the time lag and feedback mechanism incorporated into the network. This represents the time period from the generation of the input signal to its effect on the output. This time difference can impact the network's prediction accuracy and the system's stability. Properly set delay parameters help the network better capture long-term dependencies and periodicity in time series data.

3.3.2. Comparison of Different Training Methods

In this experiment, two different training strategies were compared using the same training and test sets. The parameters included a 50-layer neural network with a 1:20 time delay. The strategies are Bayesian Regularized Stochastic Gradient Descent (BRSGD) and Conjugate Gradient (CG).

BRSGD is based on Stochastic Gradient Descent and using extra techniques to regularize the Network towards reducing bias and variance. BRSGD introduces Bayesian regularization into the training process by adding an additional term into the loss function. This additional term penalizes the presence of large weights, which is introduced to provide a smoother network response. This method has been applied to improve the robustness and generalization performance of neural networks, especially in scenarios with limited training data. Bayesian inference naturally incorporates regularization, which aids in preventing overfitting, making it particularly advantageous in Artificial Neural Networks [21].

And the CG method is a technique that lies between the steepest descent method and Newton's method. It only requires first-order derivative information but overcomes the slow convergence of steepest descent while avoiding the need for storing, computing, and inverting the Hessian matrix as in Newton's method. The Conjugate Gradient method is not only one of the most useful methods for solving large linear systems but also one of the most effective algorithms for large-scale nonlinear optimization [22].

We used the same dataset and conducted model training with two methods under identical hardware conditions (GPU NVIDIA RTX 3070 with 8GB memory, processor AMD Ryzen 7 5800H with Radeon Graphics at 3.20 GHz). We evaluated the two training methods based on mean squared error (MSE), regression value (R-value), and the time required for 1000 training epochs. The comparison results are shown in Table 1.

The comparison reveals that while the CG exhibits greater fluctuation in error distribution and has lower regression fit compared to the BRSGD, it shows a significant advantage in training time.

4. Result

Based on the comparison of training methods described above, this experiment uses the CG to analyze the impact of neural network parameters on regression results. The following data presents the test results under different network parameters (Du and Dy are the same, called Delay).

Table 2. MSE and regression value under Different Parameters.

Delay	Number of Neural Network Layers
	10		20		30		40		50
	MSE	R	MSE	R	MSE	R	MSE	R	MSE	R
10	0.35	0.9979	1.10	0.9889	1.00	0.9901	0.34	0.9680	0.44	0.9614
20	0.16	0.9986	0.44	0.9960	1.30	0.9875	1.40	0.9865	1.40	0.9859
30	0.30	0.9950	0.88	0.9903	0.89	0.9908	1.30	0.9857	2.20	0.9764
40	0.95	0.9997	0.37	0.9957	2.30	0.9769	5.10	0.9421	2.90	0.9681
50	0.25	0.9966	0.49	0.9942	1.00	0.9870	1.70	0.9800	2.50	0.9661
Average value	0.40	0.9976	0.66	0.9930	1.30	0.9865	1.97	0.9725	1.89	0.9716

* Mean Squared Error (×10⁻³).

Table 2. MSE and regression value under Different Parameters.

Delay	Number of Neural Network Layers
	10		20		30		40		50
	MSE	R	MSE	R	MSE	R	MSE	R	MSE	R
10	0.35	0.9979	1.10	0.9889	1.00	0.9901	0.34	0.9680	0.44	0.9614
20	0.16	0.9986	0.44	0.9960	1.30	0.9875	1.40	0.9865	1.40	0.9859
30	0.30	0.9950	0.88	0.9903	0.89	0.9908	1.30	0.9857	2.20	0.9764
40	0.95	0.9997	0.37	0.9957	2.30	0.9769	5.10	0.9421	2.90	0.9681
50	0.25	0.9966	0.49	0.9942	1.00	0.9870	1.70	0.9800	2.50	0.9661
Average value	0.40	0.9976	0.66	0.9930	1.30	0.9865	1.97	0.9725	1.89	0.9716

* Mean Squared Error (×10⁻³).

For different model layers, the average performance of time delay is calculated, and the results show shallow model has better performance and deep model has bad performance. This means that the overfitting may happened when layer increasing.

This paper demonstrates the fitting results using examples of (10 layers, 20 time delays), (30 layers, 40 time delays), and (50 layers, 40 time delays) as follows.

Figure 3. Test Results under 10 Layers and 20 Time Delays.

Figure 3. Test Results under 10 Layers and 20 Time Delays.

Under the (10 layers, 20 time delays) testing conditions, it is observed that the initial fitting results are not ideal, but they improve over time. This is because, at the beginning, the number of sEMG signal samples available for analysis is limited, making accurate results difficult. As time progresses, the number of usable samples stabilizes, leading to more accurate fitting results. The model shows sensitivity to the nonlinear dynamic changes in force, accurately describing the dynamic variations even during the grip maintenance phase.

Additionally, the model is also relatively sensitive to the start of force application and release. It fits the linear changes in force very precisely, accurately capturing the duration of force application and release. Notably, the fitting error in estimating force magnitude does not exceed 0.04, and considering that the force data has been normalized by the subject's BMI, the error in grip strength fitting is within 1 kg for a typical BMI of 20 kg/m².

Figure 4. Test Results under 30 Layers and 40 Time Delays.

Figure 4. Test Results under 30 Layers and 40 Time Delays.

Under the conditions of (30 layers, 40 time delays), it can be observed that the initial fitting error is further amplified. Larger fitting errors are concentrated at the end of the grip maintenance phase and at the beginning after the release of force. However, the model still performs well in fitting the linear changes in grip strength and the fluctuations in grip strength during the maintenance state.

Figure 5. Test Results under 50 Layers and 40 Time Delays.

Figure 5. Test Results under 50 Layers and 40 Time Delays.

Under the (50 layers, 40 time delays) testing conditions, the errors observed in the previous group are significantly amplified. Additionally, there is a noticeable deviation in fitting the linear changes in grip strength, making it difficult to accurately estimate the rate of grip strength changes during force application and release. Moreover, during the third force application period, the model shows a decreased sensitivity to irregular fluctuations in grip strength during the maintenance phase, leading to less accurate estimations.

5. Discussion

The results indicate that the BP algorithm and time series-based neural network fitting effectively predict grip strength from sEMG signals. The model performs well in both the linear changes during force application and release, as well as in grip maintenance phases, especially under the (10 layers, 20 time delays) conditions, with minimal overall error and high sensitivity to nonlinear fluctuations. The model's generalizability is supported by its consistent performance across different subjects. The best performance is MSE-0.16 and R-0.9986 when model has 10 layers and 20 time delays.

However, there are notable limitations. Initially, there is a significant deviation between predicted and actual results in the first few data points. This issue is likely due to the baseline grip strength estimation being based on data points approximately 10 samples before the first force application, leading to insufficient samples for accurate grip strength estimation during the maintenance phase with the delay-involved neural network analysis. The initial prediction is highly dependent by the start-up setting. So this can be improved by adaptation strategy and will done in the future.

Additionally, significant deviations are observed in the latter part of the grip maintenance phase. In this experiment, subjects were instructed to maintain maximum grip on the sensor for 5 seconds. Sustaining maximum grip imposes a considerable burden on the muscles, leading to potential interference in the sEMG signals during the latter part of the maintenance phase, which affects the model's prediction accuracy.

One possible factor is that the sEMG signal output mechanisms during force application and prolonged maintenance may differ. Specifically, the signal logic might change in the last few seconds of the 5-second period, and since subjects quickly transitioned to the release phase after 5 seconds, there were fewer samples available for analyzing the prolonged maintenance phase. This limited data may hinder the model’s ability to accurately interpret these signals. Another potential factor is the special changes in sEMG signals under muscle fatigue [21]. Literature suggests that fatigued muscles exhibit different sEMG signal patterns compared to non-fatigued muscles, likely due to interference from peripheral nervous system signals aimed at preventing excessive muscle fatigue. With a limited sample size, the model may struggle to adequately analyze these fatigue-related signal variations.

A third significant deviation occurs during the initial part of the force release phase. In this experiment, grip strength was measured using a traditional method where subjects grip a sensor. Given practical conditions, it is possible that subjects might have removed their fingers from the sensor before completing the release phase (despite instructions to keep their fingers on the sensor throughout the test). As a result, even though the sensor detects zero force, the subject’s actual grip strength has not yet reached zero, and hand movements may not have fully ceased. Consequently, the sEMG signals at this point do not stabilize.

In such non-ideal scenarios, the model may interpret these sEMG signals as indicating that the subject is still exerting force. From a realistic simulation perspective, this outcome could be considered more accurate and reflective of the actual situation. However, from an idealized experimental perspective, where the sensor should have recorded zero force, this discrepancy creates a conflict between the sensor data and the model's predictions.

The experimental results indicate that increasing the number of neural network layers did not positively impact the fitting prediction outcomes. It is suspected that this increase led to overfitting, as evidenced by a noticeable drop in the regression R-value, reducing the model's sensitivity to irregular fluctuations in grip strength during the maintenance phase. Additionally, excessively long time delays also decreased the accuracy of the predictions. According to the study of Beck et al. only sEMG signals from a specific previous time period affect current muscle strength changes [24]. Thus, a time delay closer to the length of this period yields more accurate results. Conversely, too short a time delay leads to insufficient sample size for analysis, while too long a delay results in redundant information, complicating the assessment of current muscle strength. Thus, for this experiment, 10 layer and 20 time delay are the best parameters.

Prediction of dynamic grip strength based on sEMG has notable potential for practical applications. For example, in the realm of smart prosthetics, it could enhance the ability to simulate real human grip strength, thus improving the precision of tasks requiring accurate grip control. In virtual reality, it can accurately capture and reflect human posture and force, leading to more effective interactions with virtual environments and better physical simulations within these virtual settings.

6. Conclusion

This study successfully applied the time series-based NARX network to fit and predict the nonlinear dynamic changes in upper limb grip strength based on sEMG signals. This represents a small step forward in exploring the decoding of force from sEMG signals, though there remains considerable room for improvement and further research.

To address the initial instability in prediction results, a lead time could be incorporated in practical applications, allowing the model to adapt to the user's sEMG signals before fully engaging in prediction tasks. For addressing prediction deviations caused by prolonged grip strength maintenance, further investigation into muscle fatigue phenomena is warranted. Collecting training data with extended grip maintenance times could provide more sEMG signals and grip strength information under fatigue conditions, aiding the model in analyzing the mapping between sEMG signals and muscle strength in such environments.

To correct prediction deviations during the initial part of the release phase, an output detection threshold could be established. Minor grip strength fluctuations that do not exceed this threshold could be classified as zero, thereby addressing the issue of the model predicting small grip strength values even after force release is completed.

The technology holds promising application prospects for nonlinear dynamic grip strength prediction. It has the potential to significantly impact the development and enhancement of smart prosthetics, as well as the simulation of physical scenarios and optimization of human-computer interactions in virtual reality settings.