Cybernetics of Balance Control

1. Introduction

Fighting with gravity is a common challenge for all terrestrial animals, including most mammals. It means, in particular, avoiding falls on the ground while performing daily tasks, such as locomotion or food foraging. The gravity challenge is absent in animal species with different habitats (e.g., aquatic, amphibian, aerial, arboreal), replaced by the challenge of dealing with different dynamics in the survival game.

The specific case of humans is quite peculiar because, after the appearance of “Lucy” (i.e., the Australopithecus afarensis) more than 3 million years ago, the rules of the game for fighting gravity had to be drastically changed, by uncoupling the upper limbs from a direct involvement in balance control and focusing their function in a more creative way, namely the endless variety of manipulation, tool use and tool invention. However, balance control in humans is not a unique paradigm, because it implies a wide variety of contexts and tasks, as illustrated in Figure 1.

In all paradigms, it is always possible to identify a real or virtual Inverted Pendulum (IP) linking two points: a point on the ground (the CoP: Center of Pressure) and the CoM (Center of Mass of the human body and connected tools). Two force vectors are applied to the two ends of the IP: the Body Weight is applied to the CoM and the Ground Reaction Force (GRF) to the CoP. The GRF is the resultant of all the contact forces between the body and the ground. By definition, the stabilization of the body requires that the CoP be actively maintained within the support base, defined as the convex hull of all contact points between the body and the ground. In addition to the Body Weight and the GRF, the IP may be subjected to other generalized forces as a function of the balancing paradigm: for example,. a force applied to the CoP (as in the stick balancing task), a torque applied to the CoP (as in the bipedal standing) or a torque applied to the CoM (as in the tasks that exploit the gyroscopic effect), etc. The purpose of Balance Control Systems is to prevent falling, which requires dynamically aligning the CoP and the CoM along the common vertical. From the controller’s viewpoint, the CoM is the controlled variable. The CoP is the control variable, unless the CoP position is constrained because the support base is very small or practically null, as in monopodal standing by ballet dancers or tightrope balancing by funambulists. In any case, the variety of balance strategies can be reduced to two basic paradigms: The CoP strategy and the CoM strategy.

In the CoP strategy, where the CoP is the control variable, the CoP chases the CoM, anticipating its oscillations by sliding the CoP inside the support base: for example, in the case of bipedal upright standing or handstanding, the CoP sliding is induced by modulating the ankle or wrist torque; in the case of stick balancing on the fingertip, the same result is obtained by moving the hand, thus physically sliding the contact point.

In the CoM strategy, the CoM is simultaneously the control and the controlled variable. This strategy is imposed by the fact that the area of the support base is physically strongly limited, as in the case of tight-rope walking: in this case, the CoP position is virtually fixed and the CoM position is controlled indirectly by exploiting the kinematic redundancy of the human body through compensatory gestures that keep the body CoM as close as possible to the vertical from the CoP.

The two balance strategies are implemented by combining different control paradigms:

• Opportunistic control paradigm: exploitation of a dynamic, physical phenomenon as the gyroscopic effect.
• Stiffness control paradigms: exploiting the elastic properties of skeletal muscles.
• Feedback control paradigm: measuring an incipient fall index and closing the loop in real-time.
• Feedforward control paradigm: exploiting an internal body model for generating statically stable whole-body synergies in an anticipatory manner.

A significant portion of research on balance control, for example, clinical posturography of upright standing, is based on a pure inverted pendulum model of body biomechanics (a single-degree-of-freedom IP model). In particular, the underlying assumption is often that a simple reflex mechanism, similar to the stretch reflex or the pupillary reflex, can explain the stabilization of postural sway. Although the IP model is a biomechanical simplification acceptable in limited situations, human balance control involves ann extensive family of sensory-motor-cognitive paradigms that require biomechanical models of adequate complexity, multisensory data fusion for extracting specific balance-relevant information, coordination of multiple DoFs (Degrees of Freedom), and anticipatory capabilities for self-generated postural disturbances.

Moreover, there is ground to believe that a standard computational structure underlies the different balance paradigms, illustrated in Figure 1, independent of the specific tasks and involved groups of muscles, in agreement with the pioneering studies on Enactivism, Embodied Cognition, and Self-Organization of living organisms [1,2], leading to the formulation of cognitive neuroscience. The fundamental “cybernetic” concept is that cognition arises through a dynamic interaction between an acting organism and its environment, mediated by a sensing and acting body and computationally organized as an internal model of synergy formation and control (Figure 2). It is also suggested that the computational analysis of this internal model can be guided by the general framework of Cybernetics [3], i.e., the set of computational methodologies useful for understanding “control and communication in the animal and the machine” in general.

The central claim of this review is to highlight how biomechanics and control strategies are intermingled in the acquisition and consolidation of balance skills, namely the large family of unstable tasks, ranging from activities of daily life to funambolic actions. In particular, the review focuses on the pros and cons and the intrinsic limitations of the stiffness strategy, the crucial importance of the feedback delay for closed-loop stabilization of postural oscillations, the rationale of intermittent vs. continuous feedback control, and the necessity of an internal body model for the anticipatory prediction and compensation of self-generated disturbances of the stability of purposive actions.

2. Opportunistic Control Paradigms

Spinning on ice by a professional skater is undoubtedly impressive. Still, this human balancing task exploits a simple dynamic, physical phenomenon known as the gyroscopic effect, which explains as well the passive stabilization of a spinning top (Figure 3): this effect expresses the tendency of a rotating body to maintain a steady direction, resisting a perturbing influence as the tendency to fall due to gravity. More specifically, the gyroscopic effect is that any torque tending to incline the axis of the rotating body in a given direction actually causes an inclination of the axis in a plane perpendicular to that direction, modifying the direction of the perturbation and ultimately settling into a rotatory motion of the axis. This is a passive effect that does not require active control for balance. In the case of the spinning ice skater, active control is not needed for stabilizing the upright posture but only for modulating the spinning speed (by opening/closing the arms, i.e., by changing the moment of inertia of the body around the vertical axis) or for initiation/termination of the spinning by means of a specific skilled gesture. Thus, the control of balance while spinning is an opportunistic control paradigm in the sense that it exploits a physical effect with a minimum of active intervention: such a physical effect allows an artifact shaped by human craft (e.g., a spinning top) to behave in a very similar way without any active control whatsoever.

It is worth observing that the gyroscopic effect is not the only possible opportunistic paradigm adaptable for balance control. For example, we may consider passive dynamic walking [4], namely the capacity of a class of purely passive two-legged machines for which walking is a natural dynamic mode, without control or energy input: once positioned on a shallow slope or on a treadmill, the external stimulation due to gravity is sufficient to evoke the stepping patterns typical of human walking. A similar pattern of quasi-passive stepping behavior is observed in decerebrated cats, positioned on a treadmill [5], which serves as the triggering source. In both cases, the dynamic stability of locomotion is achieved primarily by the opportunistic exploitation of the dynamics of falling around the standing leg; in contrast, the swinging leg prepares the body to stop the fall.

Another example of an opportunistic control paradigm is the light-touch effect [6], which affects the stabilization of the standing body in the upright position. The point is that such an apparently trivial task is a multi-factorial dynamic process, with persistent sway patterns whose amplitude is systematically increased by specific environmental conditions: 1) Eliminating a sensory channel (e.g., vision); 2) Decreasing the size of the support base (e.g., tandem position); 3) Reducing the stiffness of the support base (e.g., foamy surface). In all such cases, the sway amplitude can be decreased substantially by an additional source of feedback: light-touch, i.e., a very light finger contact with an external, stable object, where the amplitude of the contact force is irrelevant from a biomechanical point of view but plays the role of a crucial ecological information source: 1) Improving the accuracy of the online estimation of the sway angle; 2) Decreasing the delay in the delivery of such an estimate. Both effects are intrinsically related to the general concept of opportunistic control because they express the (cognitive) exploitation of the physical relationship between the human body and the environment.

3. Stiffness Control Paradigms

The stiffness control paradigm exploits the non-linear elastic properties of skeletal muscles, i.e., the fact that the slope of the length-tension curve of a muscle, for a given length, increases with neural activation. The biomechanical effect is that if we consider any joint of the human body, the coactivation of antagonist muscles determines an increase in joint stiffness, i.e., the ratio between a disturbing torque and the corresponding joint rotation, thus improving the resistance of the joint to the disturbance as well as the stability of the joint posture. This strategy has pros and cons that motivate and guide its adoption in certain conditions, as well as physical/biomechanical constraints that may rule it out in others.

Let us now explore the range of situations that may induce the brain to adopt or refuse such a strategy, starting with the simple case of balance control in quiet upright standing. The analysis is related to the SIP (Single Inverted Pendulum) model (Figure 4), which approximates the standing body as a rigid, elongated structure with a single articulation at the ankle and a pair of antagonistic muscles: gastrocnemius and tibialis anterior. As explained below, although this approximation is inaccurate, it is still accepted for many practical purposes, e.g., in the clinical domain.

In the SIP model, the toppling gravitational torque

$${\mathrm{T}}_{\mathrm{g}}=\mathrm{M}\mathrm{g}\mathrm{h}\mathrm{s}\mathrm{i}\mathrm{n}\left(\mathsf{\theta }\right)~\mathrm{M}\mathrm{g}\mathrm{h}\mathsf{\theta }$$

(1)

is contrasted by the restoring elastic torque applied at the ankle

$${\mathrm{T}}_{\mathrm{a}}={\mathrm{K}}_{\mathrm{a}}\mathsf{\theta }$$

(2)

$M$ is the body mass, $h$ is the distance of the CoM from the ankle joint, $\theta$ is the sway angle, and $K_a$ is the ankle stiffness resulting from the coactivation of the calf and tibialis anterior muscles. The Stiffness Stabilization Strategy exploits the elastic properties of the ankle muscles. It consists of modulating the coactivation level of antagonistic muscles, because coactivation induces an increase in ankle stiffness, in such a way to achieve a value of stiffness greater than the rate of growth of the gravity toppling torque:

$${\mathrm{K}}_{\mathrm{a}}>\mathrm{M}\mathrm{g}\mathrm{h}$$

(3)

Thus, $Mgh$ is the critical value of ankle stiffness for implementing the stiffness stabilization strategy. This strategy was proposed by Winter et al. [7], based on a computational procedure for evaluating ankle stiffness that grossly overestimated its value, as demonstrated by other studies [8,9,10] that showed that the real value of ankle stiffness is under-critical. In particular, such concludion was achieved by direct methods, based on the analysis of postural response to small perturbation: 1) one method, using tiny perturbations of 0.05 deg with a 70 ms duration [9], estimated that the ankle stiffness is 91% of the critical value; 2) another method, using larger perturbations of 1 deg amplitude and 150 ms duration [10] estimated the stiffness at the 69% level. The second study also found that the perturbation evokes no stretch reflex, and that human subjects do not coactivate the ankle muscles.

In any case, it is worth noting that although the stiffness strategy is not sufficient per se to achieve a stable standing posture, experimental results suggest a hybrid control mechanism that integrates the passive stabilization effect, due to the intrinsic physiological properties of muscle tissue, with an active closed-loop mechanism. In particular, Figure 5 shows the typical unperceived and unconscious oscillations of the SIP model (postural sway) in terms of the time course of the CoP (the position of the ground reaction force) and the CoM (the projection of the body’s center of gravity onto the ground). The CoM curve is proportional to the time course of the toppling torque $T_g$ and the CoP curve is proportional to the torque $T_a$ provided by the ankle muscles, which includes a passive component (due to the ankle stiffness) and an active component (due to a sequence of small muscle commands).

In summary, the stiffness strategy, which provides an instantaneous feedback stabilization effect, accounts for only part of the human balancing process in upright standing; the remaining part is due to a feedback control mechanism driven by a delayed feedback signal, as explicitly discussed in the following. Before that, let us answer two related questions that allow to clarify how the brain can achieve balance control in a task-related and environment-related manner:

• Is the brain unable or unwilling to modulate the intensity of the stiffness strategy to a level that achieves complete stabilization in bipedal upright standing, and why?
• Is there evidence of the use of the complete stiffness stabilization strategy, based on over-critical levels of joint stiffness, in different contexts?

3.1. Unfeasibility of Complete Ankle Stiffness Stabilization Strategy for Upright Standing

The ankle joint is characterized by the presence of the Achilles tendon (AT), which has unique biomechanical properties. It is the thickest tendon in the human body and has the capacity to withstand large tensile forces [37]. Moreover, it is long and therefore relatively compliant, with a stress-strain curve that is approximately linear, except at very low strains. The AT is serially connected to large and powerful calf muscles (the gastrocnemii and the soleus), which, like all skeletal muscles, are characterized by highly nonlinear length-tension curves, whose compound stiffness increases with tension.

The crucial point is that in a serial connection of two elastic elements (e.g., a tendon and a skeletal muscle), the compliances add. Thus, the overall stiffness is dominated by the more compliant element. As a consequence, the brain does not adopt the stiffness strategy for the stabilization of upright standing because even at the highest coactivation level of the ankle muscles the ankle stifness would not overcome the critical level: at the normal activation level of the ankle muscles the overall ankle stiffness, which includes the combined influence of muscles and connecting tendons, is too low, as demonstrated by the previously mentioned direct measurement methods [9,10]; moreover, even increasing the coactivation level of the ankle muscles would have a minor effect on the ankle stiffness due to the high compliance and linear elasticity of. AT.

The peculiar biomechanical arrangement of a stiff muscle serially connected to a compliant tendon explains what has been described as paradoxical muscle movements [11] during postural sway. This means that the ankle oscillations of postural sway are characterized by the fact that the muscles and the body, on average, tend to move in opposite directions, namely in a counterintuitive manner compared to the typical neuromuscular activation observed in motor control. Such paradoxical patterns help us understand the posturographic patterns of Figure 5, namely the interplay between the CoM and CoP curves. The CoM curve reflects, at the same time, the time course of the sway angle and the length of the AT; occasionally, the CoP curve overcomes the CoM curve, and this is determined by a specific muscle command that shortens the muscle to pull back the stretched tendon and thus break down the incipient fall. In other words, the brain does not exploit the AT for its stiffness but for its capacity to transmit to the body the occasional contractions of the calf muscles, determined by an intermittent feedback control mechanism. As explained in the following question.

At first sight, the peculiar arrangement of the compliant AT with the stiff calf muscle, which makes the complete compliant strategy impossible and forces the brain to adopt a rather complex control stabilization strategy, may appear as a kind of evolutionary misstep. However, we should also consider the ecological motivation behind such a peculiar arrangement from a broader perspective, for example, the problem of making the landing after a high jump or a fall as safe as possible. At the moment of touching down, the falling body has kinetic energy proportional to its falling height, and this energy should be dissipated in the safest possible way. The critical issue is that most of this energy will be absorbed by the elastic properties of the AT+calf complex and follows the law that, in a serial connection of two elastic elements of different compliances, the distribution of absorbed energy is inversely proportional to the compliance of the elements. Thus, at touchdown, the most significant part of the energy will be dissipated by the tendon rather than the muscle, and this is advantageous from the physiological point of view because the muscle tissue is more fragile than the tendinous tissue. In a sense, the compliant AT has the function of a “biomechanical fuse”, protecting from rupture the calf muscles. On the other hand, the availability of such a safety function is paid for by forcing the balance control to organize a complex feedback stabilization mechanism to compensate for the insufficient ankle stiffness.

3.2. Full or Mixed Stiffness Stabilization Strategies of the Upper Limb

We already observed that the peculiar arrangement of a compliant tendon with a stiff muscle that characterizes the ankle joint is an exception in the human body. In the arm joints, for example, the elbow (Figure 6, left panel), both flexor and extensor muscles are connected to the skeleton by short, stiff tendons that transmit muscle forces directly, without the paradoxical patterns described in the previous section. In this condition, the postural stability of the elbow can be provided by the coactivation of the antagonistic elbow muscles: the higher the coactivation level, the stronger the elbow stiffness, capable of exerting a stabilizing effect on the equilibrium posture in response to external perturbations [12]. This reaction is passive, in the sense that the intrinsic properties of skeletal muscles determine it without the need for an active control loop. The active contribution of higher neural mechanisms primarily involves selecting the coactivation level as a function of the expected or estimated intensity of the external disturbance that may perturb the intended posture.

Postural stabilization of the upper limb is investigated chiefly from the point of view of the end-effector rather than the individual joints. The relation between a displacement disturbance from the equilibrium point and the restoring elastic force vector characterizes the end-effector stiffness. In the case of planar movements, this relationship is expressed by a symmetric stiffness matrix and visualized by a stiffness ellipse, whose principal axes are the eigenvectors of the matrix and the lengths are the corresponding eigenvalues. The stiffness ellipse is typically measured using a robotic manipulandum that applies small perturbations to an equilibrium point in different directions and measures the resulting elastic force vectors. Figure 7 (right panel) is an example of a stiffness ellipse evaluated for a given posture and with a spontaneously chosen level of coactivation of the three joints involved in the experiment (shoulder, elbow, wrist). Following this experimental approach [13,14], it was found that the neuromuscular control of the human hand stiffness is organized as follows:

• It is strongly anisotropic, i.e., the elastic interaction with external perturbation depends on the perturbation direction.
• Anisotropic patterns are strongly dependent on the hand’s position in the workspace.
• The size of the stiffness ellipses is linearly modulated by the degree of coactivation of the arm muscles.

The experimental characterization of hand stiffness using a robotic manipulandum served as the basis for experiments testing the organization of neuromuscular control during the stabilization of unstable tasks. Although such experiments are somewhat artificial, because the source of instability is simulated by a divergent force field generated by a manipulandum, they are nonetheless representative of many common everyday tasks that are intrinsically unstable. For example, keeping a screwdriver in the slot of a screw is unstable because excessive force parallel to the slot can cause the screwdriver to slip, and because misdirected force can cause loss of contact between the screwdriver and the screw. Such instability can be compensated for either by stiffness, feedback, or feedforward. Each strategy is characterized by pros and cons:

• The stiffness strategy is the simplest because it leverages the muscles’ intrinsic properties, making it instantaneous. On the other hand, it is fatiguing because prolonged coactivation of antagonist muscles is energetically demanding. Moreover, this strategy may not be applicable in certain conditions, such as the ankle joint during upright standing.
• The feedback strategy, i.e., the feedback control of an unstable posture using the sensory estimation of the displacement from an equilibrium point in a closed control loop, has a clear is Achilles’ heel, namely the extent of the delay of the feedback signal in comparison with the “falling time constant” that characterizes the instability.
• The feedforward control of instability is the most complex strategy because it requires a cognitive understanding of the overall situation and the selection of a body posture most likely to reduce the “danger of failing”. This strategy, which involves reasoning and learning, allows balance control to generalize, expanding the range of unstable tasks that can be mastered. In particular, it requires careful modulation of key parameters, including stiffness and/or feedback components.

3.2.1. An Example of an Unstable Manipulation Task Solved with a Stiffness Strategy

An example of an unstable manipulation task consists of a point-to-point reaching movement of the arm while the hand must compensate for the destabilizing disturbance of a divergent force field. Among the experimental studies thathave addressed this issue,, let us consider consider the fwing [15]: a subject was asked to move his hand from an initial to a final equilibrium position while holding a parallel-link direct-dair-magnett floating manipulandum. As soon as the movement started, the following force field is activated, which pushes the hand sideways, with an intention increasing with the lateral deviation from the nominal forward trajectory:

$$\{\begin{array}{c}{\mathrm{f}}_{\mathrm{x}}={\mathrm{K}}_{\mathrm{D}\mathrm{F}} \left(\mathrm{x}-{\mathrm{x}}_0\right)\\ {\mathrm{f}}_{\mathrm{y}}=0 \end{array}$$

(4)

Figure 7 illustrates the experimental situation: the force field is null along the longitudinal $y$ coordinate and divergent along the lateral $x$ coordinate: $x_0$ is the position at which the divergent field originates, with a rate of growth modulated by the $K_{DF}$ control parameter.

The difficulty of the task depends on the $K_{DF}$ parameter in comparison with the intensity of the hand stiffness in the direction of the divergent field. The rationale of the experiment is to select a value of $K_{DF}$ which is big enough to challenge a naïve subject to the point of failing the task in a preliminary familiarization phase, but not too big, in the sense of ruling out the chance that the subject will succeed in carrying out the reaching skill after suitable training. It is known that in a normal, relaxed condition, the stiffness ellipse is oriented approximately along a line that links the shoulder to the hand [13,14]. Since a naïve subject initially fails to reach the target, the following must hold:

$$\mathrm{I}\mathrm{n}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{a}\mathrm{l} \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}:{\mathrm{K}}_{\mathrm{D}\mathrm{F}}>{\mathrm{K}}_{\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{d},\mathrm{x}}$$

(5)

where $K_{hand,x}$ is the lateral component of the hand stiffness ellipse. The quoted experiments [15] show that, after training, the subjects succeed in reaching the target using the stiffness strategy, i.e., by carefully increasing the coactivation of the arm muscles. This means that the final condition exhibited after training is expressed as follows:

$${\mathrm{F}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l} \mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}:\mathrm{K}}_{\mathrm{D}\mathrm{F}}<{\mathrm{K}}_{\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{d},\mathrm{x}}$$

(6)

Moreover, such a stiffness modulation is characterized by a smart rotation of the stiffness ellipse, which is approximately horizontal, i.e., aligned with the field.

One may wonder whether adopting the stiffness strategy is a free choice or is forced by the specific organization of the experimental setup. The authors of the quoted experiment did not address this issue. However, it is suggested that a tentative answer can be derived by evaluating the time constant of the incipient sideways “fall” of the hand in the initial condition described by Equation (5), according to the following formula, where $M_x$ is the horizontal component of the inertia matrix (body plus manipulandum mass), in analogy with the stiffness matrix:

$$\mathrm{T}=\sqrt{{\displaystyle \frac{{\mathrm{M}}_{\mathrm{x}}}{{\mathrm{K}}_{\mathrm{D}\mathrm{F}}-{\mathrm{K}}_{\mathrm{h}\mathrm{a}\mathrm{n}\mathrm{d},\mathrm{x}}}}}$$

(7)

With reasonable parameter estimates, the time constant is approximately 70 ms, which is significantly smaller than the delay of the potential stabilizing feedback signals. This means that the instability induced by the destabilizing force field is too rapid for a feedback compensation mechanism to respond promptly during the execution of the reaching task, which typically lasts 1 s. Thus, in our opinion, this is why the subjects in the reported experiment had no chance to solve the task using a feedback strategy and were forced to use a full stiffness strategy.

3.2.2. An Example of an Unstable Manipulation Task that Allows a Choice of the Stabilization Strategy

In a practical application, the unstable manipulation task considered in the previous section may imply the use of a hand-held tool, e.g., a rigid pointer, wth the goal of touching or pushing a target device with great precision in a noisy environment that tends to deviate laterally the trajectory of the tool, thus making the task unstable. The results of the experiment [15] show that the subjects learn to achieve the goal using a pure stiffness strategy.

In this section, let us consider an extension of such an unstable manipulation task by assuming that the tool is not held rigidly but indirectly via an elastic holder whose stiffness can be modulated. The experimental setup developed for this purpose [16] is illustrated in Figure 8. It consists of a bimanual haptic interface that includes a pair of direct-drive, low-friction, low-weight manipulanda whose handles are grasped by the two hands of a subject. The handles are linked to a virtual tool, represented as a virtual mass, through two virtual springs with non-linear elastic properties. The subject’s task, similar to the experiment illustrated in the previous section, is to move the virtual mass from an initial equilibrium position to a designated target.

In contrast, the virtual mass is disturbed by a partially unstable force field, divergent along the horizontal axis and convergent in the vertical axis. The virtual springs are nonlinear and exhibit a quadratic stress-strain relationship, with spring stiffness increasing linearly with elongation. Thus, the dynamics of the bimanual haptic interface is determined by the following parameters: the target mass, the spring parameters, and the force field parameters. For each time instant, the subject receives a visual and a haptic feedback of the ongoing reaching task; the visual feedback is the position on a screen of the two hands, the virtual mass, and the target to be reached; the haptic feedback consistes of two force vectors applied to each hand by the corresponding manipulandum as a function of the stretched virtual spring. Moreover, the motion of the virtual mass is simulated as a function of the global force resulting from a combination of three virtual force vectors: the right spring, the left spring, and the unstable field.

The setup parameters were chosen to allow the subjects to select different stabilization strategies. For example, suppose the mass must be stabilized at the workspace center, where the field intensity is zero. This implies that two hands must be positioned symmetrically with respect to the origin e.g., symmetric with respect to the horizontal axis, regardless of their lateral separation. From such an equilibrium position, any minor disturbance may displace the virtual mass on one side or the other slightly, exposing it to the destabilizing effect of the divergent field, parametrized by the rate of growth $K_{DF}$. The potential lateral fall of the mass may be compensated for by the combined stiffness of the two springs $K_{SP}$, taking into account that such stiffness is not constant but linearly grows with the lateral separation of the hands, referred to the field origin. Thus, if such a distance exceeds the critical level for which $K_{SP}> K_{DF}$ then the mass is stabilized according to the Stiffness Stabilization Strategy (SSS). In the implemented experimental setup, the critical separation, which corresponds to the critical coactivation level considered in the previous section, is 11 cm. Moreover, the same strategy can be replicated for stabilization in different parts of the work area as well as for carrying out a point-to-point motion of the virtual mass. In summary, the SSS strategy is simple, although such simplicity is achieved at the cost of muscle fatigue: at the critical separation distance, the two hands must deliver a force of 50 N to the corresponding handle, namely a pretty challenging task if the subject is requested to carry out a sequence of targeting actions.

As an alternative to the fatiguing SSS, the subject may decide to reduce the separation distance, for example keeping the two hands close to each other. Starting from an equilibrium condition in the center of the workspace, an occasional disturbance my trigger an incipient fall on one side or the other, thus evoking a stretch of the virtual spring on the opposite side. The stabilization strategy may simply consist of the generation of an activation burst of the affected hand proportional to the perceived displacement, including the corresponding speed, of the virtual mass from the equilibrium position: in other words, this procedure identifies a positional/derivative feedback strategy (PSS) similar to the PD controller design adopted in engineering servomechanisms. It is well known that the stability of this design is critically dependent on the delay of the feedback signals relative to the intrinsic dynamics of the controlled system, especially if the controlled system is itself unstable, as in an IP model. In particular, the PSS strategy has a chance to succeed if and only if the falling time constant of the IP, or an equivalent unstable plant, exceeds the delay of the feedback signals by at least a margin. In the experimental setup considered above, the sensory delay comprises a visual and a haptic component, with a magnitude of 200-250 ms. Thus, to give the PSS strategy a chance, the dynamic parameters of the haptic interface were computed to yield a falling time constant of around 300 ms.

The experimental results of this study [16] show that naïve subjects, after a training period necessary to overcome the destabilizing effect of the force field and to perform the task repeatedly, spontaneously choose one strategy or the other, with 60% SSS and 40% PSS.

Figure 9 shows the performance of an SSS subject (left panel) and a PSS subject (right panel). In both cases, the virtual mass is initially at the center of the workspace; the task is to push the mass to one of the eight targets identified by grey circles with a 1 cm radius, located 4 cm from the initial position. In particular, the selected target in the figure is the one aligned along the y-axis. The SSS subject initially keeps the two hands symmetrically stretched along the x axis, with a separation of more than 11 cm; after the start, the subject moves both hands forward, with a larger extension than the motion of the mass, because at the end of the bimanual action, the two springs are required to compensate the vertical, convergent component of the force field. The PSS subject keeps the two hands very close together throughout the motion. The y component of the hand motion is a common, progressive forward shift toward the target position; in contrast, the x component is oscillatory with a minimal amplitude, similar to the sway patterns observed during upright standing.

After the training, both groups of subjects were informed of the two strategies and quickly learned to switch between them on command [17]. We suggest that this result is relevant from an ergonomic perspective, both for designers of devices that incorporate an unstable aspect and for the training of operators who are expected to use them effectively. The following section discusses two possible implementations of the PSS strategy, based on a continuous or intermittent delayed feedback control.

4. Feedback Control Paradigm for the CoP Stabilization Strategy

In the CoP stabilization strategy, the CoM is the controlled variable, and the CoP is the control variable: the modulated motion of the CoP is supposed to generate an antigravity torque ${\tau }_a$ to compensate for the destabilizing torque ${\tau }_g$ due to gravity. The competition between such torques is usually analyzed in relation to a Single Inverted Pendulum model (SIP). Here, we focus our attention on two paradigmatic cases, equivalent from the cybernetic point of view, although quite different regarding physiological and biomechanical details: standing and stick balancing on a fingertip (Figure 10). The extension of the analysis to a multi-joints model will be considered later.

Considering that, in both paradigmatic cases, the goal is to keep the CoM and CoP as aligned as possible along the same vertical, the difference lies in the relationship between the CoP motion and the antigravity torque. In upright standing, the direct control variable is ${\tau }_a$, as the sum of a passive torque, due to muscle stiffness, and an active torque, due to closed-loop ankle muscle contraction: the CoP displacement is the biomechanical consequence of the combined ankle torque. In the case of stick balancing, the direct active control variable is the CoP position, and the antigravity torque is a biomechanical consequence, with a null active torque at the fingertip contact point. In any case, the two paradigms are equivalent from a cybernetic point of view, as illustrated in the control diagram of Figure 11, which summarizes the interplay among the interacting processes in the balancing mechanism. The following equation characterizes the physical plant of the SIP model:

$$\ddot{\mathrm{q}}={\displaystyle \frac{\mathsf{\tau }}{{\mathrm{I}}_{\mathrm{s}\mathrm{i}\mathrm{p}}}}$$

(8)

where $I_{sip}$ is the moment of inertia of the inverted pendulum, around the ankle or fingertip point, and $\tau$ is the total torque acting on it. This torque includes three main components: the destabilizing torque due to gravity (${\tau }_g$), the stiffness control torque (${\tau }_{sc}$), and the feedback control torque (${\tau }_{fc}$). All of them are functions of the SIP state (sway angle and the corresponding velocity); thus, the control diagram is a multiple closed-loop system. However, while the gravity and stiffness loops are instantaneous because they express physical/physiological features, the PD (Proportional-Derivative) control mechanism is driven by the feedback sensory evaluation of the postural oscillation, which is delayed as shown in the following equation, where $\delta$ is the delay time:

$${\mathsf{\tau }}_{\mathrm{f}\mathrm{c}}\left(\mathrm{t}\right)=\mathrm{P}\mathrm{ }\left(\mathrm{q}\left(\mathrm{t}-\mathsf{\delta }\right)-{\mathrm{q}}_{\mathrm{r}\mathrm{e}\mathrm{f}}\right)+\mathrm{D}\mathrm{ }\dot{\mathrm{q}}(\mathrm{t}-\mathsf{\delta })$$

(9)

As in standard servomechanism design, the PD controller in Figure 11 operates in continuous time with delayed sensory feedback. If the delay $\delta$ is small in comparison with the intrinsic time constants of the plant, the tuning of the PD parameters for implementing a stable and precise performance is straightforward and robust. In the opposite case, the amount of the delay becomes the Achilles’ heel of the feedback PD controller, determining a high risk of instability in the balance process, even more so since the physical plant (the SIP model) is unstable to start with. In both paradigms of Figure 10, the delay is significant enough to make negligible the chances of success of the continuous control strategy in Figure 11.

In the next sections, it is shown that a modification of the general control diagram of Figure 11, based on the adoption of an intermittent PD controller instead of a continuous one, can solve the problem. Before that, let us consider the physiological origin of the delay mentioned above. First of all we may say that such a (rather significant) delay has nothing to do with short-latency reflexes, as the monosynaptic stretch reflex, whose onset is 30-40 ms [18], or the pupil light reflex [19] with a much larger onset (230-320 ms), or the reaction time in professional sports [20], whose legal value according to IAAF rules is 100 ms.

The delay in the block diagram of Figure 11 reflects a complex, multimodal computational process that is not intended to trigger a pre-programmed response, such as the start of a 100 m athletic run, but is designed to continuously supply the controller with information crucial for maintaining equilibrium. In upright standing, where proprioceptive, visual, and vestibular sensing are actively integrated, it is commonly accepted that (is at least 200 ms); in stick balancing, where proprioceptive information is probably less relevant, (is larger (230 ms or more). Moreover, upright standing offers different ways to evidentiate the complex structure of the computational process that estimates on line the current value of the postural state vector: the Romberg coefficient that quantifies the effect of eliminating the visual feedback on postural stability [21] or the light touch effect [6] that demonstrates how postural stability is surprisingly improved by providing the brain with virtual information on the sway movements.

4.1. Intermittent Feedback Stabilization for the CoP Strategy: The SIP Model

The presence of a considerable delay in the feedback loop of the PD feedback controller is a severe problem for achieving stability if the feedback control signal (${\tau }_{fc}$ computed according to equation 9) is delivered continuously in time, as suggested by [22]. When setting up this kind of controller, the designer must tune the gains in such a way as to satisfy the different requirements of overall stability, speed, and damping of the transient responses, as well as asymptotic precision. The stability of the feedback controller proposed by Peterka for upright standing [22] was analyzed using the classical frequency-domain Bode plots, which show the dependence of the magnitude and phase of the open-loop transfer function on frequency [23]. In particular, from the phase plot, the Nyquist stability criterion allows computing the phase margin as the frequency at which the response magnitude crosses 0 dB. In the Peterka model, neglecting the delay in the control loop, the margin is 37.6 degrees, which is already smaller than what is acceptable in typical engineering designs. However, this margin is reduced to a meager 17.3 degrees at a 100 ms delay, and the system becomes openly unstable at a 200 ms delay.

The alternative to continuous-time PD delayed feedback control is intermittent control, where the PD delayed feedback control signal is alternately switched on and off (Figure 12, left panel). Switching is not clock-driven but is locked to the phase-plane instability that characterizes the SIP model: a Phase Detection module continuously monitors the evolution of the delayed SIP sway in the phase plane (SIP angle/CoM position vs SIP angular speed/CoM speed). It exploits the saddle point instability of the SIP dynamics (Figure 12 (right panel). The plotted phase portrait shows that the phase plane can be split in four regions: if the state vector is inside the red (unstable) regions the evolution from that time instant will be to a final fall in the absence of active control; the behavior of the SIP model in the green (metastable) regions is the opposite, i.e., the state trajectories tend to converge to the nominal equilibrium point in the center of the phase plane at least temporarily until the trajectory crosses the line between a metastable and unstable region. Therefore, we can say that such a structure of the phase portrait is a “dynamic affordance” that can be exploited by the following intermittent control policy that closes the loop (on-phase), when the state vector is recognized to enter a dangerous (red) region and opens the loop (off-phase), when entering a safe (green) region:

$$\begin{array}{c}\mathrm{O}\mathrm{n}-\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\mathrm{q}\left(\mathrm{t}-\mathsf{\delta }\right)\left(\dot{\mathrm{q}}\left(\mathrm{t}-\mathsf{\delta }\right)-\mathrm{a}\mathrm{q}\left(\mathrm{t}-\mathsf{\delta }\right)\right)>0\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}{\mathsf{\tau }}_{\mathrm{f}\mathrm{c}}=\mathrm{P}\mathrm{q}\left(\mathrm{t}-\mathsf{\delta }\right)+\mathrm{D}\dot{\mathrm{q}}\left(\mathrm{t}-\mathsf{\delta }\right)\hfill \\ \mathrm{O}\mathrm{f}\mathrm{f}-\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\mathrm{q}\left(\mathrm{t}-\mathsf{\delta }\right)\left(\dot{\mathrm{q}}\left(\mathrm{t}-\mathsf{\delta }\right)-\mathrm{a}\mathrm{q}\left(\mathrm{t}-\mathsf{\delta }\right)\right)\le 0\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}{\mathsf{\tau }}_{\mathrm{f}\mathrm{c}}=0\hfill \end{array}$$

(10)

It is also worth emphasizing that, regarding the control signal, this control paradigm is intermittent, but continuous in terms of sensory monitoring, even though it operates on delayed feedback signals. Figure 13 shows the phase portrait of sway patterns from a simulation of the SIP model with PD feedback control implemented according to Equation (10) (right panel) compared with the typical sway patterns of a standing human subject (left panel).

It appears that experimental and simulated postural oscillations are pretty similar. The crucial point is that the model achieves bounded dynamic stability, even though approximately 50% of the time (off-phases) the SIP model is driven by its intrinsic, globally unstable dynamics. Moreover the observed phase portraits can be considered as a kind of limit cycle, in contrast to the expected behavior of a stable continuous-time controller, which would tend asymptotically to the equilibrium condition. The two PD feedback control paradigms (the intermittent and the continuous time one) differ also from two other respects: (a) the intermittent paradigm is much more robust, as regards parameter tuning of the proportional and derivative gains, than the continuous paradigm; (b) the size of the sway motion requires only a minimal level of noise in the simulation of the intermittent model in comparison to the exaggerated noise level of the continuous model. As regards the robustness of the intermittent control paradigm, we may consider that the limit cycle produced by the simulation consists of the an alternation of segments of hyperbolic orbits (off-phases) and spiral orbits (on-phases), as shown in Figure 14:

• Initiating an off-phase at time $t_0$ a hyperbolic orbit is started that follows the phase portrait of the SIP model, initially approaching the equilibrium point under the influence of the stable manifold (the green line); this orbit crosses the border of the dangerous region at $t=t_1$ and starts falling away under the influence of the unstable manifold (the red line), although such an indication of instability is detected with delay at $t=t_2$ ($t_2-t_1=\delta$).
• At $t=t_2$ the on-phase is started, turning on the PD feedback control signal (Equation (10)) that stops the micro fall, producing a spiral orbit around the equilibrium point, entering the safe region at $t=t_3$, although this event is detected with a delay at $t=t_4$ , ($t_4-t_3=\delta )$ thus terminating the on phase.
• At $t=t_4$ a new off-phase is started, and the state vector feels the stabilizing effect of the stable manifold (the green line) until $t=t_5$, when a new micro-fall initiated and so on.

It is worth noting that the bounded stability of the intermittent control model is achieved even if the spiral segments activated during the on-phases are potentially spiraling away from the equilibrium point and thus are unstable. The crucial point is that the purpose of the on-phase is not to aim at equilibrium but to terminate such a phase as close as possible to the stable manifold of the phase portrait (the green line in the phase plane). In the third panel of Figure 14, the on-phase is terminated slightly before crossing the green line, and thus the following hyperbolic orbit aims at the upper-right red region. If the spiral orbit had crossed the green line, then the hyperbolic orbit would have aimed at the lower-left red region instead. The noise intrinsic to the intermittent control mechanism, described by Equation (10) (both additive noise in the delivered torque and random jitter in the delay of detection of the switching time instants), could modify the exact sequence of on-phases and off-phases, producing a pseudo-random limit cycle.

Let us now summarize the biological plausibility of the intermittent control paradigm described above. Postural sway patterns over time (Figure 5) and the corresponding phase portrait (Figure 13 right panel) are characterized by several features [24]: (1) High, positive, zero-phased correlation between motions of CoM and CoP trajectories; (2) CoP peaks anticipate CoM peaks; (3) CoM sway is not driven by random noise around asymptotic equilibrium point; (4) Positive correlation between the CoP and EMG activity of the lateral gastrocnemius, with the CoP anticipating the EMG by 250-300 ms. Moreover, there are specific indicators of intermittency of the active control of the ankle signal [25,26,27]: (a) the histogram of the CoM positions with respect to the equilibrium position is bimodal in contrast with the Gaussian distribution typical of asymptotic equilibrium; (b) the graph of the cosine of the angle between the tangent to the phase portrait of sway movements and he corresponding tangent to the phase portrait of the SIP saddle instability switches systematically between 0 and 1; (c) the amplitude of the sway movements is incompatible with an assumption of continuous stabilization that implies asymptotic stability and thus must explain sway as a consequence of control noise. In conclusion, the rationale for the delayed feedback intermittent control paradigm rests on the dynamic affordance provided by the saddle-point instability of the SIP model, and we suggest that the brain detects and learns to exploit this feature, in agreement with a general principle of minimum intervention.

4.2. Intermittent Feedback Stabilization for the COP Strategy: The CIP Model

The stick balancing task is more challenging than the standard formulation of the stabilization of upright standing, related to the SIP model, because there is no contribution of stiffness: at the stick rotational point (analogous to the ankle joint in upright standing) no muscle-related torque is available, partially compensating the gravity-related destabilizing torque. Moreover, the sensory feedback required by the PD controller is fundamentally deprived of the proprioceptive channel and can rely only on the slower, less accurate visual channel. The dynamic analysis of this task is typically carried out using the Cart Inverted Pendulum (CIP) model, as shown in Figure 15.

This model has two degrees of freedom (the stick rotation $\vartheta$ and cart position $x$) and is stabilized by the control variable $f\left(t\right)$. The instability of the CIP is only related to the former DoF, whereas the control of the latter is aimed at limiting the range of motion (RoM). The dynamic equations of the CIP model can be derived with the Lagrange formalism. A linearized version [28], based on the assumption of small values of the angle and the angular velocity, is as follows:

$$\left[\begin{array}{c}\ddot{\mathsf{\theta }}\\ \ddot{\mathrm{x}}\end{array}\right]=\left[\begin{array}{c}{\mathrm{A}}_{11}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mathrm{A}}_{12}\\ {\mathrm{A}}_{21}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\mathrm{A}}_{22}\end{array}\right]\left[\begin{array}{c}\vartheta \\ f\end{array}\right] \{\begin{array}{c}{\mathrm{A}}_{11}={\displaystyle \frac{1.5\hspace{0.17em}(\mathrm{M}+\mathrm{m})}{\hspace{0.17em}(\mathrm{M}+0.25\hspace{0.17em}\mathrm{m})}}{\displaystyle \frac{\mathrm{g}}{\mathrm{L}}}\hspace{0.17em}\\ {\mathrm{A}}_{12}=-{\displaystyle \frac{1.5}{(\mathrm{M}+0.25\mathrm{m})\hspace{0.17em}\mathrm{L}}}\\ {\mathrm{A}}_{21}=-{\displaystyle \frac{0.75\mathrm{m}\hspace{0.17em}\mathrm{g}}{\mathrm{M}+0.25\mathrm{m}}}\\ {\mathrm{A}}_{22}={\displaystyle \frac{1}{\mathrm{M}+0.25\mathrm{m}}}\end{array}$$

(11)

The feedback PD controller is a function of both the stick state ($\theta \left(t-\delta \right), \dot{\theta }(t-\delta$)) and the cart state ($x\left(t-\delta \right), \dot{x}(t-\delta )$), with four PD parameters:

$${\mathrm{f}\left(\mathrm{t}\right)=\mathrm{P}}_{\mathsf{\theta }}\hspace{0.17em}\mathsf{\theta }(\mathrm{t}-\mathsf{\delta })+{\mathrm{D}}_{\mathsf{\omega }}\hspace{0.17em}\dot{\mathsf{\theta }}(\mathrm{t}-\mathsf{\delta })+{\mathrm{P}}_{\mathrm{x}}\hspace{0.17em}\mathrm{x}(\mathrm{t}-\mathsf{\delta })+{\mathrm{D}}_{\mathrm{v}}\hspace{0.17em}\dot{\mathrm{x}}(\mathrm{t}-\mathsf{\delta })$$

(12)

If we apply the PD control of Equation (12) to the plant of Equation (11) in continuous time, with a feedback delay estimated at 230 ms, there is no chance of stabilizing the stick [28]. In contrast, an intermittent PD delayed control policy, similar to the one previously considered for upright standing, has been demonstrated to succeed, as shown by Figure 15.

Figure 16. Simulation of the intermittent controlled CIP model for a stick length of 52cm and a feedback delay of 230 ms. Oscillations of the stick angle (left panel) and the CoP position (right panel).

Figure 16. Simulation of the intermittent controlled CIP model for a stick length of 52cm and a feedback delay of 230 ms. Oscillations of the stick angle (left panel) and the CoP position (right panel).

Similarly to the intermittent control model of Equation (10) the alternation between the on-phase and the following off-phase is triggered by the following equation:

$$\begin{gathered} \mathrm{O}\mathrm{n}-\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}:\mathsf{\theta }\left(\mathrm{t}-\mathsf{\delta }\right)\left(\dot{\mathsf{\theta }}\left(\mathrm{t}-\mathsf{\delta }\right)-\mathrm{a}\mathsf{\theta }\left(\mathrm{t}-\mathsf{\delta }\right)\right)>0 \\ \mathrm{O}\mathrm{f}\mathrm{f}-\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}:\mathsf{\theta }\left(\mathrm{t}-\mathsf{\delta }\right)\left(\dot{\mathsf{\theta }}\left(\mathrm{t}-\mathsf{\delta }\right)-\mathrm{a}\mathsf{\theta }\left(\mathrm{t}-\mathsf{\delta }\right)\right)\le 0 \end{gathered}$$

(13)

In particular, it has been shown [29] that asymptotic stability can be achieved provided that the feedback delay satisfies the following condition:

$$\delta <\sqrt{{\displaystyle \raisebox{1ex}{$L$}\!\left/ \!\raisebox{-1ex}{$g$}\right.}}$$

(14)

This means that, with a 230 ms delay, the intermittent control policy defined by equations 12 and 13 can stabilize a CIP model if the stick length is longer than 52 cm (the critical length). The control signal $f\left(t\right)$ provides a combined delayed feedback of both the stick angle and the CoP position as well as the corresponding time derivatives. The proportional/derivatives coefficients of the former type are critical for the stick stability. In contrast, the coefficients of the latter kind are meant to limit the range of motion of the CoP.

For sticks equal to or longer than the critical length of 52 cm, human subjects have no difficulty keeping the stick almost upright for an indefinite period of time without specific training, similar to the oscillatory yet robust stability of human upright standing. On the other hand, human subjects with appropriate training can successfully balance a stick of 30cm or shorter. To understand the nature of this limitation it is worth mentioning that, although the CIP model has two instead of only 1 DoF, Equation (11) clearly shows that, in the absence of control action, the motion of the pendulum is independent of the motion of the cart and such motion is characterized, in the phase plane of the pendulum, by an instability of the saddle type. That said, the question is: where is this limit coming from, and how can the basic form of the intermittent feedback control paradigm be modified to simulate it in a way that achieves the level of a trained human performer?

A working hypothesis is that the saddle-point instability of the CIP balancing task, even in the more challenging situation mentioned above, is too tempting an affordance for the brain to ignore. The next step, to go beyond the limits of the basic intermittent approach, is to understand the role and function of the two special lines in the phase plane, namely the stable and unstable manifolds of Figure 17, with a primary reference to the stable (green) one.

The crucial issue is that, in the off-phases of intermittent control, the timing of a hyperbolic orbit related to a given initial angular disequilibrium is strongly dependent on the distance of such an initial point from the stable manifold. This issue is illustrated in Figure 17 for two different CIP model configurations: a long stick with a short delay (left panel) vs. a short stick with a long delay (right panel). In all the simulations shown in the figure, the stick has an initial angular tilt at $t_0=t_{off}$ of -2 deg, but with a different initial angular speed, thus with a different distance above or below the green stable manifold. Each hyperbolic orbit is plotted with two parts;

• A blue part, from the initial time instant to the time instant $t_1$ at which the state vector $v={\left[\vartheta \dot{\theta }\right]}^T$ leaves the safe quadrant in the phase portrait and enters the unsafe quadrant;
• A red part, in the hazardous quadrant, until $t_2=t_{on}$, i.e., the time instant at which the activating condition of the On-phase is recognized. This means that, by definition, $t_2-t_1=\delta$.

Considering that the red part of each hyperbolic orbit has a fixed duration equal to the feedback delay $\delta$, the graphs of Figure 17 imply that the duration of the blue part ($∆t_{cross}=t_1-t_0$) is strongly variable: it depends both on the parameters of the CIP model (length and delay) and on the distance of the state vector $v_{off}={\left[{\vartheta }_{off} {\dot{\theta }}_{off}\right]}^T$from the stable manifold at the time of termination of the previous on-phase, i.e., it grows quickly with such a distance. In particular, it can be demonstrated [30] that the following relation characterizes all the points of the (green) stable manifold:

$$\left|{\displaystyle \frac{\dot{\mathsf{\theta }}}{\mathsf{\theta }\sqrt{{\mathrm{A}}_{11}}}}\right|=1$$

(15)

Thus, we may use the following definition of the distance (${\gamma }_{off}$) from the stable manifold of the initial state $v_{off}$ of the off-phase:

$${\mathsf{\gamma }}_{\mathrm{o}\mathrm{f}\mathrm{f}}=\left|{\displaystyle \frac{{\dot{\mathsf{\theta }}}_{\mathrm{o}\mathrm{f}\mathrm{f}}}{{\mathsf{\theta }}_{\mathrm{o}\mathrm{f}\mathrm{f}}\sqrt{{\mathrm{A}}_{11}}}}\right|$$

(16)

As a consequence, for the points on the stable manifold ${\gamma }_{off}=1$ such a distance is null; for the points above the stable manifold ${\gamma }_{off}>1$, and for the point below the manifold ${\gamma }_{off}<1$. Using this definition of distance, it is possible to compute the duration of ${∆t}_{cross}$ as follows [30]:

$$∆{\mathrm{t}}_{\mathrm{c}\mathrm{r}\mathrm{o}\mathrm{s}\mathrm{s}}={\displaystyle \frac{1}{2\sqrt{{\mathrm{A}}_{11}}}}\mathrm{l}\mathrm{n}\left({\displaystyle \frac{1+{\mathsf{\gamma }}_{\mathrm{o}\mathrm{f}\mathrm{f}}}{\left|1-{\mathsf{\gamma }}_{\mathrm{o}\mathrm{f}\mathrm{f}}\right|}}\right)$$

(17)

This equation allows us to evaluate the range of values of ${\gamma }_{off}$ for which the following condition holds:

$$∆{\mathrm{t}}_{\mathrm{c}\mathrm{r}\mathrm{o}\mathrm{s}\mathrm{s}}>\mathsf{\delta }$$

(18)

Such a condition supports the bounded stability of a given stick, for a given feedback delay $\delta$, because it implies that the blue part of a hyperbolic orbit is shorter than the red part, thus preventing the divergent evolution of the cycle in the phase plane. In particular, having set the delay at 230 ms, a simulation with the stick length of 100 cm is characterized by a range of values of ${\gamma }_{off}$, for which $∆t_{cross}>\delta$, is relatively large; in contrast, with a stick length of 32 cm, such a range is much smaller and thus a little amount of noise is sufficient to occasionally terminate the current on-phase at a state ${\gamma }_{off}$ for which the condition above is violated. If and when this condition occurs, the stick sway will quickly evolve to a fall.

However, the analysis illustrated by Equations 15-18 and Figure 17 suggests how to modify the activation conditions of the on-phase and off-phase of the intermittent controller to approach the performance of trained human subjects, e.g., stabilizing a 32 cm stick with a 230 ms feedback delay.

A proposed modification of the stabilization policy focuses on the explicit selection of the switching times: $t_{on}$ that terminates the hyperbolic orbit of the off-phase by turning on the feedback control action and $t_{off}$ that terminates the on-phase spiral orbit by switching off the control. In the basic intermittent control policy such critical switching times are determined by the simple test on the current but delayed measurement of the state vector (Equation (13)). In the extended formulation of the intermittent policy [30], the two switching times are evaluated by means of a simple prediction mechanism: $t_{on}$ for the next cycle is evaluated at the beginning of the off-phase, i.e., at $t_{off}$ of the previous cycle, exploiting Equation (17) in order to balance the hyperbolic orbit; $t_{off}$ for the next cycle is evaluated at the beginning of the on-phase, i.e., at $t_{on}$ of the previous cycle, performing a simulation of Equation (11) with the purpose of terminating as close as possible to the safe manifold because this promotes the slow evolution of the hyperbolic orbit. Figure 18 shows a simulation of the extended intermittent policy for a very short stick (32cm) and very long feedback delay (230ms). The figure illustrates that the hyperbolic orbits of the off-phase, from the green to the red manifolds, are approximately symmetrical, thus balancing the limit cycle; moreover, the terminations of the spiral orbits of the on-phase are very close to the green manifold, as required by the sought-after bounded stability.

4.3. Hybrid Stabilization for the COP Strategy: The DIP/VIP Model

The SIP (Single Inverted Pendulum) model has been challenged by several recent studies, which clearly show that movement around the hip joint is not negligible [31], thus suggesting the substitution of the SIP biomechanical model with a DIP (Double Inverted Pendulum) model, related to the ankle and hip joints (Figure 19).

Motion capture analysis has shown that the typical range of variation of the kinematics of the two joints during postural sway is both substantial and comparable:

• Ankle (angle ±0.2 deg; speed ±0.4 deg/s; acceleration ±15 deg/s²);
• Hip (angle ±0.3 deg; speed ±0.8 deg/s; acceleration ±50 deg/s²).

Moreover, the kinematic analysis of the sway patterns has revealed a strong correlation between the ankle and joint profiles [31,32]:

• Acceleration profiles exhibit a strong anti-phase correlation.
• Velocity profiles exhibit a minor anti-phase correlation.
• Rotational profiles exhibit a mild in-phase correlation.

Where is this pattern of coordination coming from? One explanation [31] is that such patterns are the direct result of an optimal bi-axial controller whose goal is to minimize the acceleration of the global CoM. An alternative explanation, summarized in the following [32], is a straightforward extension of the intermittent control policy, previously illustrated for the SIP model.

The proposed extension recognizes that the biomechanics of the plant to be controlled is bi-articular, thus substituting the SIP model with the DIP model. The extension of the control policy is hybrid, with two independent controllers, one for each joint:

• The ankle joint is stabilized with the same intermittent delayed feedback control expressed by Equation (10) for the SIP model, with the difference that the angle $q$, with the corresponding angular velocity $\dot{q}$, is not the angle $q_1$ but $q_{com}$, namely the angle of the Virtual Inverted Pendulum (VIP) that links the ankle to the CoM of the body (see Figure 19).
• For the hip joint, it is assumed that the control policy is simply a stiffness strategy that consists of the following equation ${\tau }_{hip}=K_{hip} q_{hip}$ where the coactivation of the hip muscles is modulated to allow the hip stiffness $K_{hip}$ to satisfy the following constraint:

$${\mathrm{K}}_{\mathrm{h}\mathrm{i}\mathrm{p}}>{\mathrm{m}}_{\mathrm{H}\mathrm{A}\mathrm{T}}\mathrm{ }\mathrm{g}\mathrm{ }{\mathrm{h}}_{\mathrm{H}\mathrm{A}\mathrm{T}}$$

(19)

The two controllers are independent, although with some interaction through the $q_{com}$term that indirectly links the two strategies. We may name such a hybrid stabilization strategy DIP/VIP model. Ultimately, the main support for this working hypothesis may come only from the simulated model’s consistency with the kinematic invariants of natural sway. However, it is also worth considering the plausibility of using the stiffness strategy for the hip joint, in contrast to the ankle joint, which is based on two points:

• The gravity destabilizing torque at the hip is much smaller than that at the ankle, so the required critical hip stiffness is smaller than the critical ankle stiffness.
• The hip muscles lack compliant tendons, such as the Achilles tendon, and are generally more powerful.

In the absence of direct measurements of the hip stiffness, such considerations are sufficient to support the hybrid DIP/VIP control model. However, the more substantial support comes from the model’s simulation results [32] (Figure 20).

The three panels of Figure 20 show, respectively, the evolution of the angular rotation (left panel), angular speed (central panel), and angular acceleration (right panel) of the two DoFs, plotted one against the other. The model’s simulation results accurately reproduce the ankle-hip joint coordination patterns observed in the experimental data, particularly the strong anti-phase correlation in the acceleration profiles. Moreover, the coordination patterns generated by the hybrid model simulations are very weakly dependent on the gain of the stiffness control, i.e., how much the hip stiffness exceeds the critical value: changing the gain from 1.2 to 20 has little effect on the results shown in Figure 20.

5. Feedback Control Paradigm for the CoM Stabilization Strategy

In the CoP stabilization strategy, which is usually considered the default paradigm, the CoM is the controlled variable, and the CoP is the control variable. In the less investigated paradigm, the CoM strategy, CoP, and CoM must exchange roles because environmental conditions strongly constrain the CoP’s range of motion. This happens, for example, in classical ballet (standing en pointe during an arabesque), in artistic gymnastics (beam exercises), and in circus balance skills (tightrope walking).

In particular, when balancing the standing body on a tightrope or a very narrow bar, the support base is very narrow in one direction while it is sizable in the other. Thus, a tentative tightrope walker, after aligning their sagittal plane with the tight rope, will naturally manage to constrain the sway of the body in the sagittal plane by adopting the CoP strategy, as when standing on earth, but will soon discover that this strategy is totally ineffective in the coronal plane. In such situations, what the brain can do for avoiding the fall is to invert the roles of the CoP and CoM information in the stabilization process: instead of modulating the CoP position, for constraining the sway of the CoM, the brain can modulate the CoM position in order to keep it as close as possible to the vertical line centered on the virtually unmovable CoP.

The CoM strategy may be implemented in a variety of ways by recruiting and moving body parts or artificial tools: in tight wire walking (Figure 21), the balancing trick is to use a balancing pole, which is shifted back and forth in the coronal plane to modulate the medio-lateral position of the CoM.

More specifically, a hybrid control paradigm has been developed for tightrope balancing [33] with two different and independent controllers for the two oscillatory planes, namely the sagittal plane and the coronal plane:

• Sagittal plane: CoP control strategy, related to the phase plane of the antero/posterior rotation of the body ($q_{ap}$ angle in Figure 21), with an alternation of on-phases and off-phases, in agreement with Equation (10). The control action, activated during the on-phases, is a torque applied to the ankle joints as a function of the delay of the joint rotation.
• Coronal plane: CoM control strategy, related to the phase plane of the medio/lateral rotation of the body ($q_{ml}$ angle in Figure 21). Also in this case, there is an alternation of on-phases and off-phases, with activation conditions similar to those employed in Equation (10):

$$\begin{gathered} \mathrm{O}\mathrm{n}-\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}:\mathrm{ }\mathrm{ }{\mathrm{q}}_{\mathrm{m}\mathrm{l}}\left(\mathrm{t}-\mathsf{\delta }\right)\mathrm{ }\mathrm{ }\left[{\dot{\mathrm{q}}}_{\mathrm{m}\mathrm{l}}\left(\mathrm{t}-\mathsf{\delta }\right)\mathrm{ }+\mathsf{\alpha }\hspace{0.17em}{\mathrm{q}}_{\mathrm{m}\mathrm{l}}\left(\mathrm{t}-\mathsf{\delta }\right)\right]\ge 0 \\ \mathrm{ }\mathrm{ }\mathrm{O}\mathrm{f}\mathrm{f}-\mathrm{p}\mathrm{h}\mathrm{a}\mathrm{s}\mathrm{e}:\mathrm{ }\mathrm{ }{\mathrm{q}}_{\mathrm{m}\mathrm{l}}\left(\mathrm{t}-\mathsf{\delta }\right)\mathrm{ }\mathrm{ }[{\dot{\mathrm{q}}}_{\mathrm{m}\mathrm{l}}\left(\mathrm{t}-\mathsf{\delta }\right)\mathrm{ }+\mathsf{\alpha }\hspace{0.17em}{\mathrm{q}}_{\mathrm{m}\mathrm{l}}\left(\mathrm{t}-\mathsf{\delta }\right)]<0\mathrm{ }\mathrm{ } \end{gathered}$$

(20)

During the on-phase, the control action, as shown in Figure 21, is a smooth shift of a hand-held balance pole in the opposite direction of the body’s medio-lateral imbalance angle $q_{ml}$ until the off-phase condition is achieved. The motion of the pole is modulated in agreement with a delayed feedback PD controller.

The rationale of using a long balance pole is that, with such a movable element, the moment of inertia around a forward rotation axis is almost doubled with respect to the natural inertia of the body, and this simplifies the stabilization task because the falling time constant of the body in the coronal plane is substantially increased. For example, the results of the simulation experiment illustrated in Figure 22 show that the parameters of the balance pole (length 12 m and weight 13 kg) are similar to those of famous professional tightrope walkers, such as Nick Wallenda or Philippe Petit. Moreover, by assuming standard anthropometric parameters of the human body (height 1.8 m, weight 78 kg), the time constant of the sideways fall in the coronal plane is increased from 347 ms to 542 ms if the funambulist holds the balancing pole. This effect is crucial for the successful stabilization in the coronal plane because the CoM strategy is intrinsically less reliable than the CoP strategy employed in the sagittal plane. In contrast, holding the balance pole aligned sideways has little effect on the sway in the sagittal plane.

The challenge of the hybrid control paradigm implies, in particular, the feasibility of exporting the state-space intermittent stabilization algorithm from the CoP Strategy to the CoM strategy, with the motivation of exploiting the “dynamic affordance” provided by the saddle-like instability of an inverted pendulum, and the working hypothesis that the two state-space, delayed feedback intermittent controllers may independently stabilize body sway in the two planes. The simulation in Figure 22 illustrates the performance of the hybrid control strategy, demonstrating that the two strategies can coexist successfully.

6. Feedforward Control Paradigm for Balancing While Acting

Human purposive actions are characterized by sequences of postures, linked by coordinated motions of the different parts of the body. Although most research on motor control has focused on the transient gestures between one posture and the next, the crucial role of stance is evident when considering challenging actions, such as those in martial arts or craftsmanship. In the former case, it is emphasized that defensive or offensive moves imply a strong and stable base: stability and mobility are the two complementary aspects of a good stance. The role of posture for craftmanship is linked to the discipline of ergonomics. One of the basic principles is to emphasize the importance of Neutral Postures, namely, body configurations in which the body is aligned and balanced, thus minimizing stress on muscles, tendons, nerves, and bones and allowing for maximum control and force production. In any case, the transition from one posture to another is known to include an essential component of feedforward control.

It was first observed by Belenkiy et al. [34] that during a movement such as raising the arm while standing, the first muscles to be activated are the leg muscles involved in postural control, shortly prior (some 50-100ms) to the activation of the prime mover. Such feedforward control mechanisms were named anticipatory postural adjustments (APAs) [35,36], interpreted as parallel feedforward commands aimed at minimizing disequilibrium induced by self-generated disturbances. APAs are usually combined with postural strategies initiated by sensory feedback signals signaling a perturbation, known as CPAs (Compensatory Postural Adjustments) [37,38,39]; the use of APAs considerably reduces the burden on CPAs and results in greater postural stability.

However, APAs (in combination with CPAs) describe only the hidden/unconscious component of feedforward postural stabilization strategies, in the sense that a comprehensive computational model requires to involve fundamental cognitive aspects of action representation, thus evolving the modeling framework from APAs to generalized whole-body postural-focal dynamics: this is the anticipatory synergy formation process that integrates the maintenance of upright stability with the focal, intentional coordination of the body for driving the designated end-effector to reach and interact with identified targets. Such an anticipatory synergy formation process is organized around the following computational building blocks:

• The degrees of freedom problem [40]
• Kinematic figural invariance [41]
• The equilibrium point hypothesis [42]
• Motor imagery [43]
• The body schema concept [44]
• Neural simulation theory [45]

The imbalance between the large number of DoFs of the human body and the much smaller number of task parameters that identify activities of daily life is usually called Motor Redundancy, emphasizing somehow negatively the complexity of the control problem, or Motor Abundance, [46], thus highlighting the flexibility and capacity to adapt a task to limitations due to unforeseen environmental conditions; moreover, motor abundance supports as well the ability to use different movements, produced by either the same or other parts of the body, to perform a task under various conditions, according to the principle of motor equivalence [47].

In any case, the synergy formation process is structured in such a way that the redundant/abundant DoFs are coordimated to induce the designated end-effector to exhibit robust kinematic-figural invariants: point to point reaching movements are approximately straight, independent of initial position, direction, and amplitude, with an invariant bell-shaped speed profile [48]; moreover 2D/3D gestures are characterized by an invariant anticorrelation of the curvature and speed profiles [49].

The feedforward nature of the synergy-formation process is also supported by the concept of motor imagery, namely the mental execution of an action without overt movement or peripheral (muscle) activation [43]. As a matter of fact, overt (real) and covert (mental) actions commonly alternate in life: the great majority of them is apparently silent but cognitively crucial in the sense that the primary purpose of motor imagery is prospection, i.e., the mental action of looking forward into the future for predicting the consequences of a course of action and/or replanning it. Moreover, research on motor imagery has demonstrated that not only overt & covert actions are topologically equivalent (i.e., the same brain areas are activated), but they are also temporally equivalent with respect to kinematic invariants [50,51].

At the same time the existence of motor imagery implies also the existence of a body schema [44], i.e., a mental representation of the geometry and the kinematics of the human body that may support simulations of whole-body gestures, in agreement with the neural simulation theory (NST) proposed by Marc Jeannerod [45]. This theory is based on the hypothesis that the motor system is part of a Simulation Network that is activated under a variety of conditions related to action, whether self-intended or observed in others. The mental activity induced by the simulation network generates a flow of neural patterns, called S-states, which characterize both overt and covert actions in the same manner. In other words, the activation of brain areas in the motor system during S-states puts Action Representation (the core of Motor Cognition) in a true motor format, so that it can be regarded by the brain as a real action, even in a covert/imagined context. NST is not directly involved in motor control, but it is the core of Motor Cognition and thus does not specifically produce muscle activation patterns. NST was not developed by the author to the level of a detailed computational formulation. However, the rationale for such a computational model is to perform the following functions: recruiting and coordinating the activation of the body’s redundant DoFs (synergy formation) and generating spatio-temporal kinematic patterns (a flow of S-states) consistent with the kinematic-figural invariants of biological motion.

A specific implementation of NST is PMP (the Passive Motion Paradigm) [52,53,54], which implements NST by Animating the Body Schema using a small number of Force Fields and Primitive Generators, typically applied to the end-effector and other parts of the body schema (Figure 23). The basic idea of the PMP model is that the coordination of the redundant DoFs of a kinematic chain can be obtained without using ill-posed versions of inverse kinematics but with a series of well-posed transformations: (1) a transformation (via the transpose Jacobian matrix of the kinematic body schema) that maps a virtual attractive force field applied to the end effector into the corresponding torque field applied to all the redundant DoFs of the kinematic chains; (2) a transformation that maps the torque field into a vector of joint rotation speeds (via a compliance matrix); and (3) a transformation (via the Jacobian matrix) that maps the joint rotation speeds into the corresponding velocity vector of the end effector. Such an attractive force field can be combined with other attractive or repulsive force fields, exploiting the additivity of a force-based representation of body dynamics, such as the specification of joint or balance limits.

For example, in the simulation illustrated by Figure 24 a force field (the “focal” field) is applied to the hand and attracts it to a designated target, in agreement with the ideomotor theory of action proposed by James [55]. The PMP applies the concept of “passive motion” by distributing the simulated external perturbation, due to the attractive force field, to the network representing the complete body schema: the passive motion consists of updating the control input of each element of the network so as to cancel the “stress” induced by the attractive field.

The simulation illustrated in the figure involves a standing body, and thus the attractive focal force field might destabilize the body schema by pulling the projection of the CoM outside the support base. To avoid this destabilizing effect, the PMP model for this task includes another force field, applied to the pelvis area of the body schema, with a repulsive force at the limits of the support base. The PMP exploits the additive property of the force fields and thus the animation exemplifies the emulation models of action [56,57], in agreement with the minimal intervention principle [58].

The simulation shown in the figure shows that the end-effector reaches the sequence of target points specified by the task and, at the same time, maintains the CoM within the support base. The animation produced by the PMP model, as a computational implementation of the NST, may be viewed as a generalization of the APA concept, integrating the intentionality of goal-driven synergy formation with the feedforward compensation for the destabilizing effects of whole-body actions. Such a feedforward balancing paradigm is not an alternative to the other paradigms considered in the previous sections, namely the stiffness and intermittent feedback paradigms. Still, they may well coexist with variable weights, adapted to the specific task constraints and environmental conditions: the feedforward paradigm represents the highest cognitive level of the cybernetics of balance control.

7. Conclusions and Outlook

There is no doubt that one of the essential foundation concepts at the origin of Cybernetics was a strong transdisciplinary in the study of biological organisms and artificial systems in terms of a general “control and communication paradigm”, capable of processing information to self-regulate, adapt, and achieve goals autonomously. The extended development of this concept went both ways: from biology to technology and vice versa. In particular, a working hypothesis supported by some “enthusiastic” cyberneticians was that the best way to understand and model biological organisms is to apply information and communication theories and technologies and, at he same time, the best way to conceive and design zoomorphic or anthropomorphic artificial systems is to adhere to a bio-inspired roadmap.

After one or two decades, however, cybernetics’ transdisciplinarity fragmented, giving rise to independent disciplines focused on separate fields. In particular, two related research areas that Norbert Wiener would have considered typically cybernetic in the original sense, i.e., artificial intelligence and humanoid robotics, were developed in the following decades without any significant bio-inspired contributions. On the other hand, research in cognitive neuroscience, starting in the 1990s, re-discovered the transdisciplinary cybernetic emphasis through enactivism and embodied cognition [1,2], disseminating such concepts to a new wave of bio-inspired intelligent humanoid robots, focusing on cognitive robotic architectures and develpmental robotics [59,60], in alternative to the disembodied nature of AI as well as the marginally embodied substance of the so-called embodied AI [61,62].

This conceptual background highlights the proposed review, along with the transdisciplinary double roadmap focused on a specific human activity: the representation and modeling of unstable tasks. It has been shown that in a variety of situations, human subjects build a hybrid combination of strategies, adapted and modified during use, without focusing on a detailed and universal optimization procedure, but combining the different stabilization paradigms while balancing the pros and cons of each strategy in the context of the specific tasks, according to a minimum intervention principle.

For example, opportunistic control paradigms can be exploited as a dynamic ecological affordances, thereby leveraging the intrinsic computation performed by the human body in its interaction with the physical environment. On the other hand, entrainment, resonance, or synchronization cannot occur if the dynamics is too fast.

The pros of the stiffness paradigm, whether implemented by coactivation of antagonistic muscle groups (in the biological case) or by artificial variable-stiffness actuators or high-gain PD controllers, are a simple noise-rejection mechanism that improves the precision of the planned gesture. However, there is a possible double drawback if overrelying on stiffness: a high energetic cost and/or a cognitive drawback because the high rigidity implied by a high stiffness organization of complex tasks is known to be detrimental from the point of view of the skill learning process, which is supposed to evolve from an initial high-stiffness phase to a more relaxed condition.

The feedback control paradigm is limited by the delay in acquiring feedback sensory signals relative to the intrinsic dynamics. This is the case for human subjects in upright balancing and stick balancing, leading to the substitution of continuous PD feedback control with intermittent PD feedback control. In practice, this is equivalent to configuring a hybrid control paradigm that integrates opportunistic control, i.e., exploiting the saddle-point metastability of the SIP/DIP/CIP models, with long-delay feedback control. Such a hybrid configuration would not be necessary in state-of-the-art humanoid robots, as the available sensing/microelectronic technologies provide sensory feedback at least 1,000 times faster than biological control mechanisms. However, hybrid control paradigms, such as the one described for biological balance control, might be required in medical or space applications of robotics or teleoperation that involve large sensory delays, bridging the gap between the two sides of cybernetics.