Remote Control for Robot Walking via Gesture Detected by Inertial Sensor

: Wearable technology has been proposed as a potential tool to change the way of human life, such as the smart bracelet and the Google Glass. In the wearable technology, the inertial sensor has great signiﬁcance in tracking the object movements. The paper focused on detecting the movements of user’s ﬁnger based on the inertial sensor to give the control signals. Firstly, the attitude matrix, which represented the transformation relation of carrier coordinate system and the navigation coordinate system, was obtained. Secondly, the attitude matrix was expressed based on the quaternions. Thirdly, the ﬁnger gesture was processed by the attitude matrix to get the attitude angle. Finally, the robot was controlled by attitude angle to make the moving action. The experimental results showed the detection of the ﬁnger movement is effective.


Introduction
Nowadays, the wearable technology has drawn the attention of researchers and engineers to improve the HMI (Human Machine Interface), a useful product to provide convenience.Meanwhile, the inertial sensor with powerful data processing ability and a wearable advantage has been applied into lots of domains.In medical field, the researchers developed the automatic computer-aided balance assessment system.This method employed ambient assisted living architecture based on inertial sensors to examine the balance ability of the patients [1].Similarly, the inertial sensor is also used in a wearable body fall monitoring device [2].When the user's body posture changes greatly, the inertial sensor will automatically detect the change of angle to judge whether the user falls.Furthermore, the device will alarm by wireless communication device.In addition, a variability analysis of upper limb therapeutic movements using wearable inertial sensors is presented [3].The objective quantification of human movements contributes to better understanding the patient condition, and might serve as a measure of efficacy of the rehabilitation treatment [4].In sports, the estimation process of limb lengths is automated with the algorithm calculating curvature using the measurements from inertial sensors [5].Furthermore, the inertial sensor can be also used to detect the human gait.It measured the acceleration information of human walking, calculating the pace through the software algorithm precisely [6].In science and technology, gesture recognition is often explored for applications such as hand written character recognition with a handheld pen [7], or a handheld controller to operate a television [8].The researchers examined gesture recognition algorithms that use an inertial sensor worn on the forearm to get the gestures.A set of six gestures were selected fit within the context of an active video game instead of using the mouse [9].
Compared with their devices, we designed a device which uses the inertial sensor to detect the change of the finger's gesture and send commands to the robot through wireless communication.
Because the angle of finger is much smaller, we have to design the algorithm which had a higher precision and faster processing speed to realize the function of detecting the change of finger gesture.
One method describes the relative position between the navigation coordinate system and the carrier coordinate system is based on Euler angles [10][11][12], in which the attitude angle is calculated by solving the differential equations.However, the computer numerical integration will carry out a large number of trigonometric functions.Furthermore, it results in a 'singularity' problem and the inaccuracy of robot control.In this paper, we use quaternions to calculate the attitude matrix to get the attitude angle.The method just needs four differential equations to solve with less calculation.Moreover, it can avoid into the 'singularity' problem and improve the efficiency of the calculation.For the applications, the gesture of finger is captured to control the movement of NAO robot.
The content of this paper is divided into the following parts: Section 2 introduces how to detect finger gesture and process the data of finger gesture.Section 3 discusses the threshold selection and arranges experiments to demonstrate the performance of developed system.Section 4 gives the conclusion and discusses further research.

Inertial Navigation Coordinate System and Its Transformation
Carrier coordinate system is called b system (O x b y b z b ), the origin is in the carrier's center of gravity, and is fixedly connected with the carrier.The axis x b point to the front along the longitudinal axis, in addition, the axis y b point to the right along the traverse axis.Furthermore, the axis z b is perpendicular to the horizontal plane, which is composed of the axis x b and y b axis.Correspondingly, navigation coordinate system is called n system (O x n y n z n ).The inertia components of the strap-down inertial navigation system (SINS) are fixed on the carrier, so the output value of the sensor is the output value of the carrier coordinate system.Therefore, we must transform these measurements to another coordinate system for calculate the navigation parameters, that is, the navigation coordinate system.
The data from the carrier coordinate system to the navigation coordinate system must be transformed before the strap-down algorithm.As is shown in Figure 1, we can use three times of Euler rotation to transform the carrier coordinate system to the navigation coordinate system.It means that the navigation coordinate system needs to rotate ψ degree counterclockwise around the z n axis.Secondly, it needs to rotate θ degree counterclockwise around the x n axis.make the navigation coordinate system rotate around y n axis for about γ degree.The transition from navigation coordinate system to the carrier coordinate system is made in the following formula (1).
where ψ , θ ,γ represent yaw, pitch and roll, respectively.The transformation relation between the b system and n system can be expressed by the following.
Three times of Euler rotation can create three cosine matrices, the direction cosine matrix C b n can be obtained by multiplying the three matrices with the formula(3) [13].
, C n b can be obtained in the following.

Attitude Matrix Algorithm Based on Quaternions
In order to solve the real-time attitude angle, the current attitude matrix C n b must be determined.In addition, the quaternions can solve the problem of different spatial coordinate transformation.The method needs to solve the quaternion differential equation when calculating the attitude matrix which is described by the quaternion [14].The following formula ( 5 quaternion differential equation, where q(t) represents quaternion, the symbol • indicates quaternion multiplication.
The formulas (6) (7) are expressed as matrix form, that is, where ω b nb is a matrix expression of three-axis angular velocity [15].
where T 11 , T 22 , T 33 is an element in the direction cosine matrix in formula(4), and q 0 , q 1 , q 2 , q 3 are the elements in the quaternion attitude matrix.The elements of the cosine matrix can be calculated directly from the elements of the quaternion.Therefore, the attitude angle is solved by the method of inverse trigonometric function as shown in the formula (11).

Process from Data of Finger Gesture to Movement of the robot
In this remote system, four control instructions correspond to the four types of finger gestures.The finger gestures include the finger lifts up, points to the front and parallels to the ground, turns right and turns left.The finger movements are respectively defined as Finger_U p ,Finger_Horizontal , Finger_Right and Finger_Le f t .Because these finger movements mainly change the values of Pitch and Yaw, only these two values are considered.When an operator makes the finger movements mentioned above, the gesture data are recorded by inertial sensor, as shown in Figure 2. The blue line and red line represent Pitch and Yaw.In order to accurately detect the finger gestures, we select the finger points to the front and parallels to the ground as the reference position.In addition, during the initialization, the initial angles of Pitch and Yaw are set as 0 degree.The finger movements can lead to the change of angle, and then the robot can be controlled.In order to detect the change of the angle accurately, the thresholds T 1 and T 2 are selected.The ideal discrimination rules are shown in the following formula (12).
where Finger_U p describes the finger lifts up, Finger_Horizontal describes the finger points to the front and parallels to the ground, Finger_Right describes the finger turns right, Finger_Le f t describes the finger turns left.
The relation of the gesture and the control of robot walking is as the following: When the user lifts his/her finger, we will find that the blue line goes up sharply and appears a peak, and then the robot is controlled to go forward.When the finger comes back to the reference position, the blue line will come back to the 0 degree and the robot will be controlled to stop directly.In addition, when the user's finger turns right, the red line will go up, and then the robot will be controlled to go right.Furthermore, when the user's finger turns left, the red line will decline quickly and emerge a valley.The robot will be controlled to turn left.The changes of the angle and the movements of the robot are shown in Figure 3.

The Threshold Analysis
Due to the inertial sensor is set on the finger, the measurements will contain the disturbance and noise.Thus we should set the minimum value of angle to avoid the influence of noise.On the other hand, when the operator lifts his or her finger but without the attention to control the robot movement, the data will be measured by the inertial sensor, too.To avoid this situation, we set a maximum value to prevent the wrong movements.If the angle is large enough beyond the maximum value, the robot will also remain stationary.
In order to find out the reliable thresholds, we use the off-line experiments to analyze the measurements.During the experiment, the operator makes the corresponding finger gestures, as shown in Figure 4.These gestures will be transformed to corresponding instructions including Finger_U p, Finger_Horizontal, Finger_Right and Finger_Le f t.The recorded data of angle changes are shown in Figure 5 and Figure 6.
Figure 5 shows that the range of Pitch between 0 degree and 30 degrees may be caused by the slight jitter of the finger, which will give a wrong command.So we set the minimum angle T 1 of Pitch as 30 degrees, which indicated by the green line.And we set the maximum angle T 2 as 60 degrees, which indicated by the yellow line in the Figure 5. Therefore, the thresholds of 30 degrees to 60 degrees can be selected which can reflect the change of Pitch angle.When the finger turns right, the angle of Yaw will be a positive degree.On the contrary, when the finger turns left, the angle will be a negative degree.Similar to the Pitch, we select the threshold T 3 as ±20 degrees and T 4 as ±40 degrees to accurately reflect the change of Yaw.In the Figure 6 the green lines indicate T 3 and the yellow lines indicate T 4 .The rules to detect the movement of finger are as shown in the formulas (13) (14).We note that Pitch belongs to [T 1 ,T 2 ], it means that the finger is lifted up.Pitch is equal to 0 degree, it means that the finger is pointed to the front.Yaw belongs to [T 3 , T 4 ], it means that the finger is turned right.Yaw belongs to [−T 4 , −T 3 ], it means that the finger is turned left.

Controller for NAO robot
A controller which runs on the robot is developed using the python language under the windows environment.It maintains two functions running on different threads: instruction receiving and behavior performing.Here, the robot acts as the server port, while the inertial sensor is the client port.The server receives the instruction from the client with the form of input stream.At the same time, the controller gives four types of character instructions: "G", "S", "L" and "R" whose definitions are listed in Table 1.During the entire control process, the robot needs to receive the command from the bound port continuously, so that it is able in time to obtain the instruction from the client.

Command Definition
G controlling the robot to go forward S controlling the robot to stop L controlling the robot to turn left R controlling the robot to turn right

Robot Movement
In order to verify the accuracy of the design of the system, we use the finger position of the navigation robot from the initial position to reach the destination.First of all, we establish the relationship between the inertial sensor and the robot.Then, the robot will be controlled to go forward, stop, turn right and turn left by the inertial sensor which is fixed on the operator's finger.The angle of finger measured by the inertial sensor as shown in Figure 7 and the experimental scene of the robot is shown in Figure 8.

Conclusion
The integration between wearable technology and robot control based on the inertial sensor is considered in this paper.A method was proposed for remote control of robot via gesture detection based on inertial sensor.The system uses the inertial sensor to detect the moving direction of the finger and sends corresponding commands to the robot through wireless communications.In order to improve the accuracy, quaternion is used in our method.Experiment verified the effectiveness and reliability of the proposed method.There are some significant advantages of this method as follows: 1) The system based on inertial sensor is more portable due to the wearable feature, and the control command switching is more natural and smoother.
2) This finger control system can use the finger gestures to control the robot.
3) It shows a better precision compared with other approaches and makes the operation more convenient.

Figure 2 .Figure 3 .
Figure 2. Measured data of all kinds of finger movements

Figure 4 .
Figure 4.The finger fixed on the inertial sensor.

Figure 5 .
Figure 5.The range of Pitch angle.

Figure 6 .
Figure 6.The range of Yaw angle.

PreprintsFigure 7 .
Figure 7.The angle of finger measured by the inertial sensor.

Table 1 .
Definition of commands.