A Chip-Level Testing Platform of Unmanned Vehicle Autopilot Systems with FPGA-Based Hardware-in-the-Loop Simulation

1. Introduction

As unmanned vehicles such as quadcopters [1,2], fixed-wing [3], unmanned ground vehicles [4], and unmanned surface ships [5] are widely used in various fields, the electromechanical equipment of these unmanned vehicles is becoming increasingly complex, and the tasks they undertake are becoming more and more diverse. However, intensive flight missions may induce fault that can lead to conflicts, collisions, and crashes. Therefore, for the expected growth of unmanned vehicle equipment, people are paying even greater attention to the reliability of autopilot system. As a life-critical system for unmanned vehicles, the reliability evaluation [6] of the autopilot system is a significant research content for airworthiness testing.

In recent years, the hot research content such as sensor fault diagnosis(FD) [7,8], the fault tolerant control(FTC) [9,10] and prognostics and health management(PHM) [11,12] technology in the field of unmanned vehicles shows that the unmanned vehicle autopilot system is suffering from various types of faults, which revealing that the unmanned vehicle autopilot system still has considerable vulnerability. In order to efficiently verify the reliability of the autopilot system, a testing platform that can accomplish the fault simulation of the autopilot system has a very important role for the above research content. At present, the reliability assessment of most unmanned vehicle autopilot systems relies on the real-aircraft flight tests that inject a variety of faults during flight. Although the real-aircraft experimental method can genuinely reflect the actual situation, faults can only occur under specific conditions, and fatal faults will cause severe damage to unmanned vehicles. In addition, it is difficult to ensure the consistency of the faults under real-aircraft testing (to ensure that the same faults have the same fault action time and the same fault impact degree in multiple tests).

Hardware-in-the-loop(HIL) simulation [13], as a very popular simulation method in recent years, can make up for the problems of randomness, consistency and repeatability that are difficult to be solved by real machine testing methods. The publicly released UAV simulation platforms can be categorized into two types: turntable platform and generic platform. As a typical representative of the rotary HIL simulation platform, the asymptotic tracking controller test platform proposed by Zhao [14] et al. has outstanding advantages when attitude control and external factor interference are considered, but it is difficult to inject rich sensor fault types due to the use of actual unmanned vehicles. Generic platforms proposed by large tech companies or research teams, such as XTDrone [15], Gazebo [16], Jmavsim [17], Airsim [18], and FilghtGoggles [19] are more concerned with the validation of control law algorithms and intelligent algorithms, and very few are able to simulate various types of faults and complete the reliability assessment of unmanned vehicle autopilot systems. A few other researchers, Gong [20] et al. proposed a semi-physical simulation platform for UAV autopilot system to generate various fault data, but the platform pay more attention to the characteristics of the fault data realizations without considering the impact of the fault behavior on the whole system in depth. Khan [21] et al. proposed a fault injection framework to simulate UAV flight and evaluated the reliability of the flight autopilot system undergo multiple GPS faults but did not consider other sensors in the hardware system. In addition, the above HIL platforms, besides failing to meet the strong demand for fault generation, also generally suffer from difficulties in adapting to different unmanned vehicles, compatibility problems between the software or hardware platforms of the autopilot system, and difficulties in meeting the low real-time performance brought about by the bandwidth of data transmission.

Field Programmable Gate Array(FPGA) [22] are a special type of digital circuitry that is unique in that it can be programmed at the hardware level to perform specific logic functions. Compared to Application Specific Integrated Circuit(ASIC) in sensor chips, FPGA offer significant flexibility, parallel processing, and operating frequency advantages that make them a powerful tool in the field of chip-level design and verification [23]. Thus, in this paper we simulate in FPGA the chip-level behavior of components of various sensor products of an autopilot system, including sensor outputs, and these simulated signals are subsequently used in flight control algorithms to ensure stability and correct response of the vehicle. Further, various flight conditions and sensor failure scenarios can be easily simulated with FPGA, which are very helpful in verifying the performance and robustness of sensor chips and autopilot systems under extreme conditions.

In this article, we propose a chip-level testing platform for reliability evaluation of unmanned vehicle autopilot systems, focusing on solving the generalized adaptability of the simulation platform as well as generating highly reliable sensor faults. The main research content and corresponding contributions are as follows:

(1)

This paper proposes a multi-sensor digital modeling method for adapting to various UAV autopilot software platforms, which is divided into two parts: measurement model and communication modeling, and is verified on the FPGA platform. The simulation sensor model outputs the same digital signal as the real sensor product, realizing the interaction of sensing data between the simulation model and the autonomous driving hardware platform.

(2)

This article proposes a coarse-grained failure model of inertial measurement unit(IMU), barometer, magnetic compass sensor and global position system(GPS) receiver under the action of multiple common failure factors for the necessary sensor chips of the autonomous driving hardware platform. Used to provide rich fault data for UAV health management research work.

(3)

This article proposes a HIL platform for reliability verification of various autopilot systems, adapting to a variety of popular autonomous driving software platforms and hardware platforms as well as various types of UAV systems, including fixed-wing, multi-purpose Rotors, unmanned ships and unmanned vehicle systems. The platform has many excellent features such as high fidelity, ultra-high real-time performance, and black box testing.

2. Proposed HIL System Structure

In the study [24], we introduced a standardized framework for UAV autopilot systems based on FPGA HIL simulation. Building upon this foundation, we propose a testing platform that dedicated to fault generation for autopilot sensor system. As shown in Figure 1, the platform is divided into four parts: Real-time HIL Platform(RHILT), Unmanned Vehicle Autopilot System(UVAS), Host Computer(HC) and Radio Control(RC).

2.1. RHILT

It is primarily divided into two parts: system-on-chip(SoC) and FPGA. The SoC executes a mathematically complex unmanned vehicle model and employs the RT-Linux system to ensure synchronization of the model’s clock with the real-world clock. In contrast, the FPGA operates a high-frequency update communication adapter model to ensure the real-time nature of sensor communication. See Section 3 for the details of the unmanned system model involved in this module. As well, the sensor model is discussed in detail in Section 4.

Moreover, the PS sends flight simulation data packets at a higher frequency, including byte streams of the airframe type, location, attitude, etc. Concurrently, the PS possesses a fault injection interface that supports online modification of model parameters for the purpose of injecting faults.

2.2. UVAS

An UVAS consists of a control hardware system and a control software system, and the most popular open source platforms for the consumer market are listed in [25]. In general, the hardware system is mainly used to provide the computational environment (embedded processor) and sense motion information (sensors), while the software system is mainly used to solve the control laws (Extended Kalman Filter algorithm) to achieve autonomous navigation control. For safety reasons, the software system will also include algorithms for fault-tolerant control, fault diagnosis and health management.

In order to ensure that the autopilot system can work properly on the proposed testbed, it is necessary to block (or remove) the on-board sensors in the hardware system and reconnect the autopilot system to the pins of the sensor chip simulated by the fpga system of the real-time simulation computer in order to receive the digital communication signals from the sensors. The above solution does not require any modification to the software system, which greatly protects the controller software algorithm from being leaked, and is of practical significance for black-box testing of different autopilot products from different UAV companies.

2.3. HC

The mainframe is tasked with running the application. The following three applications are included.

(1)

The vision software uses the latest unreal engine 5 to render flight simulation animations, and the simulation model of the UAV frame and terrain environment (including obstacles) can simulate the objective world with a high degree of realism. It receives flight simulation data in real time to determine the UAV attitude and position and other dynamic performance characteristics. At the same time, the fault injection effect can be qualitatively assessed by observing the UAV flight process.

(2)

The fault injection tool is used to trigger fault generation online. In this test method, engineers add fault test cases by setting fault modes, fault durations and fault intensities to simulate various fault cases that may occur in the sensor system.

(3)

On the one hand, the ground station formulates automatic test tasks (such as cruising, level flight, or hovering), and on the other hand, it monitors sensor data in real time to quantitatively evaluate the fault injection effect.

2.4. RC

As an optional tool, the RC has the same mission planning functions as the ground station. Engineers can manually control the movement of the unmanned vehicle to the desired state and complete some difficult flight actions (such as yaw, roll extreme sports, etc.).

3. Unmanned Vehicle Model

The unmanned vehicle model mainly describes the dynamic and kinematic motion process of the unmanned vehicle in the working environment. A complex unmanned vehicle model can be regarded as composed of several small subsystems, as shown in Figure 2, including actuator subsystem, environment subsystem, force and moment subsystem, vehicle body subsystem and Sensor Hub.

The actuator subsystem is the primary power source of control unmanned vehicles, but the accurate power model is extremely complex. From the control efficiency and the general characteristics of the motor system, the transfer function of the simplified model of the power unit is given as follows

$$\begin{array}{c}\hfill {\varpi }_i=\frac{1}{T_{\mathrm{m}}s+1}\left(C_{\mathrm{R}}{\sigma }_i+{\varpi }_{\mathrm{b}}\right)\end{array}$$

(1)

where i is the sequence number of the actuator, ${\varpi }_i$ is the motor speed, ${\sigma }_i$ is the throttle command that is equal to the PWM value, $T_{\mathrm{m}}$ is the motor dynamic response time constant,$C_{\mathrm{R}}$ and ${\varpi }_{\mathrm{b}}$ are constant parameters that can be obtained from the website1. Then, the driving force ${}^b\mathit{F}_A$ and driving torque ${}^{\mathrm{b}}\mathit{M}_A$ generated by the actuator are

$$\begin{array}{c}\hfill \{{}^b\mathit{F}_A,{}^{\mathrm{b}}\mathit{M}_A\}=f(\mathit{E},{\displaystyle \sum _i}{\mathit{\varpi }}_i^2)\end{array}$$

(2)

where $\mathit{E}$ is the control efficiency matrix taking into account the different rack layouts.

The support forces ${}^{\mathrm{b}}\mathit{M}_G$ and support moments ${}^b\mathit{F}_G$ from the ground during take-off and landing, as well as the aerodynamic resistance ${}^b\mathit{F}_D$ and damping moments ${}^{\mathrm{b}}\mathit{M}_D$ during flight, are also considered in depth. The total force ${}^{\mathrm{b}}\mathit{F}$ and moment ${}^{\mathrm{b}}\mathit{M}$ of the unmanned carrier is expressed as

$$\begin{array}{c}\hfill \begin{array}{c}{}^{\mathrm{b}}\mathit{F}={}^b\mathit{F}_A+{}^b\mathit{F}_G+{}^b\mathit{F}_D\hfill \\ {}^{\mathrm{b}}\mathit{M}={}^{\mathrm{b}}\mathit{M}_A+{}^{\mathrm{b}}\mathit{M}_G+{}^{\mathrm{b}}\mathit{M}_D\hfill \end{array}\end{array}$$

(3)

The unmanned vehicle provides the necessary force and moment to control the position and attitude of the system through its actuators. A six-degree-of-freedom(6DOF) model can describe the motion of any unmanned vehicle. Define unmanned vehicle flat Earth position as ${}^{\mathrm{e}}\mathit{p}\stackrel{\Delta }{=}{\left[xyz\right]}^T$, flight speed vectors in body-fixed axes concerning body frame as ${}^{\mathrm{b}}\mathit{v}$, likewise in the flat Earth reference frame as ${}^{\mathrm{e}}\mathit{v}$, angular rates vectors in body-fixed axes as ${}^{\mathrm{b}}\mathit{\omega }\stackrel{\Delta }{=}{\left[{\omega }_{x_b}{\omega }_{y_b}{\omega }_{z_b}\right]}^T$, body euler angle as $\Theta \stackrel{\Delta }{=}{\left[\varphi \theta \psi \right]}^T$. The dynamic and kinematic characteristics of the unmanned vehicle can be described as follows

$$\begin{array}{c}\hfill \dot{\mathit{X}}=F(t,\mathit{X},\mathit{\sigma })\end{array}$$

(4)

where t is real time factor, $\mathit{X}\stackrel{\Delta }{=}{{[}^{\mathrm{e}}\mathit{p}{ }^{\mathrm{e}}\mathit{v}{ }^{\mathrm{b}}\mathit{\omega }\Theta ]}^T$, $\dot{\mathit{X}}$ is the next state estimate, $\mathit{\sigma }$ is the PWMs vector from the output of the autopilot system.

Inspired by the works presented in [26], the rigid motion law of the unmanned vehicle can be expressed by differential equations, including force and moment equations, navigation equations, and motion equations.

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{}^{\mathrm{e}}\dot{\mathit{p}}={}^{\mathrm{e}}\mathit{v}={\mathit{R}}_{\mathrm{b}}^{\mathrm{e}}·{}^{\mathrm{b}}\mathit{v}\hfill \\ {}^{\mathrm{b}}\dot{\mathit{v}}=-{\left[{}^{\mathrm{b}}\mathit{\omega }\right]}_{\times }·{}^{\mathrm{b}}\mathit{v}+{}^{\mathrm{b}}\mathit{F}/m\hfill \\ \dot{\Theta }=\mathit{W}·{}^{\mathrm{b}}\mathit{\omega }\hfill \\ \mathit{J}·{}^{\mathrm{b}}\dot{\mathit{\omega }}={-}^{\mathrm{b}}\mathit{\omega }\times \left(\mathit{J}·{}^{\mathrm{b}}\mathit{\omega }\right)+{}^{\mathrm{b}}\mathit{M}\hfill \end{array}\right.\end{array}$$

(5)

where ${\mathit{R}}_{\mathrm{b}}^{\mathrm{e}}$ is the rotation matrix from the body-fixed frame to the earth inertia frame, ${\left[{}^{\mathrm{b}}\mathit{\omega }\right]}_{\times }$ is ${}^{\mathrm{b}}\mathit{\omega }$ skew symmetric form, $\mathit{W}$ is a matrix shorthand, $\mathit{J}$ is the moment of inertia, ${}^{\mathrm{b}}\mathit{F}$ and ${}^{\mathrm{b}}\mathit{M}$ are the vectors of external forces and moment, respectively, satisfying the following:

$$\begin{array}{c}\hfill {\mathit{R}}_{\mathrm{b}}^{\mathrm{e}}=\left[\begin{array}{ccc}{C}_{\theta }{C}_{\psi }& C_{\psi }{S}_{\theta }{S}_{\varphi }-S_{\psi }{C}_{\varphi }& C_{\psi }{S}_{\theta }{C}_{\varphi }+S_{\psi }{S}_{\varphi }\\ C_{\theta }{S}_{\psi }& S_{\psi }{S}_{\theta }{sin}_{\varphi }+C_{\psi }{C}_{\varphi }& S_{\psi }{S}_{\theta }{C}_{\varphi }-C_{\psi }{S}_{\varphi }\\ -S_{\theta }& S_{\varphi }{C}_{\theta }& C_{\varphi }{C}_{\theta }\end{array}\right],\end{array}$$

$$\begin{array}{c}\hfill \mathit{W}\stackrel{\Delta }{=}\left[\begin{array}{ccc}1& T_{\theta }{S}_{\varphi }& T_{\theta }{C}_{\varphi }\\ 0& C_{\varphi }& -S_{\varphi }\\ 0& S_{\varphi }/S_{\theta }& C_{\varphi }/C_{\theta }\end{array}\right],{\left[{}^{\mathrm{b}}\mathit{\omega }\right]}_{\times }\stackrel{\Delta }{=}\left[\begin{array}{ccc}0& -{\omega }_{z_b}& {\omega }_{y_b}\\ {\omega }_{z_b}& 0& -{\omega }_{x_b}\\ -{\omega }_{y_b}& {\omega }_{x_b}& 0\end{array}\right]\end{array}$$

where $S_{(·)}$, $C_{(·)}$ and $T_{(·)}$ denote $sin(·)$, $cos(·)$ and $tan(·)$, respectively.

4. Sensor Digital model

In consumer electronics, manufacturers favor high-performance MEMS sensors. For example, IMU sensors use the piezoelectric or Coriolis effect to detect changes in motion, and the workflow as shown in Figure 3a includes signal amplification, digitization, and data communication control logic. In order to simulate actual flight testing of an autopilot system on the HIL platform, a sensor simulation workflow as shown in Figure 3b is recommended. This simulation generates raw sensor data, adjusts the data to meet the requirements of the autopilot software through scaling and conversion, and simulates the data transfer using appropriate communication protocols.

4.1. Sensor Data Source

The types of sensor data are diverse. The first type of sensor data is to perceive its own attitude, such as acceleration and angular velocity, which can be calculated by the 6DOF model; The second type of sensor data is related to working environment factors, such as temperature, air pressure and magnetic field strength, these require the use of standard modules with high reliability; The third type of sensor data is observation, such as the longitude, latitude and altitude data required for GPS positioning, which need to be converted between different reference systems. The program of calculating the sensor raw data is as shown in Algorithm 1. It is worth mentioning that the Flat Earth to LLA Model, WGS84 Model, ISA Atmosphere Model and World Magnetic Model from lines 5 to 8 in Algorithm 1 are standard library models [27].

Algorithm 1: Calculate sensor data

4.2. Measurement Model

4.2.1. IMU Model

The output of the MEMS IMU are affected by many error sources, such as scale factor errors, biases, and random drift. In the field of inertial navigation, the acceleration and angular velocity measurements of the IMU [28] can be expressed as:

$$\begin{array}{c}\hfill {\mathit{u}}_a={\mathit{C}}_a{·}^b\mathit{a}+{\mathit{b}}_a+{\mathit{\epsilon }}_a\end{array}$$

(6)

$$\begin{array}{c}\hfill {\mathit{u}}_g={\mathit{C}}_g{·}^b\mathit{\omega }+{\mathit{b}}_g+{\mathit{\epsilon }}_g\end{array}$$

(7)

where ${·}^b\mathit{a}$ and ${·}^b\mathit{\omega }$ are the real values of the acceleration and angular velocity in the body frame, respectively. Furthermore, ${\mathit{C}}_a$ and ${\mathit{C}}_g$ are defined as follows:

$$\begin{array}{c}\hfill {\mathit{C}}_a=\left[\begin{array}{ccc}{s}_x^a& {\mu }_{xz}^a& -{\mu }_{xy}^a\\ -{\mu }_{yz}^a& s_y^a& {\mu }_{yx}^a\\ {\mu }_{zy}^a& -{\mu }_{zx}^a& s_z^a\end{array}\right],{\mathit{C}}_g=\left[\begin{array}{ccc}{s}_x^g& {\mu }_{xz}^g& -{\mu }_{xy}^g\\ -{\mu }_{yz}^g& s_y^g& {\mu }_{yx}^g\\ {\mu }_{zy}^g& -{\mu }_{zx}^g& s_z^g\end{array}\right]\end{array}$$

where the diagonal elements $S_a\stackrel{\Delta }{=}\left[\begin{array}{ccc}{s}_x^a& s_y^a& s_z^a\end{array}\right]$ and $S_g\stackrel{\Delta }{=}\left[\begin{array}{ccc}{s}_x^g& s_y^g& s_z^g\end{array}\right]$ of the $C_a$ and $C_g$ matrix are the acceleration and angular velocity scale factor errors respectively. ${\mu }_{xz}^a,{\mu }_{xy}^a,{\mu }_{yz}^a,{\mu }_{zy}^a,{\mu }_{zx}^a$ are the six installation error angles between the accelerometer frame a and the body frame b;${\mu }_{xz}^g,{\mu }_{xy}^g,{\mu }_{yz}^g,{\mu }_{zy}^g,{\mu }_{zx}^g$ are the six installation error angles between the gyro frame g and the body frame b; ${\mathit{b}}_a$ and ${\mathit{b}}_g$ denotes the acceleration zero bias and angular velocity zero bias respectively; ${\mathit{\epsilon }}_a$ and ${\mathit{\epsilon }}_g$ denotes the acceleration rate noise and angular velocity rate noise respectively, assumed to be Gaussian white noise [29] with characteristics

$$\begin{array}{c}\hfill \begin{array}{c}E\left[{\mathit{\epsilon }}_a\right]={\mathit{0}}_{3\times 1}\hfill \\ E\left[{\mathit{\epsilon }}_a^2\right]={\mathit{\sigma }}_{a,noise}^2·f_s·{\mathit{I}}_{3\times 3}\hfill \end{array}\end{array}$$

(8)

where ${\sigma }_{a,noise}$ is the white noise variance coefficient, $f_s$ is the sampling rate. The zero bias ${\mathit{b}}_a$ and ${\mathit{b}}_g$ will change with time and can be established as a first-order Markov process

$$\begin{array}{c}\hfill {\dot{\mathit{b}}}_a=-\frac{1}{{\tau }_a}{\mathit{b}}_a+{\mathit{n}}_{{\mathit{b}}_a}\end{array}$$

(9)

with characteristics

$$\begin{array}{c}\hfill \begin{array}{c}\mathrm{E}\left({\mathit{b}}_a\right)={\mathit{0}}_{3\times 1}\hfill \\ \mathrm{E}\left({\mathit{b}}_a^2\right)={\sigma }_{a,bias}^2·f_s·{\mathit{I}}_{3\times 3}\hfill \\ {\mathit{n}}_{{\mathit{b}}_a}=\sqrt{\frac{2f_s·{\sigma }_{a,bias}^2}{{\tau }_a}{\vartheta }_{a,bias}}·{\mathit{I}}_{3\times 1}\hfill \end{array}\end{array}$$

(10)

where the noise driving the bias ${\vartheta }_{a,bias}\sim N(0,1)$, time constant ${\tau }_a$, bias variation ${\sigma }_{a,bias}$ can be determined by Allan variance and autocorrelation analysis.

Considering the lever-arm effect due to the fact that the accelerometer is not at the center of gravity, the accelerometer true vector is

$$\begin{array}{c}\hfill {}^b\mathit{a}={}^b\dot{\mathit{v}}+{}^b\mathit{\omega }\times ({}^b\mathit{\omega }\times \mathit{d})+{}^b\dot{\mathit{\omega }}\times \mathit{d}-\mathit{g}\end{array}$$

(11)

where, $\mathit{d}$ represents the distance between the IMU installation position and the center of gravity position. Beside, ${\mathit{u}}_g={}^b\mathit{\omega }$.

The dynamic characteristics of a tri-axis sensor are generally expressed by a second-order programming transfer function [30] as

$$\begin{array}{c}\hfill G_i\left(s\right)=\frac{w_{i,n}^2}{s^2+2{\xi }_i{w}_{i,n}s+w_{i,n}^2}·e^{-\tau s}\end{array}$$

(12)

where, $w_n$ denotes the natural frequency, $\xi$ denotes the damping ratio, and $\tau$ denotes the time constant.

4.2.2. Compass Model

Because they are both three-axis sensors, magnetic compasses have similar measurement models and dynamic transfer functions to accelerometers or magnetometers, which are expressed as

$$\begin{array}{c}\hfill {\mathit{u}}_m={\mathit{C}}_m{·}^b\mathit{m}+{\mathit{b}}_m+{\mathit{\epsilon }}_m\end{array}$$

(13)

where $C_m$ denotes scale factors and misalignment error, ${\mathit{b}}_m$ is the zero bias, ${\mathit{\epsilon }}_m$ is the rate noise, which have the same statistical characteristics as (8) and (10). And it has the same dynamic characteristics as (12).

4.2.3. Barometer Model

The barometer is a typical uniaxial sensor which measures physical quantities without multi-channel characteristics, and its measurement model is expressed as follows

$$\begin{array}{c}\hfill u_p=p_a+b_p+{\epsilon }_p\end{array}$$

(14)

where $u_p$ is the pressure altitude measurement value, $p_a$ is the true air pressure height, ${\epsilon }_p$ is the barometer drift error which has the same Gaussian white noise characteristics as (8), and $b_p$ is the barometer wind disturbance error [31] which are expressed as

$$\begin{array}{c}\hfill b_p=\frac{T_0}{\beta }\left[{\left(\frac{P_S^{\prime }}{P_0}\right)}^{-\frac{\beta R}{g}}-{\left(\frac{P_S^{\prime }-\frac{1}{2}\rho {\upsilon }^2}{P_0}\right)}^{-\frac{\beta R}{g}}\right]\end{array}$$

(15)

where $T_0=288.15K$ (the capital letter K is the temperature unit of Kelvin) is the standard sea level temperature, $\beta =-6.5K/km$ is the temperature lapse rate constant, $P_0=1013.25hPa$ is the standard sea level pressure, g is the gravity acceleration constant, $R=287.05287m^2/K·s^2$ is the air constant, $\rho =1.23kg/m^3$ is the standard air density, $P_s$ is the measured air pressure value after wind interference, $\upsilon$ is the wind speed.

4.2.4. GPS Receiver Model

In contrast to the aforementioned three types of MEMS measurement sensors, GPS receivers are classified as calculation sensors which receive multiple satellite signals and then calculate the position of the node in the earth Latitude-Longitude-Altitude (LLA) global coordinate.

Define the node position measurement with respect to the Earth fixed coordinate system as ${\mathit{u}}_{GPS}\stackrel{\Delta }{=}\left[\mu \iota h^{\mathrm{T}}\right]$. The measurement model for calculating the position of a node by the GPS receiver [32] is expressed as

$$\begin{array}{c}\hfill {\mathit{u}}_{GPS}=f_{LLA}\left({}^e\mathit{p}\right)+{\mathit{\epsilon }}_{GPS}\end{array}$$

(16)

where $f_{LLA}(·)$ denotes the transformation from Flat Earth to LLA, ${\mathit{\epsilon }}_{GPS}$ is the measurement noise induced by the communication channel with the satellite, which can also be described by the same Gaussian white noise model as (8).

Define the node velocity measurement with respect to the Earth fixed coordinate system as ${}^e\mathit{\nu }_{GPS}={[}^e{\nu }_N^{GPS}{ }^e{\nu }_E^{GPS}{ }^e{\nu }_D^{GPS}{]}^{\mathrm{T}}$. The measurement model for calculating the velocity of a node by the GPS receiver is expressed as

$$\begin{array}{c}\hfill {}^e\mathit{\nu }_{GPS}={\mathit{R}}_b^e·{}^b\mathit{v}+{\mathit{\epsilon }}_{\mathit{\nu }}\end{array}$$

(17)

where ${\eta }_{\mathit{V}}$ f is the measurement noise of the velocity caused by the flying perturbation.

4.3. Communication Adapter Model

In the autopilot system, critical sensors perform efficient data transfer with the processor via board-level communication buses, which involve communication interface protocols such as serial peripheral interface (SPI), inter-integrated circuit (IIC), and universal asynchronous transceiver (UART). In the proposed HIL platform, the communication adapter model simulates the digital control logic unit of the actual sensor chip to enable the conversion of sensor data into binary digital signals suitable for board-level transmission.

Here, we take the sensor products in Pixhawk4 as an example to describe the model of the sensor adapter in detail. The names of the various types of sensor products in the Pixhawk4 autopilot system are as follows: the IMU is ICM20689, the magnetic compass is IST8310, the barometer is MS5611 and the GPS receiver is Ublox M8N GPS. The simulation model of the above four sensor sensors based on the finite state machine(FSM) modeling approach as shown in Figure 4. The model is divided into two major phases: data processing and analogue communication. The data processing phase includes functions such as data type conversion, byte splitting, and FIFO storage, while the analogue communication phase covers response mechanisms such as chip identification, programmable register configuration, and data transmission. For this complex control logic process, the model adopts a finite state machine-based approach to modeling, which ensures the high efficiency of the communication process and the reliability of the system.

5. Sensor Fault Model

Fault is a common phenomenon in which the characteristics or parameters of an electronic equipment system deviate beyond the acceptable range, thereby degrading the performance of the system. The possible malfunction behavior of the autopilot system is concentrated on the sensors.

5.1. Measurement Model Fault Scheme

5.1.1. Scale Fault

As the sensor is exposed to harsh environmental conditions (such as high temperature, high humidity, strong magnetic field, strong corrosion, etc.), the performance of the sensor material may be changed and degraded, resulting in distortion in the output of electronic devices such as transistor resistance, capacitance and inductance in the chip circuit. The scale fault output is shown as sensor data scaling, the fault scheme $\Lambda {\mathit{S}}_j$ is represented as

$$\begin{array}{c}\hfill \Lambda {\mathit{S}}_j={\mathit{S}}_j·\Delta s_j\end{array}$$

(18)

where $\Delta s_j$ indicates the proportional fault factor, j can be replaced by the characters $a,g,m,p,GPS$.

5.1.2. Bias Fault

Sensor bias faults often arise from several causes, such as component aging or damage, temperature effects, power supply variations, mechanical stress or damage, calibration mistakes, and environmental factors like magnetic fields, humidity, or chemical interference. These factors, individually or in combination, cause the sensor output to deviate from the true value by a fixed amount over the entire measurement range, the fault scheme $\Lambda {\mathit{b}}_j$ is represented as

$$\begin{array}{c}\hfill \Lambda {\mathit{b}}_j={\mathit{b}}_j+\Delta b_j\end{array}$$

(19)

where $\Delta b_j$ indicates the bias fault factor.

5.1.3. Drift Fault

Sensor drift faults are the gradual deviation of a sensor’s output from the true value over time, caused by material aging, environmental influences like temperature and humidity, and mechanical wear of components like ADCs and amplifiers. Drift represents a continuous, cumulative change in output, distinct from fixed offsets or proportional changes. For example, a temperature-affected sensor may gradually change its output due to changes in material properties with temperature, or a sensor subjected to mechanical stress over a long period of time may experience a gradual deviation of the output signal from the initial calibration point due to small changes in physical form. The measured signal ${\mathit{u}}_j\left(t\right)$ of a drift fault scheme is expressed as

$$\begin{array}{c}\hfill \Lambda \left({\mathit{u}}_j\left(t\right)\right)={\mathit{u}}_j\left(t\right)+\Delta \kappa (t-t_s)\end{array}$$

(20)

where $\Delta \kappa$ is the drift fault factor, $t_s$ is the occur time.

5.1.4. Noise Fault

Sensor noise faults involve undesired random fluctuations in output, caused by external electromagnetic interference, internal electronic noise like thermal or shot noise, and environmental factors such as mechanical vibrations and temperature changes, particularly in sensitive sensors. The noise fault scheme $\Lambda {\mathit{\epsilon }}_j$ is represented as

$$\begin{array}{c}\hfill \Lambda {\mathit{\epsilon }}_j=\Delta {\epsilon }_j\left(\mathit{N}\right)\end{array}$$

(21)

where $\Delta {\epsilon }_j$ is the noise fault gain factor, $\mathit{N}$ is the noise fault model.

5.1.5. Spike Fault

Sensors with microstructures like cantilevers and membranes can degrade or fail from external shocks, causing ruptures or detachment from anchor points, potentially leading to short circuits or performance issues. For instance, excessive pressure can rupture micro-membranes, and strong impacts can misalign internal elements, like causing a mass block in accelerometers to touch electrodes. Severe impacts may also damage tiny circuits and solder joints, affecting signal transmission and processing. And the spike fault can be regarded as a special form of scale fault, when the amplitude $\Delta s_j$ is as large as possible and the spike fault duration time $\Delta t_j$ is as small as possible, it can be described as follows

$$\begin{array}{c}\hfill \Delta s_j\to \infty ,\Delta t_j\to dt\end{array}$$

(22)

where $dt$ indicates a very small time.

5.2. Communication Model Fault Scheme

5.2.1. Noise Fault

Due to the short distance communication, the sensor communication circuit uses a baseband digital circuit, that is, transmits high-level or low-level signals representing binary digital logic “0” or “1”. The widely used transistor-transistor-logic(TTL) and complementary metal-oxide-semiconductor(CMOS) digital integrated circuit(IC) devices are affected by the characteristics of materials, effect of proton radiation [33], high-energy particles, etc., so the communication circuit does not have the ideal electrical signal characteristics, but parasitics a considerable amount of interference signals. The Figure 5 illustrates the general interference phenomena in digital communication circuits. For different digital circuits, the low-level interval of $0V-V_{off}\left(max\right)$ is generally defined as digital logic “0”, and the high-level interval of $V_{on}\left(min\right)-V_{cc}$ is defined as digital logic “1”. However, there is a “Non-useable area” between $V_{off}\left(max\right)$ and $V_{on}\left(min\right)$, and the level signal falls in this area. Digital circuit logic interpretation will become unpredictable. Figure 5a shows a segment of digital signal that is disturbed by noise. The noise signal in Figure 5b is superimposed on the $V_{cc}$ supply voltage, the voltage signal is kept above the minimum level $V_{on}\left(min\right)$, and the digital circuit logic interpretation will not be affected. But when the noise level becomes sufficiently large and the noise spike causes the high voltage level to drop below this lowest level, the digital circuit may interpret this spike as a logical “0”. When the noise impact fault occurs, the level signal of the pin is a randomly varying 0 or 1, and the fault model is as follows

$$\begin{array}{c}\hfill {\hslash }_N\left(x\left(n\right)\right)=rand\left([0,1]\right).\end{array}$$

(23)

where $x\left(n\right)$ is the sensor chip pin digital signal, n is the sampling time.

5.2.2. Open Circuit Fault

Sensor open-circuit faults typically result from a break in the circuit path, caused by physical damage to wires or solder joints, environmental corrosion, or damage to internal components from overheating or voltage issues, preventing current flow. Sensor open circuit faults can be represented as

$$\begin{array}{c}\hfill {\hslash }_O\left(x\left(n\right)\right)=x\left(n\right)\&{\Delta }_{open}\end{array}$$

(24)

where & denotes $AND$ operation sign, when ${\Delta }_{open}=1$ there is no fault, while when ${\Delta }_{open}=0$ it indicates an open-circuit fault.

5.2.3. Short Circuit Fault

Sensor short-circuit failures occur when an unintended conductive path redirects current flow, caused by insulation damage, moisture intrusion, external conductive materials like metal shavings, or internal component damage due to over-voltage, overheating, or physical shock. Sensor short circuit faults can be represented as

$$\begin{array}{c}\hfill {\hslash }_S\left(x\left(n\right)\right)=x\left(n\right)|{\Delta }_{short}\end{array}$$

(25)

where | denotes $OR$ operation sign, when ${\Delta }_{short}=0$ there is no fault, while when ${\Delta }_{short}=1$ it indicates an open-circuit fault.

5.3. Fault Inject Progress

Above all, we obtain the sensor measurement model fault parameters and communication model fault parameters, and the sensor measurement values with fault modes are further expressed as

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{u}_o^{*}\left(t\right)=\Lambda S_j·u_j\left(t\right)+\Lambda b_j+\Lambda {\epsilon }_j\hfill \\ u_o\left(t\right)=\Lambda \left(u_o^{*}\left(t\right)\right)\hfill \end{array}\right.\end{array}$$

(26)

where $u_o^{*}\left(t\right)$ represents the node measurement value before the drift fault processing unit in the signal chain of the measurement model. Likewise, the sensor communication digital signal with failure mode is further expressed as

$$\begin{array}{c}\hfill \left\{\begin{array}{c}x\left(n\right)={\hslash }_N\left(x^{*}\left(n\right)\right)\hfill \\ x\left(n\right)={\hslash }_O\left(x^{*}\left(n\right)\right)\hfill \\ x\left(n\right)={\hslash }_S\left(x^{*}\left(n\right)\right)\hfill \end{array}\right.\end{array}$$

(27)

where $x^{*}\left(n\right)$ represents the measurement value of the intermediate node in the communication model signal chain.

In addition, according to the form of fault existence, it can be divided into intermittent faults and permanent faults, which shows that faults have a time dimension. Therefore, the fault duration time $\Delta t_d$ needs to be fully considered when designing the failure mode, indicating that the sensor is in an unhealthy state during this period.

We can develop a test case library necessary for the fault injection process based on engineers’ development and testing experience and civil aircraft airworthiness test items and evaluation indicators. In order to form a standard test process, we stipulate that each test case command sequence needs to set the fault injection trigger time $t_s$, test stop time $t_e=t_s+\Delta t_d$, expected sensor fault components, fault impact factors as well as flight tasks, etc. Therefore, we can achieve the purpose of injecting faults online by modifying the fault model parameters.And the software and hardware platforms involved, are shown in Table 1.

6. Experiments and Verification

In this section, we design an MPSoC-FPGA-based HIL platform shown in the Figure 6 to fulfill the functional requirements of the testbed. First, we develop and validate the unmanned vehicle model, sensor model and fault model in Simulink and Stateflow environments based on the MBD modeling approach. Then the embedded code of each model is generated by the automatic code generation tool, followed by adding the fault injection interface and simulation data sending interface program in the IDE. Finally, the executables are cross-compiled and imported into the testbed device to run. In terms of performance, the platform uses two ARM-A53 1.2GHz cores to ensure that the unmanned vehicle model, the sensor measurement model and its part of the fault model computation results are accurate and up to 2KHz running frequency, and uses FPGA 100MHz to ensure that the sensor communication model and its part of the fault model ultra-high update frequency requirements.

6.1. HIL Platform Function Verification

The most distinctive feature of the proposed HIL platform is to simulate the working mode of actual sensor products. In the case of the flight autopilot system, data interaction with sensors through digital communication is the same as the workflow of natural flight. The solution adopted in this paper removes the limitation of general hardware in the loop simulation scheme, so it has broader support. The functional verification of platform suitability was carried out from three aspects: flight control software platform, flight control hardware platform and unmanned system type.

We selected a number of representative hardware and software platforms that are widely used in scientific research and commercial fields for experimental verification. As shown in Figure 7, we selected two hardware platforms, PX4 and APM, and three hardware platforms, Holybro Pixhawk 4, CubePilot Cube Orange, and FEISILAB ZYH7. The two critical points of the HIL platform to adapt different kinds of flight controllers are as follows: the software platform modifies configuration options to select the access mode of sensors as external, and it is necessary to shield internal sensors. The hardware platform needs to reserve the necessary communication interface, optional with multi-function carrier board.

We established a variety of analytical models of unmanned systems, including multi-rotor model, fixed wing model, unmanned ship model and unmanned vehicle model, and completed the adaptation verification of the above four unmanned systems. This work shows that the proposed platform can be easily compatible with a variety of typical unmanned systems.

6.2. Simulation Reliability Assessment

To verify the reliability of the HIL platform, experiments and simulations are performed with accelerometer temperature drift fault test presented in Figure 8a. In the horizontal static environment, we obtained the acceleration values of x-axis, y-axis and z-axis of the ICM20689 sensor onboard the Pixhawk4 autopilot under the temperature variation range of ${17}^{\circ }C-{50}^{\circ }C$ are presented as the ‘EXP Data’ curve in Figure 8b, from which it can be clearly observed that the accelerometer appears drift fault phenomenon with the change of temperature.

From the data curve, the curves of simulation and experiment are highly coincident, which reveals the performance ability of HIL platform to simulate real fault behavior. Further, we adopted the evaluation method in literature [34] to quantitatively analyze the HIL reliability. We choose the performance credibility index ${\eta }_p$ here. The root mean square error $\sigma$ is selected as the performance parameter, the threshold parameter ${\epsilon }_{\mathrm{p}}\approx 10\%·\left|p_e\right|$ and the the simulation error for each axis is $e_p=\left|{\sigma }_e-{\sigma }_s\right|$. In the expectation of achieving a desired passing mark ${\eta }_t\ge 0.6$, the credibility evaluation index ${\eta }_p$ was calculated as followed

$$\begin{array}{c}\hfill {\eta }_p^i=\frac{0.75·{\epsilon }_p^i}{\sqrt{{\left(0.75·{\epsilon }_p^i\right)}^2+{e_p^i}^2}}\end{array}$$

(28)

where i represents the x-axis, y-axis and z-axis. Then, the simulation credibility for each period ${\eta }_p$ is obtained with the results listed in the Table 2. By combining accelerometer x-axis, y-axis, and z-axis credibility indices, the average simulation credibility is ${\overline{\eta }}_{\mathrm{t}}=92.1251\%$. Since the obtained credibility index ${\overline{\eta }}_{\mathrm{t}}$ is far above the passing mark ${\eta }_p\ge 0.6$, the simulation results can be considered as credible as experimental results.

6.3. Proposed Platform Application

6.3.1. Fault Generation

Multiple UAV flight missions are planned in the HIL platform, then injecting scale fault, bias fault, drift fault, noise fault, spike fault, internal open-short circuit fault for sensitive element, and open/short-circuit for the communication circuit into the accelerometer during the flight, as shown in Figure 9, we choose a simple expert model to perform fault identification and health assessment on fault data. It is worth noting that fault diagnosis and health assessment algorithms are not the core content of this article. This application case shows that test cases with multiple sensors, multiple failure modes, and different fault impact levels can be injected into the proposed HIL platform. The outstanding contribution of this work is that we can provide high-quality and large-scale test data for health management research work.

6.3.2. Reliability Testing

Here we present a study case of a UAV reliability flight. The autonomous flight mission shown in Figure 10b is scheduled in the HIL platform, then the accelerometer noise fault and open-circuit fault for communication circuit are injected successively in the fight mission. The data recorded by the HIL platform is shown in Figure 10, and the fault effect is shown in Figure 10a. The position and attitude of the drone are shown in Figure 10b. The test results as shown in Figure 10c presented the noise fault caused the drone body to undergo severe vibration and had a certain impact on the scheduled mission. On the other hand, the drone did not lose control due to severe noise fault, which shows that the tested unmanned system software platform has strong anti-noise performance; However, when the accelerometer suffered an open-circuit fault, the drone crashed immediately, causing huge damage to the fuselage. This shows that the unmanned system under test did not have hardware redundancy design.

This application case shows that we can perform reliability testing of unmanned systems on the proposed HIL platform in a low-cost and efficient manner. When the unmanned system model is sufficient and the fault model is rich enough, in theory, the relevant testing work of any unmanned system can be completed on the proposed HIL platform. In addition, since the proposed HIL platform supports complete flight testing and fault injection, it can also perform fault tolerant control of dynamic systems, and the research work [35] has been well verified.

6.4. Videos and Source Code

The video gives an overall introduction to the test platform and an operational demonstration of fault generation, using the PX4 software platform, Pixhawk4 hardware platform and DJi F450 quadcopter drone:

https://youtu.be/CMeeeZYQ1eQ

We have released the Matlab/simulink code:

https://github.com/MsXFang/MPSoC-FPGA-HIL-Platform

Engineers can easily adapt to the target open source platform by simply modifying the unmanned vehicle model and sensor model parameters.

7. Conclusions

Aiming at the demand of reliability verification of automatic driving system, this paper proposes an economical and practical HIL platform. In this paper, coarse-grained simulation models of various sensor products for unmanned vehicle and autopilot systems are first given. Then, the multi-mode failure models of various sensors are further given, which can realize credible chip-level failure simulation. Finally, the design and development of the HIL platform is completed on the MPSoC-FPGA development board, and the compatibility, accuracy and functionality of the HIL platform are further verified. The results show that the HIL platform proposed in this paper has outstanding advantages in intelligent unmanned control system. Future work will focus on refining various behavioral-level fault models so as to provide a richer library of fault use cases.