Control System Design of a Pitch‐Decoupled VTOL UAV Using Reinforcement Learning

1. Introduction

Unmanned aerial vehicles (UAVs), including vertical takeoff and landing (VTOL) UAVs are currently widely used in both civilian and military applications. The demand for UAVs continues to grow because of their extensive range of applications in many important areas, including rescue operations, film production, package delivery, aerial photography, agriculture, reconnaissance, and military operations. Such wide-ranging applications stimulate the development of new UAV models that are more practical, versatile, easy to control, and energy-efficient.

The design and manufacturing of contemporary VTOL UAVs encounter various challenges, including control-system accuracy, energy consumption related to structural and mechanical design, and versatility of application. VTOL UAVs represent one of the most rapidly developing and scientifically significant classes of UAVs. The design of their control systems often requires fundamentally new approaches and methodologies, owing to their structural characteristics and diverse flight regimes.

Generally, there are three main types of multirotor VTOL UAVs, as well as several non-standard configurations [1,2,3]. The simplest configuration, typically used in small-sized vehicles, is a hybrid model that combines a multirotor UAV and a fixed-wing UAV (Figure 1(a)). The other two principal types of multirotor VTOL UAVs, conventionally referred to as tilt-rotor and tilt-wing VTOL UAVs, are widely used in practical applications (Figure 1(b),(c)) [4]. The main distinction between tilt-rotor and tilt-wing VTOL UAVs and the hybrid model in Figure 1(a) is the requirement for additional actuators (servomotors), which rotate either the rotors relative to the wings or the entire wings with fixed rotors relative to the UAV body. These rotations are necessary when switching between vertical takeoff and landing modes and the horizontal flight mode [5].

Irrespective of the multirotor VTOL UAV type, there are three distinct flight phases or modes:

• Multirotor mode (vertical takeoff and/or landing),
• Transition mode (switching between vertical and horizontal flight),
• Fixed-wing mode (horizontal flight).

Essentially, the control of takeoff, landing, and hovering modes can be achieved using conventional control algorithms developed for standard multirotor UAVs (quadcopters, octocopters, etc.) and fixed-wing aircraft. However, the transition mode is unique to VTOL UAVs and presents numerous challenges because of the highly nonlinear and complex nature of the vehicle dynamics during this mode. In addition, each VTOL UAV configuration shown in Figure 1 requires individual consideration because of its specific structural characteristics.

During the transition phase, strong mechanical and aerodynamic forces act on the UAV body due to variations in the thrust-vector angle. This may result in thrust loss, attitude instability, and increased complexity of the overall control problem.

In general, traditional linear controllers, such as PID, LQR, and pole-placement controllers, do not perform satisfactorily during the transition phase. Consequently, many researchers investigate various nonlinear control approaches and hybrid strategies, including gain-scheduling, sliding mode control, backstepping, and others. In some studies, the transition-control problem is often formulated as an optimal-control problem aimed at deriving optimal control laws for the transition phase. For example, in [6], the transition control of a ducted-fan VTOL UAV is formulated as a constrained Markov decision process, and constrained policy optimization is used to learn safe control policies that regulate state transitions while satisfying safety constraints. However, the solutions proposed in [6] are rather sophisticated and require significant computational resources and complex numerical algorithms [7,8].

One of the major drawbacks of the tilt-rotor and tilt-wing VTOL UAVs in Figure 1(b) and 1(c), compared with the hybrid configuration in Figure 1(a), is the necessity for additional actuators that provide transitions between takeoff/landing and horizontal flight modes.

To overcome this disadvantage, Apkarian [9,10,11] proposed a new type of tilt-rotor VTOL UAV called the pitch-decoupled UAV, which was patented in the United States [9]. The overall view of the pitch-decoupled VTOL UAV equipped with a two-degree-of-freedom robotic arm is shown in Figure 2. The main idea behind J. Apkarian’s patent is illustrated in Figure 3.

Similar to standard hybrid VTOL UAV in Figure 1(a), the fuselage of the pitch-decoupled UAV consists of two parts: the quadcopter part and the fixed-wing aircraft part (hereafter referred to as the quad and wing parts). The rotors are passively coupled to the aircraft frame about the pitch axis through a parallelogram hinge mechanism. During takeoff and landing, the propellers are positioned vertically; therefore, the UAV operates similarly to a conventional quadcopter (Figure 2 (a)). During longitudinal (horizontal) flight, the propellers rotate by ${90}^{\circ }$ degrees relative to the UAV frame and function similarly to those of a standard tilt-rotor VTOL UAV (Figure 2 (b)). A unique feature of this design is that the front and rear propellers are kinematically linked, and their rotation relative to the UAV body is achieved not by additional servomotors, as in the configuration shown in Figure 1(b), but by differential thrust generated by the front and rear rotor pairs. To improve control of the pitch axis, the UAV is additionally equipped with a fifth rotor located near the tail. The VTOL UAV also includes ailerons and an elevator to provide additional flight control during fixed-wing operation.

The main structural and kinematic features of the pitch-decoupled VTOL UAV, compared with a conventional tilt-rotor UAV, are considered in [10]. Based on a simplified mathematical model, it is shown that the vertical component of the rotor thrust depends only on the angle $\mathit{\gamma }$, which is defined as the difference between aircraft pitch angle and the relative tilt angle of the rotor hinge mechanism. The major challenges associated with controlling this type of VTOL UAV using cascaded PID controllers are discussed in the tutorial [11].

The novel pitch-decoupled VTOL UAV concept has attracted considerable attention from researchers in this field. In [12], a single PID-based controller is proposed that seamlessly manages the transition from vertical to longitudinal flight. A more sophisticated nonlinear dynamic model of the pitch-decoupled VTOL UAV, formulated as a constrained system of three rigid bodies using the Lagrangian approach, is presented in [13]. The model developed in [13] also includes an aerodynamic subsystem that generates forces and moments acting on the wing and fuselage of the aircraft. A detailed mathematical model of the pitch-decoupled VTOL UAV equipped with a robotic arm is developed in [14], and a PID-based control system for hovering mode is investigated using a Simulink model.

This paper is devoted to the development of a neural controller for a pitch-decoupled VTOL UAV equipped with a robotic arm, which provides smooth transitions between vertical takeoff and landing modes and horizontal flight mode.

The paper is organized as follows. Section 2 presents the kinematic analysis of the pitch-decoupled VTOL UAV equipped with a two-degree-of-freedom (DOF) robotic arm using the Denavit-Hartenberg formalism. In Section 3, the dynamic equations of the UAV’s angular rotations are derived using the Euler-Lagrange equations. Section 4 presents the translational dynamics of the UAV. Section 5 is devoted to the design of the attitude control system based on the developed mathematical model and conventional PID controllers. Section 6 presents the design of a matrix neural controller based on the reinforcement learning actor-critic method.

2. Kinematics of the Pitch-Decoupled UAV

In this section, a detailed kinematic scheme and the kinematic equations of the pitch -decoupled VTOL UAV equipped with a two-DOF robot manipulator are developed based on the parameters, which can be regarded as certain modifications of the well-known Denavit-Hartenberg (DH) parameters used in robotics [15]. Such an approach allows both the relative angular positions and linear displacements to be represented in a united form for an arbitrary number of reference frames (coordinate systems) using homogeneous transformation matrices [16,17]. The generalized coordinates and reference frames chosen for the derivation of kinematic equations are shown in Figure 4, where the quad part of the VTOL UAV is modeled as a flat plate with four fixed rotors.

Let $\left\{O_I\right\}$ denote a right-hand inertial reference frame with the origin at $O_I$ and the axes denoted $X_I,$

$Y_I,$ and $Z_I.$, and let $\left\{O_i\right\}$($i=0,1,\mathrm{2...},6$) denote orthogonal reference frames with origins at $O_i$ and axes $X_i,$

$Y_i,$ and $Z_i.$

In Figure 4, $\left\{O_0\right\}$ is a reference frame with the origin $O_0$ at the center of mass (CM) of the VTOL UAV with the robotic arm. The axes of $\left\{O_0\right\}$ are aligned with the axes of the inertial frame $\left\{O_I\right\}$. The orientation of the VTOL UAV with respect to $\left\{O_0\right\}$ is determined by three successive rotations of the intermediary frames $\left\{O_1\right\}$, $\left\{O_2\right\},$ and $\left\{O_3\right\}$ through the yaw, roll, and pitch angles. These angles determine the first three generalized coordinates $q_1$ (yaw), $q_2$ (roll) and $q_3$ (pitch) of the UAV. The corresponding homogeneous transformation matrix can be written in the form:

$T_3(q_1,q_2,q_3)=\left[\begin{array}{l}{R}_0(q_1,q_2,q_3) 0_{3x1}\\ 0_{1x3} 1\end{array}\right]$ (1)

In the matrix $T_3(q_1,q_2,q_3)$ (1), $0_{3x1}$ and $0_{1x3}$ are zero vectors of the indicated sizes, and$R_0(q_1,q_2,q_3)$ is the following orthogonal rotation matrix [18]:

$R_0(q_1,q_2,q_3)=\left[\begin{array}{l}c{q}_{1}c{q}_3-s{q}_{1}s{q}_{2}s{q}_3 -c{q}_{2}s{q}_1 c{q}_{1}s{q}_3+c{q}_{3}s{q}_{1}s{q}_2 \\ c{q}_{3}s{q}_1+c{q}_{1}s{q}_{2}s{q}_3 c{q}_{2}c{q}_1 s{q}_{1}s{q}_3-c{q}_{1}c{q}_{3}s{q}_2\\ -c{q}_{1}s{q}_3 s{q}_2 c{q}_{2}c{q}_3\end{array}\right],$ (2)

where c and s are shorthand forms for cosine and sine, respectively. Note that the matrix$R_0(q_1,q_2,q_3)$ (2) determines the orientation of the UAV’s body-fixed frame $\left\{O_3\right\}$ with respect to the inertial frame $\left\{O_I\right\}$.

The next two generalized coordinates, $q_4$ and $q_5$, describe rotations of the robotics arm with respect to the body-fixed frame $\left\{O_3\right\}$. Finally, the sixth generalized coordinate, $q_6$, is the pitch angle of the quad part of the UAV relative to the wing part [10]. Each of these three generalized coordinates $q_4, q_5,$and $q_6$ has an associated reference frame and a corresponding homogeneous transformation matrix $A_i\left(q_i\right)$. To develop the configuration kinematic equations for the pitch-decoupled UAV, we introduce the following DH-like parameters and basic transformations: $a_i$, the linear displacement of the reference frame $\left\{O_i\right\}$ relative to $\left\{O_{i-1}\right\}$ along the axis $X_i$; $d_i,$ the linear displacement of the reference frame $\left\{O_i\right\}$ relative to $\left\{O_{i-1}\right\}$, along the axis $Z_i$; ${\alpha }_i,$ the rotation (twist) of $\left\{O_i\right\}$ about the axis $X_i$; $q_i,$the rotation of $\left\{O_i\right\}$ about the axis $Z_i$. Each basic transformation has an associated homogeneous transformation matrix. The product of these matrices yields the resulting homogeneous transformation matrix

$A_i\left(q_i\right)= \left[\begin{array}{cccc}\cos \left(q_i\right)& -\sin \left(q_i\right),& 0& a_i\\ \cos \left({\alpha }_i\right)\sin \left(q_i\right)& \cos \left({\alpha }_i\right)\cos \left(q_i\right)& -\sin \left({\alpha }_i\right)& 0\\ \sin \left({\alpha }_i\right)\sin \left(q_i\right)& \cos \left(q_i\right)\sin \left({\alpha }_i\right)& \cos \left({\alpha }_i\right)& d_i\\ 0& 0& 0& 1\end{array}\right]$ (3)

I = 4, 5, 6.

The introduced parameters $a_i, d_i, {\alpha }_i, q_i$, and the matrix $A_i\left(q_i\right)$ (3) differ from the standard DH parameters [15] but are more convenient for describing the kinematics of the UAV in Figure 4. Based on the kinematic model shown in Figure 4, we have

$A_4\left(q_4\right)=\left[\begin{array}{l}c{q}_4 -s{q}_4 0 0\\ s{q}_4 c{q}_4 0 0\\ 0 0 1 d_4\\ 0 0 0 1\end{array}\right]$, $A_5\left(q_5\right)= \left[\begin{array}{l} c{q}_5 -s{q}_5 0 0\\ 0 0 1 0\\ -s{q}_5 -c{q}_5 0 d_5 \\ 0 0 0 1\end{array}\right],$ (4)

$A_6\left(q_6\right)=\left[\begin{array}{l}c{q}_6 -s{q}_6 0 a_6\\ 0 0 1 0\\ -s{q}_6 -c{q}_6 0 0\\ 0 0 0 1\end{array}\right]$ (5)

Besides the presented reference frames $\left\{O_0\right\}-\left\{O_6\right\}$, an additional frame $\left\{O_b\right\}$ is introduced. This frame is not associated with any generalized coordinate but is required to describe the position and orientation of the tail rotor and of the quad part of the UAV. The corresponding homogeneous transformation matrix has a simple form

$A_b= \left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& d_b\\ 0& 0& 0& 1\end{array}\right]$ (6)

By multiplying the introduced matrices $A_i\left(q_i\right)$ in accordance with the kinematic model in Figure 4, we obtain the position and orientation of each reference frame relative to the initial reference frame $\left\{O_0\right\}$. Performing the multiplications yields the following homogeneous transformation matrices:

$T_4=T_3{A}_4, T_5=T_4{A}_5=T_3{A}_4{A}_5, T_b=T_3{A}_b, T_6=T_b{A}_6.$ (7)

To calculate the torques generated by the rotors, it is necessary to determine the rotor displacements with respect to the CM $O_0$ of the UAV. Note that the positions $r_1^6, r_2^6,$

$r_3^6, r_4^6$of the four rotors on the quad part in the reference frame ${\{O}_{6\}}$ are determined as follows:

$r_1^6=\left[\begin{array}{l}0.7071L_R\\ 0\\ -0.7071L_R\\ 1\end{array}\right] ; r_2^6=\left[\begin{array}{l}-0.7071L_R\\ 0\\ 0.7071L_R\\ 1\end{array}\right] ; r_3^6=\left[\begin{array}{l}0.7071L_R\\ 0\\ 0.7071L_R\\ 1\end{array}\right] ; r_4^6=\left[\begin{array}{l}-0.7071L_R\\ 0\\ -0.7071L_R\\ 1\end{array}\right] ,$ (8)

where $L_R$ is the distance from the rotors to the origin $O_6,$ and the fourth, a unity, coordinate is added to conform with the homogeneous matrix notations.

Similarly, the displacement vector of the fifth (tail) rotor in the reference frame $\left\{O_b\right\}$ can be defined as

$r_5^{O_b}={\left[-\begin{array}{cccc}{L}_5& 0& 0& 1\end{array}\right]}^T. \left(9\right)$

Multiplication of the rotor displacement vectors by the corresponding transformation matrices yields

$r_1^0=T_6{r}_1^6, r_2^0=T_6{r}_2^6, r_3^0=T_6{r}_3^6, r_4^0=T_6{r}_4^6, r_5^0=T_b{r}_5^{0_b},$ (10)

the first three coordinates of which determine the rotor displacements (rotors positions vectors) in the reference frame ${\{O}_o$}, which is attached to the CM of the system. In what follows, we denote the indicated three-dimensional vectors by $r_1^R, r_2^R, r_3^R, r_4^R, r_5^R$. Furthermore, let us denote ${\gamma }_6$ a three-dimensional vector composed of the first three entries of the second column of the matrix $T_6$, and $Z_b$, a vector formed in the same manner from the third column of $T_b$:

${\gamma }_6=\left[\begin{array}{l}{T}_6(1,2)\\ T_6(2,2)\\ T_6(3,2)\end{array}\right], Z_b=\left[\begin{array}{l}{T}_b(1,3)\\ T_b(2,3)\\ T_b(3,3)\end{array}\right].$ (11)

Note that the vector ${\gamma }_6$ determines the direction of the thrusts $F_1, F_2,$

$F_3, F_4$, generated by the first four rotors, in the frame $\left\{O_6\right\}$, and $Z_b$ determines the direction of the thrust $F_5$ in the frame $\left\{O_b\right\}$.

Then, the total torque $\overline{M}$generated by the five rotors can be expressed as

$\overline{M}=\left[M_b+k_{\phi }^5{F}_5{Z}_b\right] + {\displaystyle \sum _{i=1}^4\left[M_i+{\left(-1\right)}^{i-1}{k}_{\psi }{F}_i{\gamma }_6\right]} .$ (12)

In the equation (12),

$\begin{array}{l}{M}_i=-\left(r_i^R\times {\gamma }_6\right)F_i,\\ i=1,2,3,4\end{array}$ (13)

$M_b=\left(r_5^R\times Z_b\right)F_5,$ (14)

where $\times$ is the symbol of cross product, are the torques acting on the UAV and generated by all five rotors, $k_{\psi }$ is a constructive parameter determining the reaction torques acting on the airframe by the rotors of the quad part, and $k_{\phi }^5$ is the same parameter of the tail (fifth) rotor [3].

Note that in the vertical takeoff and landing flight modes, the generalized coordinate $q_6$ (the passive pitch angle $\gamma$ of the quad part) is equal to zero and the total vertical thrust $F_{\Sigma }$ of the quad part rotors is

$F_{\Sigma }=F_1+F_2+F_3+F_4.$ (15)

This thrust is directed along the body-fixed axis $Z_3$. In the horizontal flight mode, the passive pitch angle is equal to ${90}^{\circ }$, and the total thrust $F_{\Sigma }$ (15) is directed along the axis $X_3$.

It should also be noted that, besides the rotor thrusts, the generalized coordinates$q_1,$

$q_2,$

$q_3$, which determine the orientation of the UAV, are affected by tilts of the elevator and the ailerons. This point is discussed further in the derivation of the dynamics equations of the pitch-decoupled UAV.

The block diagram shown in Figure 5 illustrates schematically the kinematic links of the pitch-decoupled UAV.

Here, the arrows directed from left to right represent the torques or forces generated by the rotors, actuators of the robotic arm ($F_4, F_5$), the ailerons ($F_6$), and the elevator plus the fifth (tail) rotor$(F_7);$ ${\tau }_r$ and ${\tau }_y$ are the torques generated on the quad frame about the roll and yaw axes, respectively, and $R_{\gamma }$ is the rotation matrix

$R_{\gamma }=\left[\begin{array}{l}\cos (\gamma ) -\sin (\gamma )\\ \sin (\gamma ) \cos (\gamma )\end{array}\right]=\left[\begin{array}{l}\cos (q_6) -\sin (q_6)\\ \sin (q_6) \cos (q_6)\end{array}\right]$ (16)

3. Rotational Dynamics of the Pitch-Decoupled UAV

The dynamics equations of angular rotations of the pitch-decoupled VTOL UAV are derived using the Euler-Lagrange equations, which for a mechanical system with six degrees of freedom have the following form [19,20]:

$\begin{array}{l}{\displaystyle \frac{d}{dt}}{\displaystyle \frac{\partial L}{\partial {\dot{q}}_i}}-{\displaystyle \frac{\partial L}{\partial q_i}}=F_i\\ i=1,2,3,4,5,6,\end{array}$ (17)

where

$L=K-P$ (18)

is the Lagrangian, defined as the difference between the total kinetic $K$ and potential $P$ energies; $q_i$ are the generalized coordinates, and $F_i$ are generalized forces arising from all external and dissipative forces and torques, including those arising from potential energy and from the elevator and ailerons deflections [21].

Let $\omega$ denote the vector of angular velocity of the UAV’s wing part (of the fuselage), to which the body-fixed frame $\left\{O_3\right\}$ is attached. Then the kinetic energy $K_0$ of the wing part (or, in other words, of the UAV’s fuselage) is equal to

$K_0={\displaystyle \frac{1}{2}}{\omega }^T{I}_0\omega ,$ (19)

where

$J_0=\left[\begin{array}{l} I_{0xx} -I_{0xy} -I_{0xz}\\ - I_{0xy} I_{0yy} -I_{0yz}\\ - I_{0xz} -I_{0yz} I_{0zz}\end{array}\right]$ (20)

is the inertia tensor of the fuselage.

The vector $\omega$ in (19) can be parameterized by generalized coordinates $q_1$, $q_2$, and $q_3$ (by yaw, roll, and pitch angles) as [22]

$\omega =\omega \left(q_0,{\displaystyle \frac{d{q}_0}{dt}}\right)=S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}},$ (21)

where $q_0={\left[q_1,q_2,q_3\right]}^T$, and

$S(q_0)=S(q_1,q_2,q_3)=\left[\begin{array}{l} -\cos (q_2)\sin (q_3) \cos (q_3) 0\\ \sin (q_2) 0 1\\ \cos (q_2)\cos (q_3) \sin (q_3) 0\end{array}\right].$ (22)

Then, the kinetic energy $K_0$ (19) can be rewritten in the form

$K_0={\displaystyle \frac{1}{2}}{\displaystyle \frac{d{q}_0^T}{dt}}{S}^T\left(q_0\right)I_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}.$ (23)

Note that the generalized coordinates $q_4,$

$q_5$, and $q_6$ in the kinematic model shown in Figure 4 are associated with three rigid bodies. Each of these bodies (two links of the robotic arm and the quad part) has one degree of freedom relative to the bodies, to which they are attached. This allows the kinetic energies $K_4,K_5$, and $K_6$ of these bodies to be represented in a unified form, using the homogeneous transformation matrices, as

$K_4={\displaystyle \frac{1}{2}}{\displaystyle \sum _{j=1}^4{\displaystyle \sum _{k=1}^{4}tr\left(\frac{\partial T_4}{\partial q_j}{J}_4\frac{\partial T_4^T}{\partial q_k}\right)\frac{d{q}_j}{dt}\frac{d{q}_k}{dt}}},$ (24)

$K_5={\displaystyle \frac{1}{2}}{\displaystyle \sum _{j=1}^5{\displaystyle \sum _{k=1}^{5}tr\left(\frac{\partial T_5}{\partial q_j}{J}_5\frac{\partial T_5^T}{\partial q_k}\right)\frac{d{q}_j}{dt}\frac{d{q}_k}{dt}}},$ (25)

$\begin{array}{l}{K}_6={\displaystyle \frac{1}{2}}{\displaystyle \sum _{j=1}^3{\displaystyle \sum _{k=1}^{3}tr\left(\frac{\partial T_6}{\partial q_j}{J}_6\frac{\partial T_6^T}{\partial q_k}\right)\frac{d{q}_j}{dt}\frac{d{q}_k}{dt}+{\displaystyle \sum _{j=1}^{3}tr\left(\frac{\partial T_6}{\partial q_j}{J}_6\frac{\partial T_6^T}{\partial q_6}\right)\frac{d{q}_j}{dt}\frac{d{q}_6}{dt}}+}}\\ {\displaystyle \frac{1}{2}}tr\left({\displaystyle \frac{\partial T_6^T}{\partial q_6}}{J}_6{\displaystyle \frac{\partial T_6}{\partial q_6}}\right){\left({\displaystyle \frac{d{q}_6}{dt}}\right)}^2.\end{array}$ (26)

where

$J_i=\left[\begin{array}{l}\left(-I_{ixx}+I_{iyy}+I_{izz}\right)/2 I_{ixy} I_{ixz} m_i{\overline{x}}_i\\ I_{ixy} \left(I_{ixx}-I_{iyy}+I_{izz}\right)/2 I_{iyz} m_i{\overline{y}}_i\\ I_{ixz} I_{iyz} \left(I_{ixx}+I_{iyy}-I_{izz}\right)/2 m_i{\overline{z}}_i\\ m_i{\overline{x}}_i m_i{\overline{y}}_i m_i{\overline{z}}_i m_i \end{array}\right]$ (27)

$i=4,5,6,$are the pseudo inertia matrices, and $m_i{\overline{x}}_i,m_i{\overline{y}}_i,$ $m_i{\overline{z}}_i$ are first inertia moments with respect to the corresponding rotation axis [20,21].

It should be noted that the transformation matrix $T_6$ in (26) does not depend on the generalized coordinates $q_4$ and $q_5$.

To make the following expressions more compact, let us introduce special notations for the traces of transformation matrices:

$\begin{array}{l}{a}_{jk}^{(4)}(q)=tr\left({\displaystyle \frac{\partial T_4}{\partial q_j}}{J}_4{\displaystyle \frac{\partial T_4^T}{\partial q_k}}\right), \\ j,k=1,2,3,4,\end{array}$ (28)

$\begin{array}{l}{a}_{jk}^{(5)}(q)=tr\left({\displaystyle \frac{\partial T_5}{\partial q_j}}{J}_5{\displaystyle \frac{\partial T_5^T}{\partial q_k}}\right), \\ j,k=1,2,3,4,5,\end{array}$ (29)

$\begin{array}{l}{a}_{jk}^{(6)}(q)=tr\left({\displaystyle \frac{\partial T_6}{\partial q_j}}{J}_6{\displaystyle \frac{\partial T_6^T}{\partial q_k}}\right), \\ j,k=1,2,3,6.\end{array}$ (30)

Then the total kinetic energy of the UAV’s rotational motion can be represented in the form

$\begin{array}{l} K= K_0 +K_4 +K_5 +K_6 ={\displaystyle \frac{1}{2}}{\displaystyle \frac{d{q}_0^T}{dt}}{S}^T\left(q_0\right)I_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}+{\displaystyle \frac{1}{2}}{\displaystyle \sum _{j=1}^4{\displaystyle \sum _{k=1}^4{a}_{jk}^{(4)}(q)\frac{d{q}_j}{dt}\frac{d{q}_k}{dt}}} +\\ {\displaystyle \frac{1}{2}}{\displaystyle {\displaystyle \sum _{j=1}^5}{\displaystyle \sum _{k=1}^5{a}_{jk}^{(5)}(q)\frac{d{q}_j}{dt}\frac{d{q}_k}{dt}}} +{\displaystyle \frac{1}{2}}{\displaystyle {\displaystyle \sum _{j=1}^3}{\displaystyle \sum _{k=1}^3{a}_{jk}^{(6)}(q)\frac{d{q}_j}{dt}\frac{d{q}_k}{dt}}}+{\displaystyle {\displaystyle \sum _{k=1}^3}{a}_{k6}^{(6)}\frac{d{q}_k}{dt}\frac{d{q}_6}{dt}}+{\displaystyle \frac{1}{2}}{a}_{66}^{(6)}(q){\left({\displaystyle \frac{d{q}_6}{dt}}\right)}^2.\end{array}$ (31)

The potential energy of the pitch-decoupled UAV is given by the equation

$P=-m_0\overline{g} {\overline{r}}_0-{\displaystyle \sum _{i=4,5,6}{m}_i\overline{g} }{T}_i{}^i\overline{r}_i,$ (32)

where $\overline{g}=\left[g_x g_y g_z 0\right]$ is the gravitational vector, $m_0, m_i (i=4,5,6)$ are the masses of the fuselage and the corresponding rigid bodies, ${}^i\overline{r}_i$ are the positions of the CM of the $i$th body in the associated coordinate frame [21].

Substitution of the equations (31) and (32) into (17) yields the dynamics equations of the pitch-decoupled UAV. For the vector $q_0$ of the Euler angles $q_1,$

$q_2,$ and $q_3$ we have:

$S^T\left(q_0\right)I_{0}S\left(q_0\right){\displaystyle \frac{d^2{q}_0}{d{t}^2}}+S^T\left(q_0\right)I_0{\displaystyle \frac{dS\left(q_0\right)}{dt}}{\displaystyle \frac{d{q}_0}{dt}}+{\displaystyle \frac{d{S}^T\left(q_0\right)}{dt}}{I}_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}} -$

${\displaystyle \frac{\partial }{\partial q_0}}\left[{\displaystyle \frac{d{q}_0^T}{dt}}{S}^T\left(q_0\right)I_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\right] + {\Xi }_{123}=S^T\left(q_0\right)\overrightarrow{M}+S^T(q_o) \left[\begin{array}{l}0\\ M_{Ai}\\ M_{El}\end{array}\right]-\left[\begin{array}{l}{M}_4\\ M_5\\ 0\end{array}\right],$ (33)

where

${\displaystyle \frac{\partial }{\partial q_0}}\left[{\displaystyle \frac{d{q}_0^T}{dt}}{S}^T\left(q_0\right)I_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\right]= \left[\begin{array}{l}{\displaystyle \frac{d{q}_0^T}{dt}}{[{\partial }_{q_1}S\left(q_0\right)]}^T{I}_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\\ {\displaystyle \frac{d{q}_0^T}{dt}}{[{\partial }_{q_2}S\left(q_0\right)]}^T{I}_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\\ {\displaystyle \frac{d{q}_0^T}{dt}}{[{\partial }_{q_3}S\left(q_0\right)]}^T{I}_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\end{array}\right]$ , (34)

${\partial }_{q_i}$ denotes the partial derivative with respect to $q_i$; $\overrightarrow{M}$ is the vector of torques (12); $M_{Ai}$ and $M_{El}$ are the torques produced by the ailerons and elevator tilts; $M_4$ is an internal torque, generated by the first actuator of the robotic arm; ${\Xi }_{123}$ is a three-dimensional vector with the components

$\begin{array}{l}{\Xi }_{123}(p)={\displaystyle \sum _{r=4}^5\left[{\displaystyle {\displaystyle \sum _{k=1}^r}{a}_{kp}^{(r)}}\frac{d^2{q}_k}{d{t}^2} + {\displaystyle {\displaystyle \sum _{k=1}^r}{\displaystyle \sum _{m=1}^r\frac{\partial a_{kp}^{(r)}}{\partial q_m}}\frac{d{q}_k}{dt}}\frac{d{q}_m}{dt}+ \left.2{\displaystyle \sum _{k=1}^r{\displaystyle \sum _{m=1}^r\frac{\partial a_{km}^{(r)}}{\partial q_p}}\frac{d{q}_k}{dt}}\frac{d{q}_m}{dt}\right]+\right.}\\ {\displaystyle {\displaystyle \sum _{k=1}^3}{a}_{kp}^{(6)}}{\displaystyle \frac{d^2{q}_k}{d{t}^2}} +2{\displaystyle {\displaystyle \sum _{k=1}^3}{\displaystyle \sum _{m=1}^3\frac{\partial a_{km}^{(6)}}{\partial q_p}}\frac{d{q}_k}{dt}}{\displaystyle \frac{d{q}_m}{dt}}+2\cdot a_{pp}^{(6)}{\displaystyle \frac{d^2{q}_6}{d{t}^2}}+2{\displaystyle {\displaystyle \sum _{k=1}^3}\frac{\partial a_{k6}^{(6)}}{\partial q_p}}{\displaystyle \frac{d{q}_k}{dt}}{\displaystyle \frac{d{q}_6}{dt}}+\\ {\displaystyle {\displaystyle \sum _{m=1}^3}{\displaystyle \sum _{k=1}^3\frac{\partial a_{km}^{(6)}}{\partial q_p}}}{\displaystyle \frac{d{q}_m}{dt}}{\displaystyle \frac{d{q}_k}{dt}}+4{\displaystyle {\displaystyle \sum _{k=1}^3}\frac{\partial a_{k6}^{(6)}}{\partial q_p}\frac{d{q}_k}{dt}\frac{d{q}_6}{dt}} + 2\cdot {\displaystyle \frac{\partial a_{66}^{(6)}}{\partial q_p}}{\left({\displaystyle \frac{d{q}_6}{dt}}\right)}^2+{\displaystyle {\displaystyle \sum _{k=4,5,6}}{m}_k\overline{g} }{\displaystyle \frac{\partial T_k}{\partial q_p}}{}^k\overline{r}_k\end{array}$ (35)

$p=1,2,3.$

Equation (33) can be written in a more compact and transparent form [22]. Note that the time derivative of the angular velocity vector $\omega$ in (21) is

${\displaystyle \frac{d\omega }{dt}}=S\left(q_0\right){\displaystyle \frac{d^2{q}_0}{d{t}^2}}+{\displaystyle \frac{dS\left(q_0\right)}{dt}}{\displaystyle \frac{d{q}_0}{dt}}$ (36)

Then the equation (33) can be written as

$I_0{\displaystyle \frac{d\omega }{dt}}+S^{-T}\left(q_0\right)\left({\displaystyle \frac{d{S}^T\left(q_0\right)}{dt}}{I}_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\right. -\left.{\displaystyle \frac{\partial }{\partial q_0}}\left[{\displaystyle \frac{d{q}_0^T}{dt}}{S}^T\left(q_0\right)I_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\right]\right) +$ (37)

$S^{-T}\left(q_0\right){\Xi }_{123}(q_0,q_{RA})=\overrightarrow{M}+ \left[\begin{array}{l}0\\ M_{Ai}\\ M_{El}\end{array}\right]-S^{-T}\left(q_0\right)\left[\begin{array}{l}{M}_4\\ M_5\\ 0\end{array}\right],$where $q_{RA}={\left[q_4, q_5\right]}^T$, $M_{Ai}$ and $M_{El}$ denote the moments generated by ailerons and elevator tilts, the superscript $-T$ denotes the combination of matrix transposition and inversion, and the transposed inverse of $S\left(q_0\right)$ is

$S^{-T}(q_0)=\left[\begin{array}{l} -{\displaystyle \frac{\sin (q_3)}{\cos (q_2)}} \cos (q_3) \sin (q_3)\tan (q_2)\\ 0 0 1\\ {\displaystyle \frac{ \cos (q_3)}{\cos (q_2) }} \sin (q_3) -\cos (q_3)\tan (q_2)\end{array}\right]$ (38)

Comparing this expression with the Newton-Euler’s equations of rotational motion of a rigid body in the body-fixed frame [22]

$I_0{\displaystyle \frac{d\omega }{dt}}+\omega \times I_0\omega =\tau$ (39)

implies

$S^{-T}\left(q_0\right)\left({\displaystyle \frac{d{S}^T\left(q_0\right)}{dt}}{I}_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\right. - \left.{\displaystyle \frac{\partial }{\partial q_0}}\left[{\displaystyle \frac{d{q}_0^T}{dt}}{S}^T\left(q_0\right)I_{0}S\left(q_0\right){\displaystyle \frac{d{q}_0}{dt}}\right]\right) =\omega \times I_0\omega ,$ (40)

and the equation (37) can be written in the following compact form:

$I_0{\displaystyle \frac{d\omega }{dt}}+\omega \times I_0\omega + S^{-T}\left(q_0\right) {\Xi }_{123}(q)=\overrightarrow{M}+ \left[\begin{array}{l}0\\ M_{Ai}\\ M_{El}\end{array}\right]-S^{-T}\left(q_0\right)\left[\begin{array}{l}{M}_4\\ M_5\\ 0\end{array}\right].$ (41)

This equation, in which the last term on the left-hand side describes dynamic cross-coupling of the fuselage and the robotic arm, is more convenient for analysis and design of the attitude control system in the takeoff and landing modes.

The dynamics equations for the generalized coordinates $q_4,$

$q_5,$ and $q_6$ are

$\begin{array}{l}{\displaystyle {\displaystyle \sum _{k=1}^4}{a}_{k4}^{(4)}\frac{d^2{q}_k}{d{t}^2}}+ 2{\displaystyle {\displaystyle \sum _{k=1}^4}{\displaystyle \sum _{m=1}^4\frac{\partial a_{k4}^{(4)}}{\partial q_m}\frac{d{q}_k}{dt}\frac{d{q}_m}{dt}}}+ {\displaystyle {\displaystyle \sum _{k=1}^5}{a}_{k4}^{(5)}\frac{d^2{q}_k}{d{t}^2}}+\\ 2{\displaystyle {\displaystyle \sum _{k=1}^5}{\displaystyle \sum _{m=1}^5\frac{\partial a_{k4}^{(5)}}{\partial q_m}\frac{d{q}_k}{dt}}}{\displaystyle \frac{d{q}_m}{dt}}+ {\displaystyle \frac{1}{2}}{\displaystyle {\displaystyle \sum _{m=1}^4}{\displaystyle \sum _{k=1}^4\frac{\partial a_{km}^{(4)}}{\partial q_4}\frac{d{q}_m}{dt}\frac{d{q}_k}{dt}+ }}\\ {\displaystyle \frac{1}{2}}{\displaystyle {\displaystyle \sum _{m=1}^4}{\displaystyle \sum _{k=1}^4\frac{\partial a_{k4}^{(4)}}{\partial q_m}\frac{d{q}_m}{dt}\frac{d{q}_k}{dt}+ }}{\displaystyle \frac{1}{2}}{\displaystyle {\displaystyle \sum _{m=1}^5}{\displaystyle \sum _{k=1}^5\frac{\partial a_{km}^{(5)}}{\partial q_4}\frac{d{q}_m}{dt}\frac{d{q}_k}{dt}+ }} \\ {\displaystyle \frac{1}{2}}{\displaystyle \sum _{m=1}^5{\displaystyle \sum _{k=1}^5\frac{\partial a_{k4}^{(5)}}{\partial q_m}\frac{d{q}_m}{dt}\frac{d{q}_k}{dt}+ }}{m}_4\overline{g}{\displaystyle \frac{\partial T_4 }{\partial q_4}}{}^4\overline{r}_4+m_5\overline{g}{\displaystyle \frac{\partial T_5 }{\partial q_4}}{}^5\overline{r}_5=M_4\end{array}$ (42)

$\begin{array}{l}{\displaystyle {\displaystyle \sum _{k=1}^5}{a}_{k5}^{(5)}}{\displaystyle \frac{d^2{q}_k}{d{t}^2}} + {\displaystyle {\displaystyle \sum _{k=1}^5}{\displaystyle \sum _{m=1}^5\frac{\partial a_{k5}^{(5)}}{\partial q_m}}\frac{d{q}_k}{dt}}{\displaystyle \frac{d{q}_m}{dt}}+ \\ 2{\displaystyle \sum _{k=1}^5{\displaystyle \sum _{m=1}^5\frac{\partial a_{5k}^{(5)}}{\partial q_m}}\frac{d{q}_k}{dt}}{\displaystyle \frac{d{q}_m}{dt}}+ m_5\overline{g}{\displaystyle \frac{\partial T_5 }{\partial q_5}}{}^5\overline{r}_5=M_5,\end{array}$ (43)

where $M_5$ is internal torque produced by the second actuator of the robotic arm, and

$\begin{array}{l}{\displaystyle {\displaystyle \sum _{k=1}^3}{a}_{k6}^{(6)}}{\displaystyle \frac{d^2{q}_k}{d{t}^2}} + 2{\displaystyle {\displaystyle \sum _{k=1}^3}{\displaystyle \sum _{m=1}^3\frac{\partial a_{k6}^{(6)}}{\partial q_m}}\frac{d{q}_k}{dt}}{\displaystyle \frac{d{q}_m}{dt}}+ 2\cdot a_{66}^{(6)}{\displaystyle \frac{d^2{q}_6}{d{t}^2}}+ 2{\displaystyle {\displaystyle \sum _{k=1}^3}\frac{\partial a_{k6}^{(6)}}{\partial q_6}}{\displaystyle \frac{d{q}_k}{dt}}{\displaystyle \frac{d{q}_6}{dt}}+ \\ {\displaystyle {\displaystyle \sum _{m=1}^3}{\displaystyle \sum _{k=1}^3\frac{\partial a_{km}^{(6)}}{\partial q_6}}}{\displaystyle \frac{d{q}_m}{dt}}{\displaystyle \frac{d{q}_k}{dt}}+4{\displaystyle {\displaystyle \sum _{m=1}^3}\frac{\partial a_{66}^{(6)}}{\partial q_m}\frac{d{q}_m}{dt}\frac{d{q}_6}{dt}}+ 2\cdot {\displaystyle \frac{\partial a_{66}^{(6)}}{\partial q_6}}{\left({\displaystyle \frac{d{q}_6}{dt}}\right)}^2+ \\ m_6\overline{g}{\displaystyle \frac{\partial T_6 }{\partial q_6}}{}^6\overline{r}_6= \cos ({45}^{\circ })\left[(F_1+F_3) - (F_2+F_4)\right] \end{array}$ (44)

Equations (33), (42)-(44) can be written in the matrix form

$M(q){\displaystyle \frac{d^{2}q}{d{t}^2}}+C\left(q,{\displaystyle \frac{dq}{dt}}\right){\displaystyle \frac{dq}{dt}}+G(q)=\tau$ (45)

The entries $M_{ij}$ of the symmetric inertia matrix $M(q)$ in (45) are

M_ij = (S^TI₀S)_ij+ a_ij⁽⁴⁾ + a_ij⁽⁵⁾ + a_ij⁽⁶⁾, i,j ≤ 3,

(46)

$M_{i4}=a_{i4}^{(4)}+a_{i4}^{(5)}, M_{i5}=a_{i5}^{(5)}, M_{i6}=a_{i6}^{(6)}, i=1,2,3,$

$M_{44}=a_{44}^{(4)}+a_{44}^{(5)}, M_{45}=a_{45}^{(5)}, M_{46}=0,$

$M_{55}=a_{55}^{(5)}, M_{56}= 0, M_{66}=a_{66}^{(6)}.$

The entries of the Coriolis/centrifugal terms (Christoffel symbols) of the matrix $C\left(q,dq/dt\right)$ in (45) are determined in terms of the coefficients $M_{ij}$(46) as

$\begin{array}{l}{C}_{ij}\left(q,{\displaystyle \frac{dq}{dt}}\right)={\displaystyle \frac{1}{2}}{\displaystyle \sum _{k=1}^6\left(\frac{\partial M_{ij}}{\partial q_k} +\frac{\partial M_{ik}}{\partial q_j}+\frac{\partial M_{jk}}{\partial q_i} \right)}{\displaystyle \frac{d{q}_k}{dt}}\\ i,j=1,2,3,4,5,6.\end{array}$ (47)

The components $G_i(q)$ of the gravity vector $G(q)$ in (45) are

$\begin{array}{l}{G}_i={\displaystyle \sum _{k=4,5,6}{m}_k\overline{g} }{\displaystyle \frac{\partial T_k(q)}{\partial q_i}}{}^k\overline{r}_k, \\ i=1,2,3,\end{array}$ (48)

$G_4=m_4\overline{g}{\displaystyle \frac{\partial T_4 }{\partial q_4}}{}^4\overline{r}_4+m_5\overline{g}{\displaystyle \frac{\partial T_5 }{\partial q_4}}{}^6\overline{r}, G_5=m_5\overline{g}{\displaystyle \frac{\partial T_5 }{\partial q_5}}{}^5\overline{r}_5, G_6=m_6\overline{g}{\displaystyle \frac{\partial T_6 }{\partial q_6}}{}^6\overline{r}_6.$

Finally, the components of the vector $\tau$ in (45) are

$\left[\begin{array}{l}{\tau }_1\\ {\tau }_2\\ {\tau }_3\end{array}\right]=S^T(q_o)\overline{M} +S^T(q_o) \left[\begin{array}{l}0\\ M_{Ai}\\ M_{El}\end{array}\right]-\left[\begin{array}{l}{M}_4\\ M_5\\ 0\end{array}\right],$ (49)

${\tau }_4= M_4 , {\tau }_5=M_5, {\tau }_6= \cos ({45}^{\circ })\left[(F_1+F_3) - (F_2+F_4)\right].$ (50)

The derived differential equations of the rotational motion (33)-(50) were used in the development of the comprehensive (expanded) dynamic model of the pitch-decoupled UAV in the Simulink package environment.

4. Translational Dynamics of the UAV

The main forces acting on the pitch-decoupled UAV during the horizontal (longitudinal) flight are schematically shown in Figure 5, where $F_1, F_2, F_3, F_4, F_5$are the thrusts generated be the five rotors, $F_{\Sigma }$ is the total thrust of the quad part’s four rotors, $\mathrm{Lift}$and $\mathrm{Drag}$are aerodynamic forces.

If all forces are represented with respect to the inertial reference frame, then, according to Newton’s second law, the simplified differential equations describing the translational motion of the UAV are given by the equation

$m_{\Sigma }{\displaystyle \frac{d^2\eta }{d{t}^2}}=-m_{\Sigma }g{Z}_I+F_A+R_0(q_0)\left[\begin{array}{l}\sin (q_6)\\ 0\\ \cos (q_6)\end{array}\right]F_{\Sigma },$ (51)

where vector $\eta ={\left[{\eta }_x,{\eta }_y,{\eta }_z\right]}^T$ represents the linear position of the UAV in the inertial reference frame, $M_{\Sigma }$ is the total mass of the UAV, $F_{\Sigma }$ is the total thrust of the quad part rotors (15), $R_0(q_0)$ is the orthogonal rotation matrix (2), $g$ is the gravitational constant,

$F_A$ denotes the aerodynamic forces acting on the UAV in the wind frame. For simplicity, sideslip forces are neglected, and the aerodynamic forces are assumed to be transformed to the inertial reference frame.

To calculate the aerodynamic characteristics of the pitch-decoupled UAV, it is necessary to determine its lift force $F_L$, drag force $F_D$, and the aerodynamic pitching moment $M_p$about the pitch axis, which are given by the following expressions [23,24]:

$F_L=\left({\displaystyle \frac{1}{2}}\right) \rho V^{2}S{C}_L\left(\alpha \right), F_D=\left({\displaystyle \frac{1}{2}}\right) \rho V^{2}S{C}_D\left(\alpha \right), M_p=\left({\displaystyle \frac{1}{2}}\right)cS{C}_m\left(\alpha \right), \left(52\right)$

where $\alpha$ is an angle of attack, and the lift $C_L,$ drag ${ C}_D,$ and pitching moment $C_M$ coefficients are equal to

$C_L={\displaystyle \frac{2L}{\rho V^{2}S}}, { C}_D={\displaystyle \frac{2D}{\rho V^{2}S}}, C_M={\displaystyle \frac{2M}{\rho V^{2}cS}}. \left(53\right)$

In the expressions (52), $\rho$ is the air density, $S$ is the wing area, $V$is the flight velocity relative to the air, c is the mean aerodynamic chord [25]. Besides, the coefficients $L,$

$D$, and $M$ in (53) are actual aerodynamic lift and drag forces, and the pitching moment.

The study of the UAV’s aerodynamic characteristics, including the lift and drag coefficients, as well as the stability properties, was performed using the “XFLR5” aerodynamics simulator [26,27,28], considering external forces applied along the longitudinal and lateral axes of the UAV. Figure 6 illustrates the lift, drag and pitching-moment coefficients obtained for different aerodynamic airflow simulation runs. Figure 7 presents the results of analyzing the aerodynamic stability under external forces. The graphs in Figure 7 indicate that the UAV is aerodynamically stable in both longitudinal and lateral directions. The results of aerodynamic simulations were used in the design of the neural controller described in Section 6.

5. Control System of the Pitch-Decoupled UAV

In this Section, the control of the pitched-decoupled UAV in different flight modes is considered in detail. Unlike the transient modes common to tilt-rotors UAVs, in which the UAV switches between the vertical and horizontal flights by tilting the rotors from $0^{\circ }$ to ${90}^{\circ }$ or vice versa, special attention in Subsection 5.2 is devoted to a flight regime, in which the pitch angle $\gamma$ of the quad part is constant but not equal to the limiting (marginal) values $0^{\circ }$ or ${90}^{\circ }$. Geometrically, this corresponds to a “sloped” (climb or descent) flight of the pitch-decoupled UAV. Schematically, the sloped flight mode is illustrated in Figure 8. In this mode, the pitch angle of the plane is zero, and the given constant (but not $0^{\circ }$ or ${90}^{\circ }$) tilt of the quad part is maintained by a special feedback loop, the error signal of which is calculated as a difference of the given tilt and the signal of the potentiometer measuring the relative tilt of the quad part.

For any given “intermediate” tilt of the quad part, there are three different situations. If the vertical component $F_z$ of the total thrust $F_{\Sigma }$ is equal to the gravitational constant $g=9.81$ (Figure 8a), the UAV is in the horizontal flight with tilted rotors. If the vertical component is larger ($F_z>g$), the UAV is in the climb flight (Figure 8b). Finally, if $F_z<g,$ the UAV is in the descent flight. Note that in all these situations, the zero pitch angle of the UAV is provided by the control system of the tail rotor.

5.1. Control System in Takeoff, Landing, and Hovering Modes

We specifically discuss the flight of the pitch-decoupled UAV in the standard takeoff, landing, and hover modes because in these modes the pitch-decoupled UAV is controlled like a conventional quadcopter. Moreover, the robotic arm is usually used when the UAV is in hover mode. In the present configuration, in these modes, the quad pitch angle $\gamma$ (also referred to as $Q_{pitch}$ angle) is zero and the quad part is rigidly fixed to the UAV’s fuselage.

To proceed, we shall make some assumptions and simplifications needed for the design of the corresponding controllers in the mentioned modes. First, we assume that the mass and inertia matrix $I_6$ of the quad part are accounted for in the mass $m_0$ and in the inertia matrix $I_0$ of the fuselage, where the matrix $I_0$ is assumed to be diagonal. This simplifies the differential equations (32), (42), (43), in which all the terms containing index 6 must be eliminated. In addition, the equation (44) must be discarded.

Generally, in controlling the flight of quadcopters, the altitude $z$ and rotation angles (roll $\varphi$, pitch $\theta$, and yaw $\psi$) are usually chosen as four control variables [29].

Below, we shall assume that the angles and angular velocities of the UAV are so small that nonlinear terms in the dynamic equations of rotational motions can be neglected, the cosines of all angles are approximately equal to one, and sines of the angles are zero. At this stage, we assume that there are no dynamic cross-couplings between the fuselage of the UAV and the robotic arm.

On these assumptions, the dynamics equations of multirotor UAVs have the following linearized form [3,29]:

${\displaystyle \frac{d^2{\eta }_z}{d{t}^2}}={\displaystyle \frac{1}{m_{\Sigma }}}{F}_{\Sigma }-g$ , (54)

$I_0{\displaystyle \frac{d^2{q}_0}{d{t}^2}}=\tau$, $I_4{\displaystyle \frac{d^2{q}_4}{d{t}^2}}=M_4$, $I_5{\displaystyle \frac{d^2{q}_5}{d{t}^2}}=M_5,$ (55)

where $m_{\Sigma }$ is the mass of the UAV; g – the gravitational constant; $I_0$ is the diagonal tensor of inertia with the components $I_x,$ $I_y,$ $I_z$ on the principal diagonal; vector $\tau =$

${[{\tau }_x,{\tau }_y,{\tau }_z]}^T$ combines the principal non-conservative forces and moments applied to the UAV airframe by the aerodynamics of the four quad part rotors (assuming no external disturbances); $I_4$ and $I_5$ are inertia moments of the robotic arm links.

Denoting by $\overline{F}$ the four-dimensional vector of thrusts $F_i$($\overline{F}={[F_1,F_2,F_3,F_4]}^T$), the mapping of $\overline{F}$ to the vector ${\left[F_{\Sigma },\tau \right]}^T$ can be written in matrix form

$\left[\begin{array}{l}{F}_{\Sigma }\\ \tau \end{array}\right]=D_M\overline{F}.$ (56)

In (56), the $4\times 4$ full-rank numerical matrix $D_M$ for the quad part rotors allocation in Figure 4 is equal to

$D_M=\left[\begin{array}{l} 1 1 1 1\\ {\displaystyle \frac{\sqrt{2}}{2}}L {\displaystyle \frac{\sqrt{2}}{2}}L -{\displaystyle \frac{\sqrt{2}}{2}}L -{\displaystyle \frac{\sqrt{2}}{2}}L\\ -{\displaystyle \frac{\sqrt{2}}{2}}L {\displaystyle \frac{\sqrt{2}}{2}}L {\displaystyle \frac{\sqrt{2}}{2}}L -{\displaystyle \frac{\sqrt{2}}{2}}L\\ -k_{\psi } k_{\psi } -k_{\psi } k_{\psi }\end{array}\right],$ (57)

where $L$ is the distance of the rotors from the point $Q_6$ in Figure 4. Structurally, the matrix $D_M$ (59) describes the kinematic cross-coupling between the separate channels of the UAVs in hover mode.

The matrix block diagram of the UAV’s control system in hover mode and of the two-dimensional control system of the robotic arm is shown in Figure 9.

In Figure 9: $I$ is an $3\times 3$ identity matrix; $I_{2\times 2}$ is an $2\times 2$ identity matrix $q_{RA}$ is a two-dimensional vector with components $q_4$ and $q_5$; $P$ is a $2\times 3$ projection matrix

$P=\left[\begin{array}{l}1 0 0\\ 0 1 0\end{array}\right]$; (58)

$w_M(s)$ is the transfer function of brushless DC motors (rotors) of the quad part; $diag\{w_i^R(s)\}$ is a $4\times 4$ diagonal matrix of the transfer functions $w_i^R(s)$;

$K_{\mathrm{Re}\mathrm{g}}(s) =K_{D}diag\{w_i^R(s)\}$ (59)

is the transfer matrix of the matrix regulator with the numerical matrix $K_D$ equal to

$K_D=D_M^{-1}=\left[\begin{array}{l} {\displaystyle \frac{1}{4}} {\displaystyle \frac{\sqrt{2}}{4L}} -{\displaystyle \frac{\sqrt{2}}{4L}} - {\displaystyle \frac{1}{4k_{\psi }}}\\ {\displaystyle \frac{1}{4}} {\displaystyle \frac{\sqrt{2}}{4L}} {\displaystyle \frac{\sqrt{2}}{4L}} {\displaystyle \frac{1}{4k_{\psi }}}\\ {\displaystyle \frac{1}{4}} -{\displaystyle \frac{\sqrt{2}}{4L}} {\displaystyle \frac{\sqrt{2}}{4L}} -{\displaystyle \frac{1}{4k_{\psi }}}\\ {\displaystyle \frac{1}{4}} -{\displaystyle \frac{\sqrt{2}}{4L}} -{\displaystyle \frac{\sqrt{2}}{4L}} {\displaystyle \frac{1}{4k_{\psi }}}\end{array}\right].$ (60)

The superscripts $\mathrm{Ref}$ in Figure 9 denote the reference (input) signals.

The transfer function $w_{RA}(s)$ in Figure 9 represents the identical transfer functions of separate channels of the robotic arm and has the form

$w_{RA}(s)=w_{RC}(s)w_{RM}(s),$ (61)

where $w_{RM}(s)$ is the transfer function of the link actuators (DC motors), and $w_{RC}(s)$ is the transfer function of the controllers.

Assuming for simplicity that the transfer functions $w_i^R$ ($i=1,2,3,4$) in (59) are identical, that is $w_i^R(s)=w^R(s)$, the transfer matrix $W_O(s)$ of the open-loop UAV’s control system in hover mode takes the form

$W_O(s)={\displaystyle \frac{w_M(s)w^R(s)}{s^2}}{M}_{\Sigma }^{-1}{D}_M{K}_D,$ (62)

where the matrix $M_{\Sigma }$ has the following block-diagonal form:

$M_{\Sigma }=\left[\begin{array}{l}{m}_{\Sigma } 0_{1\times 3}\\ 0_{3\times 1} I_0\end{array}\right].$ (63)

The dynamics of the DC motors in (61) and (62) are approximated by the first-order transfer functions

$w_M(s)={\displaystyle \frac{K_M}{T_{M}s+1}}$, $w_{RM}(s)={\displaystyle \frac{K_{RM}}{T_{RM}s+1}},$ (64)

Note that if the matrix $K_D$ in the regulator $K_{\mathrm{Re}\mathrm{g}}(s)$ (59) is chosen as in (60), then the transfer matrix of the open-loop system $W_O(s)$ reduces to the diagonal form

$W_O(s)={\displaystyle \frac{w_M(s)w^R(s)}{s^2}}{M}_{\Sigma }^{-1}= \left[\begin{array}{l}{\displaystyle \frac{1}{m_{\Sigma }}} 0_{1\times 3}\\ 0_{3\times 1} I_0^{-1}\end{array}\right]{\displaystyle \frac{w_M(s)w^R(s)}{s^2}}$ (65)

This equation implies that the attitude control channels of the UAV in hover mode are decoupled (since matrix $I_0$ is assumed diagonal) and the design of controllers in these channels can be performed as for common single-input single-output (SISO) control systems [30]. The PD controllers with the following transfer function:

$w^R(s)=0.237 + {\displaystyle \frac{1.32s}{0.0026s+1}}$ (66)

were used as controllers $w^R(s)$ in (65). This transfer function was obtained by applying the graphical user interface (GUI) pidTuner in MATLAB [30]. Analogously, the PD controllers of the form:

$w_{RC}(s)=87 + {\displaystyle \frac{158s}{0.0011s+1}}$ (67)

were used as the controllers of the robotic arm.

The step responses of the attitude control system of the pitched-decoupled UAV with the robotic arm in hover mode are shown in Figure 10. The control of the UAV altitude is considered in the next section.

5.2. PID Control of the Pitch-Decoupled UAV in Transition and Sloped Modes

In this section, we concentrate mainly on the dynamics of the pitched-decoupled UAV in transition mode using standard PID controllers. A control architecture is proposed, in which, to improve the controllability of the UAV, additional velocity-control loops for the three translational axes are introduced. In this system, the control of the UAV in the transition mode essentially reduces to the velocity control of the UAV’s fuselage. The selection of parameters of PID controllers was performed by automatic design tools available in Control System Toolbox in MATLAB [30]. Dynamics analysis of the control system was performed in the Simulink environment, based on the developed nonlinear mathematical model of the UAV, in which, also, the aerodynamic forces (lift and drag) were accounted for. The matrix block diagram of control system of the pitched-decoupled UAV in the transition mode is shown in Figure 11. It should be noted that, virtually, the robotic arm is not (or, practically, cannot be) used in the transition mode. For that reason, the control system of the robotic arm is not shown in the block diagram in Figure 11. Moreover, the aerodynamic (lift and drag) forces, the influence of which is of a complex nonlinear nature, are also not presented in the block diagram. In Figure 11: $v^{\mathrm{Ref}}$ and $v$ are the inputs and outputs of the velocity control system; $w^v(s)$ is the diagonal matrix of the following PID controllers:

$w_{yaw}(s)=3+{\displaystyle \frac{1}{s}}+0.03s$, $w_{pitch}(s)=1+{\displaystyle \frac{0.05}{s}}+0.1s$, $w_{roll}(s)=3+{\displaystyle \frac{0.5}{s}}+0.03s$; (68)

$\Gamma$ is the matrix of the form

$\Gamma ={\displaystyle \frac{1}{m_{\Sigma }}}{R}_0(q_0)\left[\begin{array}{l}\sin (q_6)\\ 0\\ \cos (q_6)\end{array}\right]$; (69)

and

$w^Q(s)=2+{\displaystyle \frac{0.5}{s}}+0.02s$ (70)

is the PID controller of the passive pitch angle $\gamma =q_6$; $I_Q$ is the inertia moment of the quad part about the rotation axis.

Let us consider the transition phase of the pitch-decoupled UAV with the PD controllers $w^Q\left(s\right)$ (70) for two scenarios. Figure 12 shows the results of 3D simulation in case when the UAV switches from hover mode to longitudinal flight by tilting the quad part by $\gamma ={90}^{\circ }$. The dynamic model of the UAV was developed based on nonlinear equations ()-() and the results of aerodynamic simulation of the UAV presented in Section 4. In this scenario, the UAV starts the flight at $t=0$ and gains the prescribed altitude of 85 m. At $t=50 s$, the UAV switches to longitudinal flight. As can be seen from Figure 12, during the transition phase, the UAV’s altitude drops to 16 m (at $t=54 s$). Afterwards, due to aerodynamic characteristics of the UAV, it starts gaining velocity and altitude. The graphs in Figure 13 show the $Q_{pitch}$ and altitude response graphs.

In the second scenario, the UAV is in hover mode at the prescribed altitude 5 m, and at $t=$ 8 s, the passive pitch angle changes from $\gamma =0^{\circ }$ to $\gamma ={30}^{\circ }$. Figure 14 shows the changes in the $Q_{pitch}$ angle and the flight altitude. Figure 15 presents the horizontal velocity, as well as the aerodynamic lift and drag forces of the UAV. Figure 16 shows the total thrust generated by the rotors, which decreases after completing the transition phase due to the increase in the aerodynamic lift force.

The parameters of the PID controllers (68) in the velocity control system are determined based on the linearized model of the UAV and for the fixed value of the angle $\gamma$. In practical flights, the required angle $\gamma$ is calculated in advance and introduced manually by the operator. The required parameters of the velocity PID controllers may change substantially for different angles $\gamma$. The most common strategies to overcome that difficulty are based on using adaptive, gain-scheduling, and intelligent control techniques, or their combinations [31,32,33,34], as well as other strategies [29,30].

6. Actor-Critic Method and Neural Controller

One of the most effective approaches for designing intelligent controllers is reinforcement learning because it allows one to develop an optimal control policy through interaction with the environment, particularly for systems that are highly nonlinear, uncertain, or difficult to model accurately [35,36,37]. Among reinforcement learning algorithms, the actor-critic family has become one of the most widely used in robotics, UAVs, autonomous vehicles, and process control because it combines the strengths of policy-based and value-based methods [38,39,40].

This section describes a neural controller designed and trained using the reinforcement-learning actor–critic method. The objective is to develop a controller capable of ensuring a smooth transition of a pitch-decoupled UAV from quadcopter mode to airplane mode and back in various working conditions, while minimizing tracking errors.

Briefly, the training procedure can be described as follows. At each time step $t$, the main control agent selects an action, $A_t,$ receives a reward $R_t$ from the environment, and transitions from state $S_t$ to state $S_{t+1}.$ The ultimate objective of reinforcement learning is to maximize the expected discounted return over time

$G_t=R_{t+1}+{\gamma }^2{R}_{t+2 }+\dots {\displaystyle {\displaystyle \sum }_{k=0}^{\infty }}{\gamma }^k{R}_{t+k+1}\to \max ,$ (71)

where $\gamma$ is the discount factor.

If an agent follows policy $\pi$ at time $t$, then $\pi (a,s)$ denotes the probability of selecting action $A_t=a$ while the system is in state $S_t=s$. Maximizing the expected return yields the optimal policy for the agent.

The action-value function associated with policy $\pi$ is denoted by $q_{\pi }$ and is given by the following expression:

$q_{\pi }\left(s,a\right)={\mathbb{E}}_{\pi }\left[G_T \right| S_t=s, A_t=a ]={\mathbb{E}}_{\pi }\left[{\displaystyle \sum _{k=0}^{\infty }}{\gamma }^k{R}_{t+k+1} | S_t=s, A_t=a\right] (72)$

The central concept of reinforcement learning is represented by the Bellman equation,

$q_{*}\left(s,a\right)=\mathbb{E}\left[R_{t+1}+\gamma *\max \left\{q_{*}\left(s^{\text{'}},a^{\text{'}}\right)\right\}\right]. \left(73\right)$

This equation defines the optimal action-value function that the agent must learn iteratively. In this study, the Deep Deterministic Policy Gradient (DDPG) algorithm was employed [41,42]. DDPG uses an actor–critic architecture in which the critic serves as a value-function approximator, while the actor is trained using feedback from the critic by selecting actions that maximize the estimated critic value. The actor–critic structure used for the pitch-decoupled UAV training is illustrated in Figure 17.

The actor network is denoted by $\mu \left(s\right|{\theta }^{\mu })$ and parameterized by ${\theta }^{\mu }$, whereas the critic network is parameterized by ${\theta }^q$, and its loss function has the following form:

$L\left({\theta }^q\right)=\mathbb{E}\left[{\left(q\left(s_t,a_t|{\theta }^q\right)-y_t\right)}^2\right], \left(74\right)$

where $q\left(s_t,a_t|{\theta }^q\right)$ is the output of the critic network, and the target value is defined as:

$y_t=(R\left(s_t,a_t\right)+\gamma q\left(s_{t+1},\mu \left(s_{t+1}\right)\right|{\theta }^q)). (75)$

The overall policy gradient is given by the following expression:

${\nabla }_{{\theta }^{\mu }}={\mathbb{E}}_{s_t}\left[{\nabla }_{a}q\left(s,a|{\theta }^q\right){|}_{s=s_t,a=\mu \left(s_t\right)}{\nabla }_{{\theta }^{\mu }}\mu \left(s|{\theta }^{\mu }\right){|}_{s=s_t}\right] \left(76\right)$

Here, ${\nabla }_a q\left(s,a \right|{\theta }^q)$ and ${\nabla }_{{\theta }^{\mu }} \mu \left(s|{\theta }^{\mu }\right)$ are the gradients of the critic and actor neural networks, respectively. Consequently, the actor is trained based on the action values returned by the critic network. It is important to note that the target value $y_t$ employs the same actor-critic networks with the same parameters ${\theta }^{\mu }\mathrm{a}\mathrm{n}\mathrm{d}{\theta }^q,$ which may cause instability during training. To address this issue, target copies of the actor and critic networks, $q^{\mathrm{\text{'}}}\left(s,a|{\theta }^{q^{\mathrm{\text{'}}}}\right)$ and ${\mu }^{\mathrm{\text{'}}}\left(s|{\theta }^{{\mu }^{\mathrm{\text{'}}}}\right)$, are introduced. Their parameters ${\theta }^{q^{\mathrm{\text{'}}}}$ and ${\theta }^{{\mu }^{\mathrm{\text{'}}}}$ are updated more slowly than those of the primary networks using a soft update factor $w\ll 1$:

${\theta }^{q^{\mathrm{\text{'}}}}=w{\theta }^q+\left(1-w\right){\theta }^{q^{\mathrm{\text{'}}}} \left(77\right)$

${\theta }^{{\mu }^{\mathrm{\text{'}}}}=w{\theta }^{\mu }+\left(1-w\right){\theta }^{{\mu }^{\mathrm{\text{'}}}} \left(78\right)$

As described previously, the neural controller replaces the conventional velocity PID controllers to achieve smooth transition behavior and accurate velocity tracking. The resulting velocity control system architecture, incorporating the neural controller, is shown in Figure 18. Here, the neural controller continuously governs the passive pitch angle $\gamma =q_6$ depending on the current state of the control system, required flight regime and flying conditions.

The state vector is defined as follows:

$S^{10\times 1}= {\left[\begin{array}{ccccccccc}{v}_x& v_z& v_x^r& v_z^r& \dot{v_x}& \dot{v_z}& {\left(A_{t-1}^{2\times 1}\right)}^T& {\epsilon }_x& {\epsilon }_z\end{array} \right]}^T, \left(79\right)$

where

${\epsilon }_x=v_x^r-v_x,{ \epsilon }_z=v_z^r-v_z, \left(80\right)$are the velocity tracking errors.

The reward function is defined as follows:

$R=p_1{\displaystyle \frac{t}{T}}-p_2{\epsilon }_x^2-p_3{\epsilon }_z^2-p_4\sum _i{{A\left(i\right)}_{t-1 }}^2 , \left(81\right)$

where $p_i, i=1\dots 4$ are the weighting coefficients of the reward components, and the term $p_4\sum _i{{A\left(i\right)}_{t-1 }}^2$ represents the actuator effort penalty. The training episode is terminated if any of the following conditions is violated:

$\left\{\begin{array}{c}-45<{\epsilon }_x<45\\ -7<{\epsilon }_z<7\\ 0<v_x^r<25\\ -2<v_z^r<2.\end{array} \left(82\right)\right.$

The pitch-decoupled UAV model and the training procedure were implemented in the Simulink environment using the Reinforcement Learning Designer Toolbox for agent construction [43]. Figure 19 presents the overall training process, during which the agent progressively improves its control policy by maximizing the expected return over successive episodes.

To evaluate the behavior of the control system incorporating the neural controller, two transition-phase scenarios were investigated. In each scenario, the pitch-decoupled UAV takes off and reaches a specified altitude $Z$. Then, at $t=5 s$, the transition phase is initiated during which the UAV accelerates toward a prescribed horizontal velocity $v_x.$ Both scenarios were simulated in the Simulink environment using different values of the reference parameters $Z_1$, $Z_2$and $v_x^1$, $v_x^2$ :

Scenario 1: $Z_1= 5 m, v_x^1=25 m/s.$

Scenario 2: $Z_1= 10 m, v_x^1=10 m/s.$

Figure 20 and Figure 21 present the simulation results for Scenario 1, and Figure 22 and Figure 23 for Scenario 2, where $\gamma$ denotes the passive pitch angle and $Z$ is the altitude. In both cases, the transition phase proceeds smoothly, without overshoot or steady-state error, confirming the effectiveness of the proposed neural controller.

Figure 21 and Figure 23 show the horizontal and vertical velocity responses (blue lines) of the pitch-decoupled UAV together with their reference signals (black dot lines) for Scenarios 1 and 2, respectively. The results demonstrate that the neural controller ensures smooth transition and stable flight at the prescribed horizontal velocities while maintaining minimal tracking error. The transition phase begins when the reference horizontal velocity $v_x$ changes from zero to the prescribed constant value. The primary function of the neural controller is not to directly initiate the transition, but rather to track the reference velocities with minimal error by adjusting the $\gamma$ angle and rotors’ total thrust.

The required $\gamma$ ($Q_{pitch}$) angle, which is calculated by the neural controller, is shown in Figure 20 and Figure 22. In Scenario 2, the transition phase begins before the pitch-decoupled UAV reaches its reference altitude. Nevertheless, the neural controller successfully manages the transition without producing overshoot or steady-state errors.

The simplified, without the altitude and attitude control subsystems, block diagram of the pitch-decoupled UAV’s control system with the velocity neural controller is shown in Figure 24.

Here, $W_{\eta }(s)$ is a diagonal position controller with, usually, standard SISO PID controllers on the main diagonal. In the block diagram, there are two multivariable feedback loops, and one passively controlled (by difference of the paired thrusts) SISO loop, which governs the transitions between different flight modes during entire flight envelope. The altitude and attitude control subsystems, which operate during the hover and standard vertical takeoff and landing modes, as well as control system of the robotic arm, are discussed in Subsection 5.1.

7. Discussion

The article addresses an important challenge in pitch-decoupled VTOL UAV operation: achieving reliable and smooth transitions between hovering and fixed-wing (horizontal-flight) modes. Its principal contribution is the application of an actor–critic reinforcement-learning velocity controller to a pitch-decoupled UAV architecture that differs substantially from conventional tilt-rotor systems. In addition, the study considers a pitch-decoupled UAV equipped with a two-DOF robotic arm and derives a complete mathematical model of the system using modified Denavit–Hartenberg parameters. The proposed approach can accommodate UAVs equipped with robotic manipulators of various configurations and with an arbitrary number of degrees of freedom.

The present study focuses on investigating the main characteristics of the pitch-decoupled UAV control architecture and on designing a neural controller. Therefore, control of the pitch-decoupled UAV in conventional fixed-wing flight—for example, through the elevator, ailerons, or their combination—is not considered. In addition, the SISO tail-rotor control system is assumed to operate perfectly; that is, it is assumed to maintain the fuselage pitch angle at zero during flight. A comprehensive investigation of the pitch-decoupled UAV’s dynamics that accounts for these factors is beyond the scope of this paper.

Another distinctive feature of the study is that the UAV control architecture is formulated within the multivariable control-system framework. This formulation enables clear representation of the kinematic and dynamic cross-couplings among the subsystems and blocks that constitute the overall control system.

Compared with closely related studies on pitch-decoupled UAV control [10,11,12,13], the principal distinguishing features of the present work are the proposed DDPG-based transition controller and the inclusion of a two-DOF robotic arm in the multibody model.

Nevertheless, the proposed neural controller has several limitations, some of which are inherent to reinforcement-learning-based control. At this stage, robustness, stability, and safety have not been evaluated under variations in mass, robotic-arm configuration and payload, aerodynamic coefficients, wind disturbances, sensor noise, or partial actuator failures. These limitations, together with the need for experimental validation and practical deployment of the neural controller, define important directions for future research.

A further promising direction is the development of a hybrid control architecture for the pitch-decoupled UAV that integrates a learning-based controller with a classical model-based controller. Such architectures may provide an effective means of improving stability, robustness, and practical reliability of the UAV’s control system [33,34].

8. Conclusions

This study develops a modeling and control framework for a pitch-decoupled VTOL UAV with a two-DOF robotic arm. The platform’s passive coupling mechanism offers an attractive alternative to conventional tilt-rotor systems by using differential thrust rather than dedicated rotor-tilting actuators. The article combines kinematic and dynamic modeling, aerodynamic analysis, conventional PID-based control, and a DDPG actor–critic neural controller for transition flight. The simulation results demonstrate that the proposed neural controller provides better performance and tracking accuracy during transition modes than the conventional PID controller by continuously governing the passive pitch angle of the pitch-decoupled VTOL UAV.