Version 1
: Received: 22 October 2022 / Approved: 24 October 2022 / Online: 24 October 2022 (10:24:33 CEST)
How to cite:
Zahmatkesh, M.; Emami, S.A.; Banazadeh, A.; Castaldi, P. Attitude Control of Highly Maneuverable Aircraft Using an Improved Q-learning. Preprints2022, 2022100360. https://doi.org/10.20944/preprints202210.0360.v1
Zahmatkesh, M.; Emami, S.A.; Banazadeh, A.; Castaldi, P. Attitude Control of Highly Maneuverable Aircraft Using an Improved Q-learning. Preprints 2022, 2022100360. https://doi.org/10.20944/preprints202210.0360.v1
Zahmatkesh, M.; Emami, S.A.; Banazadeh, A.; Castaldi, P. Attitude Control of Highly Maneuverable Aircraft Using an Improved Q-learning. Preprints2022, 2022100360. https://doi.org/10.20944/preprints202210.0360.v1
APA Style
Zahmatkesh, M., Emami, S.A., Banazadeh, A., & Castaldi, P. (2022). Attitude Control of Highly Maneuverable Aircraft Using an Improved Q-learning. Preprints. https://doi.org/10.20944/preprints202210.0360.v1
Chicago/Turabian Style
Zahmatkesh, M., Afshin Banazadeh and Paolo Castaldi. 2022 "Attitude Control of Highly Maneuverable Aircraft Using an Improved Q-learning" Preprints. https://doi.org/10.20944/preprints202210.0360.v1
Abstract
Attitude control of a novel regional truss-braced wing aircraft with low stability characteristics is addressed in this paper using Reinforcement Learning (RL). In recent years, RL has been increasingly employed in challenging applications, particularly, autonomous flight control. However, a significant predicament confronting discrete RL algorithms is the dimension limitation of the state-action table and difficulties in defining the elements of the RL environment. To address these issues, in this paper, a detailed mathematical model of the mentioned aircraft is first developed to shape an RL environment. Subsequently, Q-learning, the most prevalent discrete RL algorithm will be implemented in both the Markov Decision Process (MDP), and Partially Observable Markov Decision Process (POMDP) frameworks to control the longitudinal mode of the air vehicle. In order to eliminate residual fluctuations that are a consequence of discrete action selection, and simultaneously track variable pitch angles, a Fuzzy Action Assignment (FAA) method is proposed to generate continuous control commands using the trained Q-table. Accordingly, it will be proved that by defining an accurate reward function, along with observing all crucial states (which is equivalent to satisfying the Markov Property), the performance of the introduced control system surpasses a well-tuned Proportional–Integral–Derivative (PID) controller.
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
