Intelligent Adaptive Control Using LADP and IADP Applied to F-16 Aircraft with Imperfect Measurements

What is it about?

Linear Approximate Dynamic Programming (LADP) and Incremental Approximate Dynamic Programming (IADP) are Reinforcement Learning methods that seek to contribute to the field of Adaptive Flight Control. This paper assesses their performance and convergence, as well as the impact of sensor noise on policy convergence, online system identification, performance and control surface deflection. After summarising their theory and derivation with full state (FS) and output feedback (OPFB), they are implemented on the linearised longitudinal F16 model. In order to establish an objective performance comparison, their hyper-parameters were tuned with an evolutionary algorithm: Particle Swarm Optimisation (PSO). Results show that LADP and IADP have the same performance in the presence of FS feedback, whereas LADP outperforms IADP when only OPFB is available. Output noise causes LADP based on OPFB to diverge. In the case of IADP based on OPFB, sensor noise improves the performance due to a better exploration of the solution space. The present research aims at bridging the gap between the discussed ADP algorithms and real world systems.

Featured Image

Photo by Samuel’s Photos on Unsplash

Photo by Samuel’s Photos on Unsplash

Why is it important?

The implementation of noise plays a key role in LADP based on Output Feedback approaches (unknown states) and in all IADP agorithms, since it is required to excite the controlled system input in order to learn the system dynamics. However, most research in this field assumes the use of perfect sensors and ignores the potential impact of output noise in the performance of the methods. Hence the goal of this paper is to analyse the change in performance and convergence of LADP and IADP when non-ideal sensors are introduced. Its contribution is twofold. First, it provides an extensive literature review on both algorithms. Second, it compares their performance against each other with and without the presence of output noise. For that purpose, the algorithms are implemented in a linearised F-16 aircraft model.