Machine Learning-based Reentry Guidance for the ReFEx Mission - Analysis and Comparison of Reinforcement Learning and Genetic Programming

Aug 5, 2026

This thesis develops and evaluates machine-learning guidance laws for safe atmospheric reentry of the ReFEx winged reusable launch vehicle. Reinforcement Learning and Genetic Programming are applied to generate real-time updates of angle of attack and bank angle commands once a dynamic pressure threshold is reached. Models trained in simulation are validated using a high-fidelity 6-DOF environment developed by DLR, and the resulting performance is compared across the two approaches to support selection of an algorithm suitable for mission certification.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Introduction and Objectives

Reinforcement Learning

Genetic Programming

Institute of Space Systems Executive Summary of the Thesis Machine Learning-based Reentry Guidance for the ReFEx Mission: Analysis and Comparison of Reinforcement Learning and Genetic Programming Laurea Magistrale in Space Engineering-Ingegneria Spaziale Author: Riccardo Cadamuro Advisor: Prof. Francesco Topputo Co-advisor: Dr. Francesco Marchetti Academic year: 2024-2025 1. Introduction and Objectives The reusability of launch vehicles is attracting increasing interest within the space industry, primarily due to its potential to significantly reduce mission costs. However, guiding a vehicle safely through atmospheric reentry from space remains a highly complex Guidance and Control (G&C) challenge, largely due to the uncertain and rapidly changing environmental conditions. As a result, the development of efficient and reliable G&C algorithms has become essential. Machine Learning (ML) techniques are gaining considerable attention, as they enable the offline training of models in simulated environments, which once deployed in real systems, can generate guidance commands in real-time with minimal computational overhead. The objective of this thesis is to apply Reinforcement Learning (RL) and Genetic Programming (GP) to the design of the reentry guidance law of the REusability Flight Experiment (ReFEx) mission: a winged Reusable Launch Vehicle (RLV) designed according to the Vertical Takeoff Horizontal Landing (VTHL) paradigm to follow a realistic trajectory for a general winged RLV. The ReFEx baseline guidance algorithm computes the initial reference commands for angle of attack (α) and bank angle (µ) onboard prior to the start of the reentry phase [3] . Once reentry begins (i.e. a given dynamic pressure threshold is met), these reference commands are iteratively updated in real time by applying corrections δα and δµ until the End of Experiment (EoE) is reached [4] . This work aims to: 1) use RL and GP to train a model capable of generating online the guidance command update during the atmospheric reentry phase; 2) assess the performances of the trained ML models using the high-fidelity 6-DOF simulator, internally developed by DLR, used to certify the state-of-the-art algorithm designed for the real mission; and 3) produce a comparative study between RL and GP. 2. Reinforcement Learning RL is a technique used to train an agent that learns to compute optimal actions through interaction with an environment, aiming to maximize a numerical reward signal. The problem is framed as a Markov Decision Process (MDP), where the decision maker (agent) observes the system’s state as it evolves over time and selects Executive summary Riccardo Cadamuro actions from a feasible set at each time step [6] . The function mapping states into feasible actions is called policy and it is learnt during the training process with the objective of maximizing the so-called cumulative discounted reward in Equation 1 . Gt = Rt+1 + γRt+2 + γ2 Rt+3 + ... = P γkRi+t+1 (1) The discount rate γ ∈ [0 , 1] controls the relative importance of future rewards (γ ≈ 1) versus immediate ones (γ ≈ 0) . The training is performed over episodic tasks, where each episode corresponds to a full trajectory from Entry Interface (EI) to EoE. Once the episode is complete, the initial conditions are reset and a new one is begun. This work uses the Proximal Policy Optimization (PPO) algorithm to train a Multilayer Perceptron (MLP). The PPO algorithm [5] is chosen because it is designed to deal with a continuous action space and to perform small and controlled policy update steps which strongly stabilize the training performances. It can learn the optimal behavior by interacting with the environment without having knowledge of its model (model free method) and it simultaneously trains two networks: a first one to compute the action anda second one to evaluate the action produced by the first (Actor-Critic method) . On the other hand, the MLP has bee

FAQs

What is the main objective of the thesis?

To design a reentry guidance law for the ReFEx mission using reinforcement learning and genetic programming, and to compare their performance for real-time command updates.

How does the baseline ReFEx guidance algorithm work?

It onboard-computes initial reference commands for angle of attack and bank angle before reentry, then iteratively updates these commands in real time with correction terms until the end of the experiment.

Why are reinforcement learning and genetic programming compared in this work?

Reinforcement learning trains a policy to produce continuous guidance actions, while genetic programming evolves symbolic program trees; comparing them highlights differences in training behavior, model structure, and guidance performance under the same high-fidelity validation setup.

Aug 5, 2026

Upload by Clementine