Solving the Lunar Lander Problem with Multiple Uncertainties using a Deep Q-Learning based Short-Term Memory Agent

Abstract

Efficient space travel requires intelligent and robust control mechanisms during spacecraft landing scenarios. Developing a control mechanism for a rocket trajectory problem is inherently complex. This paper introduces a novel approach using Deep Q-Learning (DQL) with Short-Term Memory (STM) to address the intrinsic challenges of this task. Unlike traditional Q-Learning methods, our DQL STM agent performs in an environment with uncertainties such as starting position, gravity, and wind in both training and simulation, allowing for enhanced robustness in difficult environmental conditions. This adaptation enables the agent to observe 𝑛�-previous state-action pairs, offering a more accurate estimation of environmental dynamics. Experiments demonstrate that this new approach yields better results under stricter testing conditions compared to previous methods. Moreover, we establish the innovative aspects of our methodology through systematic comparisons with basic Q-Learning, highlighting the merits of the DQL STM agent.