Waypoint Control of a Quadrotor Carrying a Slung Payload Using Reinforcement Learning

Abstract

In recent years, global e-commerce has experienced a significant rise. This surge has spurred a demand for efficient delivery services, specifically in the last-mile segment, which faces considerable challenges. Implementing Uninhabited Aerial Vehicles (UAVs), especially with the slung payload system, offers effective maneuverability and flexibility to address this issue. That being said, controlling a drone with a slung load poses unique challenges such as real-time adaptability and unpredictable dynamics. The objective of this work was to develop an onboard model free Reinforcement Learning (RL) controller capable of achieving precise waypoint tracking for a drone with a slung load while ensuring stability and accuracy in the presence of payload-induced perturbations. A Temporal Difference (TD) RL control algorithm was developed to enable the drone to achieve waypoint tracking capabilities through commanding changes in attitude. This algorithm was trained in simulation using a developed planar model of the drone-payload system. The trained controller was implemented on a prototype quadrotor in an indoor testing environment with an integrated motion capture system in charge of relaying position and velocity data in real time to the aircraft. Waypoint tracking was applied to the quadrotor using two separate RL agents to achieve full control in the horizontal plane. The proposed fully on-board controller was able to lead the aircraft to the desired goal successfully, closely matching simulation results. In both sets of results there was a small steady state error that could be mitigated in the future through step size optimisation along with additional training.