Rotorcraft Slung Payload Stabilization Using Reinforcement Learning

Abstract

In recent years, the use of rotorcraft uninhabited aerial vehicles (UAVs) for cargo delivery has become of particular interest to private companies and humanitarian organizations, namely due to their reduced operational costs, ability to reach remote locations and to take off and land vertically. The slung configuration, where the cargo is suspended below the vehicle by a cable, is slowly being favoured for its ability to transport different sized loads without the need for the vehicle to land. However, such configurations require complex control systems in order to stabilize the swing of the suspended load. The goal of this research is to design a control system which will be able to bring a slung payload transported by a rotorcraft UAV back to its stable equilibrium in the event of a disturbance. A simple model of the system is first derived from first principles for the purpose of simulating a control algorithm. A controller based in model-free, policy-gradient reinforcement learning is then derived and implemented on the simulator in order to tune the learning parameters and reach a first stable solution for load stabilization in a single plane. An experimental testbed is then constructed to test the performance of the controller in a practical setting. The testbed consists of a quadcopter carrying a weight suspended on a string and of a newly designed on-board load-angle sensing device, to allow the algorithm to operate using only on-board sensing and computation. While the load-angle sensing design was found to be sensitive to the aggressive manoeuvres of the vehicle and require reworking, the proposed control algorithm was found to successfully stabilize the slung payload and adapt in real-time to the dynamics of the physical testbed, accounting for model uncertainties. The algorithm also works within the framework of the widely-used, open-source autopilot program ArduCopter, making it straightforward to implement on existing rotorcraft platforms. In the future, improvements to the load angle sensor should be made to enable the algorithm to run fully on-board and allow the vehicle to operate outdoors. Further studies should also be conducted to limit the amount of vehicle drift observed during testing of the load stabilization.