Resource Management in Next Generation MEC-Enabled Wireless Networks Using Machine Learning

Abstract

With the help of unmanned aerial vehicles (UAV) and multi-access edge computing (MEC), the Internet of Things (IoT) can provide valuable insights for a smart farm. The IoT devices are installed across the farmland and they monitor the land by performing image classification tasks. Some of the tasks include identifying a fire, monitoring the farmland for pest infestations, and identifying the growth stages of the crops. These tasks must be done frequently to provide the latest information, and taking too long to complete these tasks will lead to irrecoverable damages. Each task has a predetermined deadline. To ensure these computationally heavy tasks are completed on time, the IoT devices can offload these tasks to nearby hovering UAVs to perform the tasks on their behalf. One significant limitation of IoT devices and UAVs is that these devices rely on batteries. To provide support for these nodes that have finite energy, a MEC server is added as another computing resource to further alleviate the task load. The wireless network is the cornerstone of the smart farm network, and it requires an algorithm to schedule the tasks. The decision-maker must consider the network's current energy capacity, computing resources, and deadlines for the tasks. This thesis considers a series of rule-based task scheduling algorithms as well as machine learning algorithms. The rule-based algorithms are based on round robin, or heuristics such as the UAV's current remaining energy and the networks' current computing queue. This thesis explores various reinforcement learning (RL) techniques namely Q-Learning, Deep Q-Learning (DQL), Deep Risk Sensitive (DRS), and DQL with Risk Quantification (RQ). These RL techniques jointly consider the energy and deadline limitations in their objective functions. The simulation results show that the machine learning algorithms significantly outperformed the rule-based algorithms. The RL techniques that employed deep learning had lower percentages of tasks that exceeded their deadlines. In addition, the deep neural networks DNN helped decrease the number of iterations required to achieve the optimal solution. This means that DQL and DRS were able to achieve the optimal solution faster than their tabular Q-Learning and Risk-Sensitive counterparts. In the simulation results, the majority of the tasks that did not meet their deadlines were fire detection tasks. This was because the fire tasks occurred frequently and had a shorter deadline. Unlike the other RL techniques, the RQ algorithm quantified the severity of each type of deadline violation and UAV battery levels in terms of damages. This enabled the agent to avoid actions that lead to severe damages. As a result, the RQ method was the only algorithm able to eliminate deadline violations that were fire detection tasks.