Autonomous drone surveillance in a known environment using reinforcement learning

Abstract

Target recognition, behavior detection, search and rescue missions, and other surveillance and monitoring tasks can all benefit from unmanned Aerial Vehicles (UAVs), or called drones. Using deep reinforcement learning (DRL) to implement some autonomous tasks of robotics has become an important research field with potential growth in the position of robotics movement and computer vision, as it has gradually developed into one of the most popular research fields in learning-based and model-based control, with the characteristics of frequent interaction with the environment. In this research, based on the simulator tool provided in [1], we utilize deep reinforcement learning to develop both single-agent and multi-agents that can accomplish autonomous drone surveillance tasks in a known indoor environment. We combine the benefits of both visual and obstacle information to boost efficacy while ensuring low time consumption. We also employ segmentation mask information with single-channel pixels to perform the detection task to reduce time and increase identification accuracy instead of using RGB or depth images. Finally, we devise a separate reinforcement learning training and testing technique that may both enhance training efficiency and ensure task completion: training the DRL policy without drone dynamics and considering discrete drone actions first, and then accepting dynamics to continuously control the drone in test. This method also creates a new field for sim-to-real transfer. Our experimental results show that the trained agents can detect all targets at a relatively fast speed while maintaining a high level of security, and the patrol completion rate is more than 98% in both single-agent and multi-agents task.