Event-Based Obstacle Sensing and Avoidance for an UAV Through Deep Reinforcement Learning

Event-based cameras can provide asynchronous measurements of changes in per-pixel brightness at the microsecond level, thereby achieving a dramatically higher operation speed than conventional frame-based cameras. This is an appealing choice for unmanned aerial vehicles (UAVs) to realize high-speed obstacle sensing and avoidance. In this paper, we present a sense and avoid (SAA) method for UAVs based on event variational auto-encoder and deep reinforcement learning. Different from most of the existing solutions, the proposed method operates directly on every single event instead of accumulating them as an event frame during a short time. Besides, an avoidance control method based on deep reinforcement learning with continuous action space is proposed. Through simulation experiments based on AirSim, we show that the proposed method is qualified for real-time tasks and can achieve a higher success rate of obstacle avoidance than the baseline method. Furthermore, we open source our proposed method as well as the datasets.