Generating test input with deep reinforcement learning

Abstract

Test data generation is a tedious and laborious process. Search-based Software Testing (SBST) automatically generates test data optimising structural test criteria using metaheuristic algorithms. In essence, metaheuristic algorithms are systematic trial-and-error based on the feedback of fitness function. This is similar to an agent of reinforcement learning which iteratively decides an action based on the current state to maximise the cumulative reward. Inspired by this analogy, this paper presents the approach of employing reinforcement learning to SBST to replace human designed metaheuristic algorithms. We reformulate the software under test (SUT) as an environment of reinforcement learning. At the same time, we present Gunpowder a novel framework for SBST which extends SUT to the environment of reinforcement learning. We trained Double Deep Q-Networks (DDQN) agent with deep neural network and evaluated the effectiveness of our approach by conducting an empirical study. Finally, we find that agents can learn metaheuristic algorithms for SBST achieving 100% branch coverage for trained function. Our architecture sheds light on the future integration of deep neural network and SBST.