Subtask aware policy network for long-horizon home service robot tasks

Abstract

Interactive and intelligent service robots that can understand the environment in various situations, decide the actions required to achieve task goals, interact with humans, and perform diverse tasks that help human life has emerged in various forms along with the development of machine learning and deep learning. Among these service robots, home service robots used in domestic environments require 1) understanding the surrounding environment and knowing where to focus and 2) being able to infer what to do to achieve complex long-horizon task goals. Recent studies have been conducted to replace the traditional methodologies for making decisions for intelligent service robots with AI technologies. While some studies to train policies for performing given tasks through end-to-end learning manner have demonstrated successful results, they have shown limitations to the learning performance for long-horizon and interactive tasks. As a result, research has been developed using a modular approach, in which pre-trained modules specialized in some of the necessary functions of given tasks or classical methods are separately constructed and used to create a policy model. These modular policies infer the causality of the actions based on prior knowledge to improve task performance. In this paper, we propose a network that can learn behavior policies that perform complex and long-horizon tasks with high performance through end-to-end learning while reducing the effort of building separate modules for different goal tasks or robot functionalities. We propose an Online Subtask Prediction Network (OSPNet) that can predict subtasks needed at each time step to achieve a given task goal by inferring causal relationships between actions. OSPNet predicts subtask goals based on environmental information and previous subtask prediction history. We also propose a Subtask Aware Policy Network (SAPNet) that predicts appropriate actions for performing tasks based on predicted subtask goals. The proposed policy network is designed to perform complex long-term tasks through end-to-end learning successfully. We verify the feasibility and performance of the proposed network by applying it to a room rearrangement task, one of the long-horizon home service robot tasks, and conducting experiments.