IDLE: Integrated Deep Learning Engine with Adaptive Task Scheduling on Heterogeneous GPUs

Abstract

As the deep learning (DL) has widely been used for application domains such as image classifications, natural language processing, and speech recognition, various software frameworks have been developed. They provide users with efficient programming interfaces for developing the DL applications. The optimization techniques within these frameworks generally are different from each other, which leads to different processing times for even the same applications. However, it is difficult that end users consider performance differences in processing time due to incompatible programming interface among the DL frameworks. These differences might cause redundant efforts and costs for end users to develop and maintain the applications. In this paper, we introduce an integrated deep learning engine (IDLE), a novel interface working on the top of the existing DL frameworks, which provides a convenient, flexible and scalable programming interface developing the DL applications for end users regardless of DL frameworks. Besides, we also propose a novel adaptive task scheduling scheme for training DL applications in a cluster with different GPUs. We implement our platform on the heterogeneous GPU cluster, and the results show that the proposed scheduling algorithm improves cost efficiency processing various DL applications.