Understanding adaptive deep networks with early exit

Abstract

While deep convolution neural networks show a remarkable performance in a variety of fields recently, the need for studies about reducing the cost of computation and storage to utilize the deep networks in a edge-device is increasing. Early exit can speed up the inference time through adaptively spending the amount of computation on each samples based on the confidence score of predicted outputs. The shallow network augmented in the backbone network provides an adaptive inference path for early exit and also improves the performance of the backbone network by itself. Since the degree of reducing the computational cost depends on the performance of the network, previous studies proposed a self-distillation methodology to effectively train the shallow network. However, the reason of performance improvement due to the early exit structure and the self-distillation have not revealed. In this paper, we understand the fundamental reason of the both and propose a contrastive regularization loss and an ensemble knowledge distillation. In addition, we propose a self-supervised task for learning a confidence score function to address the miscalibration problem of softmax response which has widely used as a confidence measure.