Efficient input selection for sketch recognition with convolutional neural network

Abstract

In this paper, we explore freehand sketch recognition with deep neural networks. Convolutional neural network(CNN) shows state of the art performance in many computer vision tasks including image recognition. However, there have been few attempts to design a specialized network for sketch recognition. Freehand sketches are significantly different from natural images, which makes it difficult to classify sketches with conventional CNNs intended for natural image recognition. We make an attempt to utilize characteristic properties of sketches in order to design an efficient CNN structure to recognize sketches. Since sketch is an abstract drawing, they contain much less information than natural images. Furthermore, publicly available dataset are small compared to sizes of CNNs used recently. Thus, we need to investigate if CNN architectures for sketch recognition proposed so far are appropriate in terms of their size and depth. We investigate this problem deeply, and propose new methods of training a network with sketches. We utilize partial sketches in a way that we can choose specfiific parts which affffect the network positively. With this method, we achieve improved performance on complete sketch recognition. Our work can be applied to other data types which contain temporal information as in sketches.