Attention-based image classification using semantic segmentation and convolutional neural network

Abstract

Images contain numerous information and there are thousands of studies that are used to collect that information. Image classification is one of the approaches to obtain visual information. Previous methods usually use image classification that derives information from either object-based or scene-based categorization. These methods classify different images based on objects or scenes

however, when people take photographs, they usually focus on capturing specific subjects. Thus, an attention-based subject contains semantic information in the image, and this is another basis for classification of images. In this thesis, we propose an approach for image classification based on human attention. For our approach, we use two steps: First, we use semantic segmentation to identify which image pixels belong to objects and scenes and how meaningful content is saturated because parts that receive attention are related to what the image consists of. Second, we use convolutional neural network to classify images using semantic information extracted from the first step as the input of convolutional network.