Convolutional neural network with neighborhood context selection for indoor semantic segmentation

Abstract

Semantic segmentation is the process of assigning a class label to each pixel in an image. This is a very challenging task, because in contrast to object detection, which only determines the presence or not of a certain object, semantic segmentation involves parsing a scene by distinctively defining the boundaries of each object, while labeling each pixel with its corresponding category. This thesis proposes a method to adjust the contextual information contained in a pixel neighborhood, without changing the size of the input patch. Instead, the input image size is adjusted accordingly to the depth value associated with the pixel to be labeled. The selected patch is fed to a convolutional neural network, where distinctive features are learned. Finally, the resulting predictions are refined with a coherent clustering of superpixels. The NYUv2 dataset was used to test the performance of the proposed algorithm, which has a total of 1449 labeled RGBD images, with 894 categories. The dataset was split into 795 training images and 654 test images. The proposed method gives state of the art results, with a classification accuracy of 65.1% while being also faster than other algorithms.