Learning with edge-labeling graph

Abstract

This thesis discusses some examples of computer vision problems successfully tackled with edge labeling graph. The first part of this dissertation presents a supervised image segmentation algorithm based on joint-kernelized structured prediction. In the proposed algorithm, correlation clustering over a superpixel graph is conducted using a non-linear discriminant function, where the parameters are learned by a kernelized-structured support vector machine (SSVM). For an input superpixel image, correlation clustering is used to predict the superpixel-graph edge labels that determine whether adjacent superpixel pairs should be merged or not. The proposed joint kernel is defined as a combination of an image similarity kernel and an edge-label similarity kernel, which measure the resemblance of two input images and the similarity between two edge-label pairs, respectively. Each kernel function is designed for fast computation and efficient inference. The proposed algorithm is evaluated using two segmentation benchmark datasets: the Berkeley segmentation dataset (BSDS) and Microsoft Research Cambridge dataset (MSRC). It is observed that the joint feature map implicitly embedded in the proposed joint kernel performs comparably or even better than the explicitly designed joint feature map for a linear model. The second part of this dissertation presents a novel edge-labeling graph neural network (EGNN), which adapts a deep neural network on the edge-labeling graph, for few-shot learning. The previous graph neural network (GNN) approaches in few-shot learning have been based on the node-labeling framework, which implicitly models the intra-cluster similarity and the inter-cluster dissimilarity. In contrast, the proposed EGNN learns to predict the edge-labels rather than the node-labels on the graph that enables the evolution of an explicit clustering by iteratively updating the edge-labels with direct exploitation of both intra-cluster similarity and the inter-cluster dissimilarity. It is also well suited for performing on various numbers of classes without retraining, and can be easily extended to perform a transductive inference. The parameters of the EGNN are learned by episodic training with an edge-labeling loss to obtain a well-generalizable model for unseen low-data problem. On both of the supervised and semi-supervised few-shot image classification tasks with two benchmark datasets, the proposed EGNN significantly improves the performances over the existing GNNs.The third part of this dissertation applies EGNN to incremental few-shot learning, a more practical setting where a regular classification network has already been trained to recognize a set of base classes, and several extra novel classes are being considered, each with only a few labeled examples. The goal is to learn novel classes without forgetting the performance on base classes. In meta training stage, the parameters of the graph neural network and initial support node features of the base classes are learned, such that when given a novel class classification task, the learned graph neural network can accurately predict both novel and base class queries. We evaluate the proposed method on miniImagenet few-shot learning benchmark, and showed that the proposed method improves the performances over the baselines that rely on explicit attention-based novel class weight generation.