Development of robust kernel-based learning algorithms for regression and classification problems

Abstract

Regression and classification are frequently employed machine learning tasks to estimate the underlying functional relationship between input and output variables in a data set. In many practical applications, the underlying functional relationship may be highly nonlinear, and kernel-based methods have appeared and assumed a central role in machine learning area for exploring such nonlinearities effectively. Most of the existing kernel-based learning algorithms have been developed under the assumption that a given data set consists of observations independently taken from the same distribution. Consequently, the results of such algorithms are highly sensitive to the outliers in a data set. To provide reliable results even if a given data set contains outlying observations, robust learning algorithms for regression and classification are developed in this thesis in the kernel framework. For regression, a new robust kernel-based regression algorithm is developed in the first study. The proposed method utilizes a weighting scheme based on the hat matrix similar to the generalized M-estimator of conventional robust linear regression. The diagonal elements of the hat matrix in the kernel-induced feature space are used as leverage measures to reduce the effects of outliers. Computational results from simulated examples and real data sets show the robustness of the proposed method compared to the existing approaches. Another kernel-based regression method insensitive to outliers is developed in the second study based on the relevance vector machine (RVM) with a weighting strategy. The proposed method has several advantages in that it provides statistical intervals and requires no validation data set. A semiconductor plasma etching process is used as a case study to compare the predictive performance of the proposed method with that of other regression methods. Experimental results demonstrate that the proposed method can be used for the purpose of predicting the qual...