Incremental learning of expert networks with adaptive resonance theory competition

Abstract

In this dissertation, a framework for automatic model selection is proposed to find an appropriate compromise between over-smoothing and over-fitting for online function approximation. Recently, most of model selection methods build experts network which is made up of local linear functions and corresponding basis functions. Unlike these methods, the proposed method builds experts network which is consisted of local feed-forward neural networks (FNNs) and corresponding basis functions. This strategy improves the accuracy of local approximation by the capability of FNNs (lower bias compared to local linear models) and complements the increasing variance by generation of smaller experts and continuity preservation function. The basic structure of the proposed framework is mixture of experts whose experts are FNNs. The framework generates and adjusts gating network for FNNs to fit the local data from the receptive fields that is in the gating network. For the generation and adjustment, a type of adaptive resonance theory, Gaussian ART, is incorporated into the framework with modification. Using the error of FNNs to generate and adjust the receptive fields are an important idea for the framework. As a result, the proposed framework constructs function approximator from data in incremental way. The proposed framework is significantly based on two different paradigm. Therefore, the implementation is not straightforward and hard to understand. However, real implementation is very easy and the effectiveness is shown through empirical evaluations of numerical function approximation, time-series prediction, and nonlinear plant modeling.