Adaptive neural network ensemble using prediction frequency

Abstract

Neural network (NN) ensembles can reduce large prediction variance of NN and improve prediction accuracy. For highly non-linear problems with insufficient data set, the prediction accuracy of NN models becomes unstable, resulting in a decrease in the accuracy of ensembles. Therefore, this study proposes a prediction frequency-based ensemble that identifies core prediction values, which are core prediction members to be used in the ensemble and are expected to be concentrated near the true response. The prediction frequency-based ensemble classifies core prediction values supported by multiple NN models by conducting statistical analysis with a frequency distribution, which is a collection of prediction values obtained from various NN models for a given prediction point. The prediction frequency-based ensemble searches for a range of prediction values that contains prediction values above a certain frequency, and thus the predictive performance can be improved by excluding prediction values with low accuracy and coping with the uncertainty of the most frequent value. An adaptive sampling strategy that sequentially adds samples based on the core prediction variance calculated as the variance of the core prediction values is proposed to improve the predictive performance of the prediction frequency-based ensemble efficiently. Results of various case studies show that the prediction accuracy of the prediction frequency-based ensemble is higher than that of Kriging and other existing ensemble methods. In addition, the proposed adaptive sampling strategy effectively improves the predictive performance of the prediction frequency-based ensemble compared with the previously developed space-filling and prediction variance-based strategies.