Model Averaging via Penalized Regression for Tracking Concept Drift

Cited 7 time in webofscience Cited 0 time in scopus
  • Hit : 14
  • Download : 0
A supervised learning algorithm aims to build a prediction model using training examples. This paradigm typically has the assumptions that the underlying distribution and the true input-output dependency do not change. However, these assumptions often fail to hold, especially in data streams. This phenomenon is known as concept drift. We propose a new model combining algorithm for tracking concept drift in data streams. The final predictive ensemble model has a form of a weighted average and ridge regression combiner. The coefficients of the combiner are determined by ridge regression with the constraints such that the coefficients are nonnegative and sum to I. The proposed algorithm is devised via a new measure of concept drift, the angle between the estimated weights from data and the optimal weight vector obtained under no concept drift. It is shown that the ridge tuning parameter plays a crucial role of forcing the proposed algorithm to adapt to concept drift. Our main findings include (i) the proposed algorithm can achieve the optimal weights in the case of no concept drift if the tuning parameter is sufficiently large, and (ii) the angle is monotonically increasing as the tuning parameter decreases. These imply that if the tuning parameter is wellcontrolled, the algorithm can produce weights which reflect the degree of concept drift measured by the angle. Using various numerical examples, it is shown that the proposed algorithm can track concept drift better than other existing ensemble methods. Supplemental materials, computer code and R-package, are available online.
Publisher
AMER STATISTICAL ASSOC
Issue Date
2010-06
Language
English
Article Type
Article
Citation

JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS, v.19, no.2, pp.457 - 473

ISSN
1061-8600
DOI
10.1198/jcgs.2010.08104
URI
http://hdl.handle.net/10203/285762
Appears in Collection
MA-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.
This item is cited by other documents in WoS
⊙ Detail Information in WoSⓡ Click to see webofscience_button
⊙ Cited 7 items in WoS Click to see citing articles in records_button

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0