Continual learning method by estimating previous task gradient via Taylor approximation

Abstract

I address the gap between the regularization and memory based methods for mitigating catastrophic forgetting in the continual learning scenario. Memory based methods, especially experience replay(ER), store the few samples in the replay memory to use it later tasks. Many researches show that memory based approaches can easily achieve the state of the art performance, however, those methods still need to store samples from previous tasks, which violates the common assumption of continual learning problem. Although the regularization based methods can maintain this assumption, those methods still far from the state of the art performance. To mitigate this performance gap without stored samples, I study how to approximate ER methods directly, unlike the purpose of the regularization based methods. I choose the method to approximate previous task gradients by first-order Taylor approximation with the stored gradients and diagonal hessian values at final previous task parameter point. The result in the various data-sets shows that this approximation method can make better or comparable performance with the regularization-based methods, which use fixed neural network size and without stored examples.