Enhancing incremental few-shot learning algorithm

Abstract

Few-shot class-incremental learning (FSCIL) aims to enable a neural network to learn new information well from a few data samples when new data is continually provided. In FSCIL, it is very important to prevent catastrophic forgetting, which is lost of existing knowledge. FSCIL method consists of two stages, pretraining and fine-tuning. In pretraining stage, neural network is trined with a plenty of samples of base classes. In fine-tuning stage, the pretrained network is fine-tuned with a few samples of novel classes. Various algorithms suggested until now generally focus on improvement in fine-tuning stage. However, according to F2M, when using prototype as classifier, it performs better than existing algorithms even without fine-tuning. This discovery raised a doubt on the effect of fine-tuning, which make us focus on pretraining stage. Consequently, when we try to improve the performance in pretraining stage, the performance is superior to existing algorithms even when the model is not fine-tuned. Based on this result, we conclude that Making a good embedding in pretraining is much more effective than trying to teach new data well in fine-tuning.