Defend deep neural networks against various adversarial attacks via stochastically binarized activation

Abstract

Deep learning is widely known for"black box" technology. If we use deep learning it works well, but we know that there is a lack of knowledge about why it works well. However, most of the existing Deep Neural Networks are week to the "adversarial example" created by making a small perturbation change designed artificially. Training with adversarial examples is one of the measures to make the neural network more powerful. However, applying single-step adversarial examples to adversarial defenses can make the network overfit and can not support the network’s robustness. Single-step training takes less time, but results in poor performance and problems such as over-fitting. As a way to solve it, multi-step training offers the best performance, but it takes a lot of time. Therefore, we propose a methodology, stochastically binarized activation (SBA), which is related to solves the problem of over-fitting in single-stage adversarial training and quickly achieves robustness comparable to multistep training. SBA provides random selectivity for activation functions, weakening adversarial-attack effects and allowing the network to learn robustness with only single step training. Through various experiments, we combined SBA with FGSM adversarial training to experiment with one of the most difficult white box attacks -PGD attack. The results show state-of-the-art robustness. But the computational cost of the experiment is much cheaper. Finally, by visualizing the learning process of the network, we examine qualitatively how well defenses against adversarial attacks.