Exploiting global and local attentions for heavy rain removal on single images

Abstract

Heavy rain removal from a single image is the task of simultaneously eliminating rain streaks and fog, which can dramatically degrade the image quality. Most existing rain removal methods do not generalize well for the heavy rain case. In this work, we propose a novel network architecture consisting of three sub-networks to remove heavy rain from a single image without estimating rain streaks and fog separately. The first sub-network, an auto-encoder with skip-connections and Spatial Channel Attention (SCA) blocks, extracts global features that provide sufficient contextual information needed to understand the scene geometry and remove atmospheric distortions caused by rain and fog. The second sub-network learns the additive residual information, which is useful in removing rain streak artifacts via our proposed Residual Inception Modules (RIM). The third sub-network adopts our Channel-Attentive Inception Modules (CIM) and selectively learns the essential local features for de-raining by modulating the intensities of the heavy rain images. Our three sub-networks' intermediate outputs are then combined via an adaptive blending mechanism to generate the final clean image. Our method with SCA, RIM, and CIM dramatically outperforms the previous single-image de-raining state-of-the-art methods on the synthetic datasets and shows considerably cleaner and sharper de-rained estimates on the real image datasets. We present extensive experiments and ablation studies supporting each of our method's contributions on both synthetic and real image datasets.