Preventing attention leakage in slot attention for object-centric learning

Abstract

Object-centric learning (OCL) aims for a compositional understanding of scenes, like humans recognize the visual world at the object level, by representing a scene as a set of object-centric representations. Although OCL has been successfully applied to the multi-view image and video datasets by leveraging geometric or temporal information to adopt various data-driven inductive biases, it faces challenges when applied to single-view images. This is due to the reduced availability of information regard- ing scene decomposition, resulting in the attention leakage problem where object-centric representation gives attention to not only individual objects but also the background around the objects. The attention leakage problem incurs deficient background separation, thereby object-centric representation can be constructed with the background noise. To address this challenge, we introduce SLot Attention via SHepherding (SLASH), a novel OCL framework for single-view images. SLASH integrates two simple- yet-effective modules into Slot Attention: the Attention Refining Kernel (ARK) and the Intermediate Point Predictor and Encoder (IPPE). These modules, respectively, prevent slots from being distracted by background noise and provide focus points to guide the learning of object-centric representations. We further propose a weak semi-supervision approach that leverages point-level annotation for OCL. Even though it is trained with annotation, our framework can be used without any assistant annotation during the inference phase. Experimental results demonstrate that our proposed method enhances the consistency in learning object-centric representations and delivers robust performance across four different datasets.