Multi-Source Domain Alignment for Domain Invariant Segmentation in Unknown Targets

Abstract

Semantic segmentation provides scene understanding capability by performing pixel-wise classification of objects within an image. However, the sensitivity of such algorithms towards domain changes requires fine-tuning using an annotated dataset for each novel domain, which is expensive to construct and inefficient. We highlight that irrespective of the training dataset, structural properties of scenes remain the same hence domain sensitivity arises from training methodology. Thus, in this paper, we propose a domain alignment approach wherein multiple synthetic source domains are used to train an underlying segmentation network such that it performs consistently in unknown real target domains. Towards this end, we propose a pixel-wise supervised contrastive learning framework that enforces constraints in latent space resulting in features belonging to the same class being clustered closely and away from different classes. This approach allows for better capturing of global and local semantics while providing domain invariant properties. Our approach can be easily incorporated into prior semantic segmentation approaches without the significant computational overhead. We empirically demonstrate the efficacy of the proposed approach on GTAV → Cityscapes, GTAV+Synthia → Cityscapes, and GTAV+Synthia+Synscapes → Cityscapes scenarios and report state-of-the-art (SoTA) performance without requiring access to images from the target domain.