RobustSsF: Robust missing-modality brain tumor segmentation with self-supervised learning-based scenario-specific fusion

Abstract

All modalities of multi-modal Magnetic Resonance Imaging (MRI) play crucial roles in diagnosing brain tumors. However, the missing or incompleteness of modalities poses challenges in brain tumor segmentation. Existing models have failed to achieve robust performance across all missing-modality scenarios. To address this issue, this paper proposes two main ideas. Firstly, we suggest a novel 4encoder-4decoder architecture that effectively combines “dedicated” and “single” models. Our model includes multiple Scenario-specific Fusion (SsF) decoders that construct different feature maps depending on the missing modality scenarios. Secondly, we newly define the self-supervised learning-based loss function called Couple Regularization (CReg) to train our model and achieve robust learning. The experimental results on BraTS2018 demonstrate that RobustSsF has successfully improved robustness by reducing standard deviations from 12 times to 76 times lower, also achieving state-of-the-art results in all scenarios when the T1ce modality is missing.