This paper introduces graph structure-based simultaneous localization and mapping (SLAM) using RGB-D and magnetic sensors. We also propose kidnap recovery with graph SLAM structure. The RGB-D sensor can measure the distance value of the corresponding image pixel and magnetic sensor can measure magnetic field distortion in an indoor environment. Since these two sensors have different characteristics, they have different strengths when performing SLAM. RGB-D sensor-based SLAM is suitable for complex 3D environments, and magnetic sensor-based SLAM provides better performance in feature-poor environments such as corridors. Thus, this paper introduces a robust SLAM system in indoor environment by taking advantages of both sensor characteristics. The proposed algorithm has been tested to verify the superiority of algorithms in indoor environment and kidnapping situations.