In order to provide various services using robots, autonomous navigation system for the robots is required. In this regard, it is necessary to estimate the pose of the robot from the sensor attached to the robot and to build a map representing the surrounding environment. From the past, the odometry representing the approximate pose of the robot have been usually calculated through the wheel encoder built in the wheel of the robot. The estimated pose from the odometry was corrected based on the generated map. This technique of estimating the approximate pose of the robot and generating a map in simultaneously is referred to as simultaneous localization and mapping (SLAM). However, the conventional SLAM method has limitation that it is only capable to the mobile robot. Recently, in order to overcome the limitation, only visual sensors are used to try to solve SLAM problem. Conventional visual SLAM research has been studied primarily in static environments. However, in a real environment, there are many moving objects such as cars, animals, and people. Since the pose of the robot is estimated using only the visual information, the most existing methods are affected by the dynamic motion. Therefore, in this paper, we propose a robust background-model based visual SLAM (BaM-SLAM) in a densely high dynamic environment. The proposed algorithm estimates the background-model represented by the non-parametric model from depth scenes, and then it estimates the ego-motion of the sensor using the energy-based dense-visual-odometry approach based on the estimated background-model (BaMVO) in order to consider moving objects. Since the proposed method applies the estimated background-model to ORB-SLAM2, it is possible to calculate the pose with emphasizing the static area and to remove correspondences of dynamic objects. Experimental results demonstrate the pose is robustly estimated by the proposed BaMVO and BaM-SLAM in a densely high dynamic environment. Finally, to prove the applicability of the proposed method, the proposed algorithm is applied to the path following for a robot.