This paper focuses on risk estimation problems of urban traffic accidents using deep learning approaches. There are two major challenges in the previous studies. The first challenge is the data imbalance problem that occurs numerous zeros in input data and can negatively affect the risk estimation results. The second challenge lies in neglecting the road environmental factors in risk estimation, which are also essential in causing traffic accidents. In order to address the aforementioned two problems, this study developed a hierarchical deep learning-based model with mobility and road environment data for estimating the risk of urban traffic accidents. The experiment results indicate the proposed method outperforms other existing models. The suggested method can be applied to the traffic warning system to assist people to avoid traffic accidents and further used in traffic accident prediction.