For autonomous vehicles to drive safely, it is crucial to predict the motion of other vehicles as well as to detect them. The motion of the vehicle is a challenging problem because it is influenced by many variables such as the road environment and interactions between traffic participants. In this paper, we propose a deep learning-based network that uses Convolutional Gated Recurrent Units (GRU) for robust trajectory prediction. We used sequential images rasterized the position, dimension, and heading of surrounding vehicles and High Definition map. Our method outputs future probability images, therefore, it can predict multiple paths and the output can be directly used for path planning as a cost map. We evaluate our method on the Lyft dataset and the KAIST campus dataset we collect. Then, we show the prediction accuracy on noisy perception dataset is improved compared to other methods. In addition, our method runs at 20 ms.