Traffic speed prediction is essential for efficient traffic operation and management by distributing demand concentration in time and space. To make an accurate prediction, it is required to consider spatio-temporal characteristics of the traffic evolution. Recently, deep learning-based approaches, especially Graph Neural Network (GNN) has been widely adopted to reflect the stated characteristics. However, existing GNN models mainly used for short-term prediction, whereas long-term traffic prediction is more useful by enabling earlier and efficient decisions of traffic management as well as individual travels. In this study, we propose Asymmetric Long-Term Graph Multi-Attention Network (ALT-GMAN) algorithm, an extension of the GMAN. ALT-GMAN can predict short and long-term traffic speed by considering asymmetric characteristics of forward and backward waves observed in real roadways. ALT-GMAN is tested with six months highway data of PeMS-Bay area, and MAPE for 3-hours and 6-hours prediction is evaluated as 5.53% and 6.05%, respectively. ALTGMAN outperforms the existing models in short-term speed prediction, and provides a robust performance in long-term prediction problems, too.