Transferable traffic signal control: Reinforcement learning with graph centric state representation

Cited 0 time in webofscience Cited 0 time in scopus
  • Hit : 71
  • Download : 0
Reinforcement learning (RL) has emerged as an alternative approach for optimizing the traffic signal control system. However, there is a restricted exploration problem encountered when a signal control model is trained with a predefined demand scenario in the traffic simulation. With the restricted exploration, the model learns a policy based only on partial experiences in the search space, which yields a partially-trained policy. Partially-trained policy fails to adapt to some unexperienced ('unexplored', 'never-before-seen') dataset that have different distributions from the training dataset. Although this issue has critical effects on training a signal control model, it has not been considered in the literature. Therefore, this research aims to obtain a transferable policy to enhance the model's applicability on unexperienced traffic states. The key idea is to represent the state as graph-structured data, and train it using a graph neural network (GNN). Since this approach enables to learn the relationship between the features resulting from the spatial structure of the intersection, it is able to transfer the already-learned knowledge of the relationship to the unexperienced data. In order to investigate the transferability, an experiment is conducted on five unexperienced test demand scenarios. For the evaluation, the performance of the proposed GNN model is compared with the conventional DQN model that is based on vectorvalued state. At first, the models are trained with only a single dataset (training demand scenario). Then, they are tested with different unexperienced dataset (test demand scenarios) without additional trainings. The results show that the proposed GNN model obtains a transferable policy so that it adapts better to the unexperienced traffic states, while the conventional DQN model fails.
Publisher
PERGAMON-ELSEVIER SCIENCE LTD
Issue Date
2021-09
Language
English
Article Type
Article
Citation

TRANSPORTATION RESEARCH PART C-EMERGING TECHNOLOGIES, v.130

ISSN
0968-090X
DOI
10.1016/j.trc.2021.103321
URI
http://hdl.handle.net/10203/287845
Appears in Collection
IE-Journal Papers(저널논문)CE-Journal Papers(저널논문)
Files in This Item
There are no files associated with this item.

qr_code

  • mendeley

    citeulike


rss_1.0 rss_2.0 atom_1.0