Dynamic urban traffic rerouting with fog-cloud reinforcement learning

Abstract

Dynamic rerouting has been touted as a solution for urban traffic congestion. However, its implementation is stymied by the complexity of urban traffic. To address this, recent studies suggest the efficacy of novel technologies like fog computing and deep reinforcement learning. However, there exist significant challenges in this regard: (1) sorting massive amounts of data associated with large urban networks, (2) large action space that hinders learning efficiency, (3) impairment of rerouting efficacy due to overreliance on regional/local information, and (4) the issue of congestion shifting. To overcome these challenges, this paper presents a novel two-step approach that integrates graph attention Q network (GAQ) with entropy-balanced k shortest path (EBkSP) via fog-cloud information framework to carry out vehicle rerouting in dynamic traffic environments typical of large cities. The first challenge is addressed by GAQ that uses attention mechanism to evaluate traffic and assign region indices based on the relative importance of information. The second challenge is addressed using fog nodes that reduce the action space, and the third is addressed using fog computing combined with cloud computing to facilitate the computation of global optimal information in a centralized planning decentralized execution pattern. The fourth challenge is addressed using EBkSP, which identifies vehicle optimal routes based on route popularity and vehicle priority. After conducting a numerical experiment involving two other cutting-edge vehicle rerouting models as baselines, it was established that the proposed model delivers results that are 14%-54% superior from perspectives of the mean travel speed and congestion. The superior efficacy of the proposed learning-based (compared to rule-based) models is pronounced when rerouting ratios are low. As the rerouting ratio increases, both the learning-based and rule-based model outcomes exhibit reduced likelihoods of severe congestion. However, the learning-based model consistently outperforms the rule-based model across all scenarios. The proposed model can effectively reroute vehicles under different situations of rerouting ratio and total count of vehicles and can help improve traffic mobility and reduce congestion in urban areas. It can be implemented easily by urban road agencies.