Development of vision-based predictive pedestrian collision warning service through systemic safety analysis for cooperative-intelligent transport system

Abstract

As technology advances, services to prevent vehicle-pedestrian collisions at intersections are undergoing various changes, ranging from adding arrows to a sensor-based braking system that automatically decelerates when a vehicle is expected to collide with a pedestrian in front. However, driver distraction cause collision and automatic emergency braking systems often fail when pedestrians are outside the sensor's range, especially in urban intersections. To compensate for problems with existing systems, the next-generation Cooperative-Intelligent Transport System (C-ITS) is established and Pedestrian Collision Warning Service (PCWS) is introduced to prevent pedestrian-vehicle collisions preemptively. PCWS prevents pedestrian and vehicle collision by transmitting detected pedestrian information to the vehicle in real time through pedestrian detectors installed at intersections. Since it is a safety service that requires immediate warning to vehicles approaching the intersection, the process from detection to warning should be carried out in real time. However, for commercialization, there are many problems that need to be solved, such as securing reliability of pedestrian detection, classification and communication latency. Therefore, we first conduct a risk assessment on the current pedestrian collision warning service, which is actively conducting empirical studies in local government, and explore the direction of development. As a method for this, we propose a risk assessment based on Functional Resonance Analysis Method (FRAM), which is a well-established systemic method that describes the system through functions, couplings, and variability. We conduct expert evaluations and quantitative variability simulation to evaluate the relative risk level in the system and derive strategies to increase the performance. PCWS should ensure safety by lowering the possibility of collision between vehicles and pedestrians as much as possible. However, even if a warning is made in real time based on pedestrian information on crosswalks, chances are high that the collision cannot be avoided due to insufficient avoidance time. Thus, we devise a method to predict the crossing intention of pedestrians before crossing and provide potential collision warnings to approach vehicles, which we call Predictive Pedestrian Collision Warning Service (P2CWS). Previous studies to predict pedestrian crossing intention have utilized various variables, including pedestrian location, speed, pedestrian signal, and speed of surrounding vehicles. In this study, we set head orientation as an additional variable that confirms the presence of vehicles approached by pedestrians. For comparative analysis of performance with or without head orientation, we validate the performance with real traffic scene data and confirm that considering head orientation shows better performance for pedestrian crossing prediction. Secondly, in order for predictive warning services to be commercialized in C-ITS, all processes must be done in real time. We develop an image processing-based framework that allows real-time extraction of all features before crossing. In particular, we demonstrate high accuracy using 3D pose estimation to extract pedestrian head and body orientation and extract all features at a rate of 100.53 fps that they are suitable for real-time service. Finally, we develop P2CWS that consists of a real-time feature extraction system through an image processing framework and a pedestrian crossing intention prediction system of various machine learning models. Real-time extraction features are divided into Pedestrian Features, Vehicle to Pedestrian Interaction and Environmental Contexts. The extracted features have been utilized to predict pedestrian intention through various machine learning models. To verify the performance of P2CWS, 458 pedestrian data were actually collected at the right-turning intersection in Yuseong-gu, Daejeon. As a result, we predicted pedestrian crossing intention in real time (35.76 fps) and showed high accuracy of 93.28%. We develop vision-based predictive pedestrian collision warning services through risk assessment of cooperative-intelligent transportation system. The service, which preemptively prevents collision by predicting pedestrians' crossing intentions, is expected to contribute to autonomous driving in the future by helping to secure pedestrian safety in the city and establish cooperative-intelligent transportation system without accident threats.