A Cloud-based Stream Processing Platform for Traffic Monitoring using Large-scale Probe Vehicle Data

Probe vehicle data, also known as floating car data or connected vehicle data, is the data collected from GPS-enabled sensors on vehicles. With the advancement in wireless communications and localization technologies, more and more vehicles are expected to be equipped with such sensors. Existing studies only focus on using small-scale probe vehicle data. In this paper, we are interested in developing a real-time parallel stream processing framework to extract traffic flow KPIs from large-scale probe vehicle data. The developed framework is implemented using Apache Storm on Amazon AWS, and can process one million probe vehicle messages per second. Various design considerations, such as data partition and delay processing are discussed. To evaluate the performance of stream processing framework, simulated probe vehicle data based on the actual traffic flows in Jurong Lake District (JLD) of Singapore, is generated using the microscopic simulation software VISSIM. The JLD data is replicated multiple times to represent the one million population of vehicles in Singapore. GPS errors and communication delays are added to represent the real situations before the data is fed to stream processing module. The estimated KPIs from our stream processing model are validated against the ground truth values under different penetration levels.