Video-Based Augmented Reality Stereoscopic System:: an application to traffic scenes

This communication describes an augmented reality application that merges real traffic scenes and their virtual models, then records a driver's actions as he drives through the resulting realistic simulation with an HMD and simulated car cockpit. Two calibrated cameras record stereoscopic videos from a vehicle driving on a real road. The velocity, relative and absolute position of the car are recorded synchronously with the corresponding video frame using DGPS and an odometer. The virtual world is built based on a known topographic map of the traffic scenario with a resolution of less than 10 cm. The information collected by the sensors and a feature location process on the image-frame using computer vision allows the geometric correspondence of the two worlds. The paper shows that using camera calibration and data filtering allows a car to be positioned in the virtual world with a resolution of less than 10 cm moving at 70 Km/h. This application is focused on recording the behavior of a driver who sees situations that could eventually occur on a road lane. Important behaviors include: how a driver understands the position and orientation of traffic signals, how much attention he pays to traffic signals, his reaction time when a traffic lamp changes at a critical instant or a car suddenly breaks in front of him.