Multi spectral pedestrian detection and localization

A fusion method of multi spectral video sources for pedestrian localization is presented. The question of how an image from a far-infrared (FIR) camera can be registered with an image from a CMOS video camera to extract distance information is discussed. Because the nature of thermal images is quite different to standard video images, one of the biggest challenges is to find mutual information from each of the cameras that can be combined. A new approach for multi-modal stereo-matching based on contour information as common feature is introduced. In the first step, the object contour is extracted on hot spots in the FIR image by means of extended active contour models. In the second step the stereo correspondence problem is solved with a fast active contour shape matching algorithm utilizing the epipolar constraint. Finally, a postponed image classification based on histograms of gradients decides if the region of interest encloses a relevant object such as a pedestrian.