Embedded multimodal registration of visible images on long-wave infrared video in real time

We describe an embedded system architecture that implements a real-time multimodal registration method which enables multicamera spatio-temporal feature extraction from the combination of visible and long-wave infrared image sequence. Image registration is performed by matching common features between each frame of a visible image to each frame of an infrared image sequence, in order to estimate an affine transformation between each pair of images. The parameters of this affine transformation are estimated recursively on line with the video, thus enabling image registration in real time. The registration algorithm is implemented using a combination of embedded software and dedicated hardware units on a heterogeneous reconfigurable system-on-a-chip. The hardware performs feature detection and extraction, whereas the software estimates the transformation parameters and maps each infrared video frame onto the visible image coordinates. Our prototype implementation runs with a 135MHz clock, consumes 1.8W and utilizes 29% and 54% of the configurable logic and hardware multiplier units available on the chip, respectively.