Multimodal image registration between SWIR and LWIR images in an embedded system

We present an algorithm for multimodal registration between short-wave and long-wave infrared images. We use a histogram of oriented gradients to extract features in each image, the Chi-square distance to match the features, a projective transformation to map the objective image onto the reference system, and bilinear interpolation to obtain the pixel values. We designed a heterogeneous embedded system that combines a custom hardware accelerator to perform coordinate transformation and pixel value interpolation, and a programmable processor core to perform feature extraction, feature association, and to compute the transformation parameters. We implemented our design on a Xilinx Zynq XC7Z020 system-on-a-chip, which uses 2.525W of power, 30% of the logic resources of the chip, and 60% of the available on-chip memory. The system runs at 66.6MHz, which allows us to process 640x512-pixel images at more than 60 frames per second after the initial calibration to obtain the transformation parameters.