Biologically inspired imaging sensors for multi-spectral and polarization imagery

Multi-spectral and polarization imaging have enabled and exploited a wide range of applications, from remote sensing to biomedical applications such as early cancer detection for image-guided surgery. However, state-of-the-art multispectral and polarization cameras are still based on conventional advances in optics and integrated circuits, yielding bulky form factors and poor signal reconstruction. Thus, these technologies have failed to be adopted as research or clinical imaging tools. Nature is full of examples of animals that take advantage of multi-spectral and polarization phenomena to gain an evolutionary advantage. For example, elegant low-power and compact biological visual systems-capable of multi-spectral and polarization imaging surpassing any man-made imaging system-can be found in the compound eyes of many arthropods. Here, we demonstrate radically novel, multi-spectral and polarization imaging sensors that function on the same fundamental principles as do the ommatidia of the mantis shrimp. Our bio-inspired imaging systems combine vertically stacked photodiodes, for single-pixel trichromatic vision, with an array of pixelated polarization filters, resulting in compact and low-power architectures. Our single-chip imager comprises of 1280-by-720 pixels, yielding a 62 dB and 48 dB dynamic range and signal-to-noise ratio, respectively, and operates at a maximum frame rate of 24 fps. This topology inherently co-registers in time and space the different spectral and polarization channels. This novel and ergonomic technology is enabling real-time in situ underwater polarization imaging as well as applications in biomedical fields.