Real-time Compressive Video Reconstruction for Spatial Multiplexing Cameras

In this work, we tackle the problem of real-time compressive video reconstruction in spatial multiplexing cameras using spatially modulated and undersampled focal plane array data. In this setting, a spatial light modulator (SLM) modulates the scene in the image domain by blocking some of the pixels at a higher resolution level, using an SLM such as a digital micromirror device. Here, we first propose a practical warm starting method to achieve real-time reconstruction of streaming video. We then extend it by adapting a multi-hypothesis technique that takes temporal consistency into account using residual reconstruction between frames. We implement the algorithms on a graphics processing unit and give an extensive analysis of the number of required iterations for different imaging settings. We conclude that the proposed methods achieve video reconstruction in real-time from highly undersampled compressed measurements and provide high quality frames in terms of PSNR and SSIM metrics.