Compressed ultrahigh-speed photography enabled by a snapshot-to-video autoencoder

Ultrahigh-speed optical imaging of transient scenes is indispensable in numerous areas of study, however, existing techniques, e.g., compressed optical-streaking ultrahigh-speed photography (COSUP), are limited by the long processing time and unstable image quality in existing analytical-modeling-based video reconstruction. To address these limitations, we have developed a snapshot-to-video autoencoder (S2V-AE)-a new neural network that maps a compressively recorded 2D image to a movie. The S2V-AE enables the development of single-shot machine-learning assisted real-time (SMART) COSUP, which features a reconstruction time of 60 ms and a large sequence depth of 100 frames. SMART-COSUP is applied to wide-field multiple-particle tracking at 20 thousand frames-per-second (kfps).