Automating Blastocyst Formation and Quality Prediction in Time-Lapse Imaging with Adaptive Key Frame Selection

Effective approaches for accurately predicting the developmental potential of embryos and selecting suitable embryos for blastocyst culture are critically needed. Many deep learning (DL) based methods for time-lapse monitoring (TLM) videos have been proposed to tackle this problem. Although fruitful, these methods are either ineffective when processing long TLM videos, or need extra annotations to determine the morphokinetics parameters of embryos. In this paper, we propose Adaptive Key Frame Selection (AdaKFS), a new framework that adaptively selects informative frames on per-input basis to predict blastocyst formation using TLM videos at the cleavage stage on day 3. For each time step, a policy network decides whether to use or skip the current frame. Further, a prediction network generates prediction using the morphokinetics features of the selected frames. We efficiently train and enhance the frame selection process by using a Gumbel-Softmax sampling approach and a reward function, respectively. Comprehensive experiments on a large TLM video dataset verify the performance superiority of our new method over state-of-the-art methods.