Live and Interactive 3D Photomanipulation under the Microscope using Virtual Reality

State-of-the-art microscopes, as used in cell biology, are not only capable of capturing 3D images, but also permit manipulation of (sub-)cellular structures using techniques such as optical traps, optogenetics or laser ablation. However, such microscopes are still controlled using 2D interfaces, prohibiting actual 3-dimensional manipulation. We present microscenery, a virtual reality (VR) microscope control software, designed to facilitate 3D laser ablation experiments. We combine microscopy automation with VR rendering and intuitive controller-based input to empower biologists with the precision of laser-based techniques while providing the full 3D spatial context of their sample. We describe the design goals and architecture of the software and illustrate the potential of the system by conducting a brief expert review study for 3D ablation experiments. Our results suggest VR is not only an effective interface for microscopic manipulations, but can enable novel experiments which are either impossible with traditional 2D interfaces, or prohibitively time-consuming.