An augmented reality platform for planning of minimally invasive cardiac surgeries

One of the fundamental components in all Image Guided Surgery (IGS) applications is a method for presenting information to the surgeon in a simple, effective manner. This paper describes the first steps in our new Augmented Reality (AR) information delivery program. The system makes use of new " off the shelf" AR glasses that are both light-weight and unobtrusive, with adequate resolution for many IGS applications. Our first application is perioperative planning of minimally invasive robot-assisted cardiac surgery. In this procedure, a combination of tracking technologies and intraoperative ultrasound is used to map the migration of cardiac targets prior to selection of port locations for trocars that enter the chest. The AR glasses will then be used to present this heart migration data to the surgeon, overlaid onto the patients chest. The current paper describes the calibration process for the AR glasses, their integration into our IGS framework for minimally invasive robotic cardiac surgery, and preliminary validation of the system. Validation results indicate a mean 3D triangulation error of 2.9 +/- 3.3 mm, 2D projection error of 2.1 +/- 2.1 pixels, and Normalized Stereo Calibration Error of 3.19.