A Virtual Reality Environment for Patient Data Visualization and Endoscopic Surgical Planning

Visualizing patient data in a three-dimensional (3D) representation can be an effective surgical planning tool. As medical imaging technologies improve with faster and higher resolution scans, the use of virtual reality for interacting with medical images adds another level of realism to a 3D representation. The software framework presented in this paper is designed to load and display any DICOM/PACS-compatible 3D image data for visualization and interaction in an immersive virtual environment. In "examiner" mode, the surgeon can interact with a 3D virtual model of the patient by using an intuitive set of controls designed to allow slicing, coloring, and windowing of the image to show different tissue densities and enhance important structures. In the simulated "endoscopic camera" mode, the surgeon can see through the point of view of a virtual endoscopic camera to navigate inside the patient. These tools allow the surgeon to perform virtual endoscopy on any suitable structure. The software is highly scalable, as it can be used on a single desktop computer to a cluster of computers in an immersive multiprojection virtual environment. By wearing a pair of stereo glasses, a surgeon becomes immersed within the model itself, thus providing a sense of realism, as if the surgeon is "inside" the patient.