On the usage of augmented reality in hepatic oncosurgery

Half of all patients diagnosed with colorectal cancer develop liver metastasis during the course of their disease. Surgery provides the best opportunity for long-term survival and even the chance for cure. In order to achieve an adequate tumor resection, the surgeon must have precise knowledge of the tumor location relative to the surrounding major vascular structures. However, in order to do so, current surgeons have to continually look away from the patient to see the patient CT scan or MRI on a nearby monitor. Thus, there is then a clinical need for a better system that can assist surgeons.

In this project, the feasibility of using augmented reality (AR) to assist open liver surgeries was studied. Separate volumes of liver surface, vascular and tumor structures were delineated from an animal’s liver CT scan. These were then 3D printed and further rendered by means of a custom application within the HoloLens™ head-mounted display. The accuracy of image registration was subjectively evaluated by checking the alignment of the visible structures between the virtual model and the 3D printed model. By using a combination of tracked hand gestures and voice commands, the AR setup developed successfully indicated with high accuracy the location of the tumor and vasculature of the liver model. These significant results suggest that the superimposition of structures onto the patient liver could facilitate the orientation and navigation during a surgery. This would result in the improvement of tumor resection, leading to an increasing patient safety.