Introduction Port placement and improved navigation are key issues in minimally invasive endoscopic surgery, particularly in robotically assisted surgery, where the setup and port placement is the bane of an efficient surgical work flow. A good choice of the instruments’ and endoscope’s ports provides full access to the whole operating region and adequate surgeon dexterity, so time-consuming consecutive new port placement can be avoided. We present a new method to intuitively and precisely plan the port placement and improve the surgeon’s intra-operative orientation by augmenting the limited endoscopic view. A virtual flight through the patient’s interior along the instruments’ axes is provided to easily validate possible ports. Methods Prerequisite for augmenting endoscopic images is a precise camera calibration. The endoscope calibration is a one-time procedure, where the intrinsic camera parameters including distortion coefficients are determined. Additionally, the endoscope needs to be tracked, for instance by an optical tracking system or a mechanical tracking system such as da Vinci. The missing offset between camera center and tracking target can be determined by so-called hand-eye calibration methods, where calibration images are acquired from a set of differing endoscope poses. All involved transformations during hand-eye calibration are outlined on figure 1. In a fast and accurate method the patient is registered to its pre-operative CT by matching at least four fiducials, which are attached to the patient’s thorax and are visible in its CT. They are automatically segmented in the CT. By moving the tracked endoscope around the fiducials just before the intervention, they can be automatically detected in the 2D images. Using the 2D positions of a fiducial detected in at least two images taken from differing endoscope poses, its 3D position is automatically reconstructed by triangulation techniques, as visualized in figure 2. The correspondences between the reconstructed points and their equivalents in the CT data set are automatically found by a distance-weighted graph matching approach. The transformation registering the patient to its CT data is automatically computed by a dual quaternion approach. For port placement, the pre-operative CT volume can be directly rendered with respect to the endoscope or instruments. This is achieved in real time, since all necessary computations for volume rendering are executed directly on the graphics card. Additionally, no prior time-consuming segmentation of the CT volume is needed any more. So the real endoscope images can be overlaid with virtual transparent or opaque images of pre-operative data. This enables the simulation of a virtual camera flight through the patient’s interior along the instruments’ axes to easily validate possible ports. Intra-operatively, at any time the endoscope image can be zoomed in and out, again providing the rendered CT data, if desired. This provides an enhanced orientation within the patient’s anatomy to the surgeon. To achieve realistic results during rendering, a set of pre-defined transfer functions is provided to map CT intensities to natural colors.Three different visualization modes displaying a real, augmented, and virtual endoscope image can be found on figures 3 - 5. Experiments 13 fiducials were attached to a thorax phantom. Four of them were used to register the phantom to its CT data. All fiducials were used to determine the augmentation error. From orthogonal views, the distances of the fiducials from the real images to a semi-transparent augmentation were measured directly in the video images. An average error of 2.6 mm and a maximum error of 5 mm could be determined. This is fully sufficient for a precise port placement and improved navigation within rigid anatomy. Conclusion Two major problems of current approaches to port placement and endoscopic augmentation were addressed and solved: Time-consuming segmentation of patient’s anatomy and inadequate patient registration. Our method is easy to integrate into the current clinical work flow and applicable to any endoscopic intervention, as long as the endoscope is tracked.     «

Introduction Port placement and improved navigation are key issues in minimally invasive endoscopic surgery, particularly in robotically assisted surgery, where the setup and port placement is the bane of an efficient surgical work flow. A good choice of the instruments’ and endoscope’s ports provides full access to the whole operating region and adequate surgeon dexterity, so time-consuming consecutive new port placement can be avoided. We present a new method to intuitively and precisely pla...     »