Visualization and Display

1.3.5.1 Visualization Platforms

Visualization is an important component of any surgical guidance platform. What- ever the adopted approach may be, it must present the surgical scene in a three- dimensional fashion and provide a high fidelity representation of the surgical field. For most closed-chest cardiac interventions, this information cannot be accessed via

direct vision by the clinician.

Different research groups have employed various custom-developed environments that best suited their applications [162] and were iteratively optimized to address sub- sequent challenges identified during their use [163]. Among the visualization packages reported in the literature, AnalyzeT M, 3D-Slicer, and Image-Guided Surgery Toolkit (IGSTK)are examples of some which have benefited from extensive support over the past decade.

Fig. 1.4: Surface-rendered model of the left atrium and volume-rendered peripheral vascu- lature obtained from a clinical MRI dataset (left panel), employed here to obtain measure- ments of the aorta from a volume-rendered image dataset (right panel). Image courtesy of David R. Holmes III, Mayo Clinic and Graduate School, Rochester, MN.

AnalyzeT M (Biomedical Imaging Resource, Mayo Clinic, Rochester, USA) [164] has grown into one of the longest surviving visualization packages capable of bringing together a wide variety of tools for generalized manipulation, measurement and visu- alization of multi-dimensional medical images within an interactive and user-friendly environment (Fig. 1.4).

The 3D-Slicer package (Brigham and Women’s Hospital, Harvard University, Boston, USA) [165] makes extensive use of open-source libraries such as theVisualiza- tion Toolkit (VTK)[166] and theInsight Toolkit[167]. By embracing the open-source philosophy and receiving support from the image guidance community, this package has found applications in many laboratories around the world.

The IGSTK platform (Georgetown University, Washington, DC) [168] was devel- oped for image-guided interventions and contains basic components to build specific

applications [169] (Fig. 1.5). It also supports several tracking systems, including models from NDI, Ascension, Claron, and Atracsys (Renens, Switzerland).

Fig. 1.5: IGSTK-based navigation graphical user interface employed during a clinical lung biopsy procedure. Image courtesy of Ziv Yaniv, PhD, Georgetown University, Washington DC.

Other examples include the Medical Imaging Interaction Toolkit (MITK) (Ger- man Cancer Research Center, Heidelberg), and theAtamaiViewer(Robarts Research Institute, London, Canada). The MITK is another open-source platform free for development of interactive medical image-processing software. Its newly released image-guided therapy module (MITK-IGT) supports various tracking systems and enables the development of image-guided applications [170]. TheAtamaiViewercom- prises a user interface based on Python and VTK and integrates a wide variety of components for image-guided applications, including multi-modality image visualiza- tion, anatomical modeling, and surgical tracking [171]. A more detailed description of the various functionalities of the AtamaiVieweris provided in Chapter 2, together

with the primary application implemented within the platform — the model-enhanced US-assisted surgical guidance environment.

1.3.5.2 Display Technology

In addition to robust visualization, choosing the most appropriate information display technology is another key aspect of an image guidance platform. Although we live in a technology-driven era, it could become overwhelming for surgeons to visualize, analyze, interpret, and fuse all the information available during procedures to allow optimal therapy delivery. VR and AR environments have provided solutions for enhanced visualization, ranging from fully immersive environments that do not provide the user with any real display of the surgical field, to environments that combine computer graphics with a direct or video view of the real surgical scene. The first head-mounted display (HMD)-based AR system was introduced by Sutherland et al. in 1968 and combined real and virtual images by means of a semi-transparent mirror. Operating binoculars and microscopes were also augmented using a similar approach, as described by Kelly et al. [172] and Edwardset al. [173] for applications in neurosurgery, and further improved and exploited by Birkfellner and colleagues [174, 175] for maxillofacial surgery.

Distinct from the user-worn devices, AR window-based displays allow augmen- tation without using a tracking system. This technology emerged in 1995 with the device introduced by Masutani et al. [176]. The proposed system consisted of a transparent mirror placed between the user and the object to be augmented. An- other example was the tomographic overlay described in [177, 178], which made use of a semi-transparent mirror to provide a direct view of the patient together with a CT slice correctly positioned within the patient’s anatomy. A comprehensive review of medical AR displays is provided by Sauer et al. [179] and Sielhorstet al. [180].

Despite these advances, cardiac interventional guidance is still hampered from a visualization perspective, as it uses traditional OR displays to make imaging in- formation available to the physician. Besides the minimally invasive, robot-assisted procedures performed using the da VinciT M surgical system, which provides the sur-

geon with a real-time stereoscopic view of the surgical field, most ORs still employ standard overhead monitors for information display. These devices are 2D displays and cannot efficiently represent 3D data.

Lo et al. [181] explored several avenues toward optimizing the display delivery of a VR-enhanced US navigation environmentin vitro, using phantom experiments, and in vivo, during pre-clinical swine studies. These options included a simple computer monitor display, standard OR overhead monitors accessible to each member of the clinical team, HMDs worn by the surgeons, which provided a fully immersed repre- sentation of the navigation environment, and a dual-projector stereoscopic display. Three different display paradigms were also tested: a simple user-operated display that integrated two fixed orthogonal views of the surgical scene; an interactive stereo- scopic display offering views of the surgical environment updated in real-time by optically tracking the HMDs; and a dynamic, user-controlled display that allowed the operator to adjust the camera angle as needed during navigation [181]. Most users, including collaborating surgeons, were comfortable using overhead monitors, but found the HMDs more intuitive, despite their progressive discomfort experienced with prolonged use.

A similar stereoscopic visualization paradigm was explored by del Nido’s group [182] at Harvard working on US-guided intracardiac interventions on the beating heart. Their study investigated the feasibility of two display systems for intracardiac navigation of a catheter-based ASD patch delivery in swine models: a stereoscopic 3D echocardiography display and a standard 3D US view. The former technology led to shorter procedure times and increased navigation precision, suggesting that stereoscopic displays have the potential to improve safety of intracardiac, beating- heart interventions.

1.4

Accuracy Considerations

From a clinical perspective, the success of an intervention is assessed according to the therapeutic outcome. From an engineering view point, navigation accuracy is constrained by the limitations of the IGI system. The overall targeting error within

an IGI framework is dependent on the uncertainties associated with each of the com- ponents [183]. Jannin et al. [184] emphasized that a proper IGI system validation should estimate the errors at each stage in the image-guided therapy process, and study their propagation through the entire workflow. Following these suggestions, the accuracy challenge can be posed as a series of questions: What is the tolerable clinical error associated with the procedure? How accurate is the pre-operative mod- eling and planning? How accurate is the image-/model-to-patient registration? How accurate is the surgical tracking system? What is the overall targeting accuracy of the IGI system?