Computational Modeling for Enhancing Soft Tissue Image Guided Surgery: An Application in Neurosurgery

Abstract

With the recent advances in computing, the opportunities to translate computational models to more integrated roles in patient treatment are expanding at an exciting rate. One area of considerable development has been directed towards correcting soft tissue deformation within image guided neurosurgery applications. This review captures the efforts that have been undertaken towards enhancing neuronavigation by the integration of soft tissue biomechanical models, imaging and sensing technologies, and algorithmic developments. In addition, the review speaks to the evolving role of modeling frameworks within surgery and concludes with some future directions beyond neurosurgical applications.

INTRODUCTION

From the early beginnings of modern computing, applications of computational modeling to surgery can be found in the literature and number in the thousands dating back over half a century. However, only with the recent dramatic advances in computer/graphic processing units and efficiencies in memory as well as the development and realization of sophisticated numerical methods nearer the turn of the century, we have witnessed a tremendous increase in these applications such that fundamental new paradigms for exploring surgical applications have become possible. Within this backdrop, any review of the current literature will undoubtedly be somewhat limited as continued important contributions are appearing rapidly. As a result, this review will likely reflect only a snapshot of the field but hopefully captures an impression of important directions for the future of computational modeling and surgery.

The purpose of this review is to highlight the initiatives in one specific example of biomechanical modeling being undertaken by many investigators to solve a challenging problem within the field of image-guided neurosurgery (IGNS), namely how to correct for soft-tissue brain deformations during invasive surgery to enhance surgical guidance. When one thinks of the complex structures and eloquent functional areas of the brain, precise navigation during surgery or intervention is essential for maintaining neurological function. Whether the goal is to place electrodes for deep brain stimulation to correct movement disorders, ablate an epileptogenic source, or resecting a life-threatening glioblastoma multiforme, the challenge of navigating to target safely is of the utmost importance for the patients our technologies treat. All of these procedures are difficult and require a unique assembly of technologies appropriate for the patient and surgical environment.

However, there is an added purpose of the review that is beyond the solution to the IGNS “brain shift” problem with models. Thematically, image guided surgery with its evolution to conventional stereotaxy is really one of, if not the first, patient-specific therapies, i.e. patient-specific X-ray imaging data driving surgery arose out of the very first uses of X-ray imaging in 1896.⁷⁶ It is interesting to note that in some respect only now has conventional medicine really matched pace, e.g. patient-specific tumor genetic profiles for tailored drug therapy. This review now suggests that patient-specific computational biomechanical models are being added to this armamentarium and are serving important roles in patient-specific planning, diagnostics, and therapy delivery. So as the review unfolds, we should remember that every aspect of the discussion below is directed at patient-specific applications; and at the conclusion of the review, we will look to future extensions of patient-specific model-enhanced therapies.

IMAGE GUIDED SURGERY

With the advent of stereotactic techniques, neurosurgery has benefited greatly with the precise positioning of surgical instruments and probes using frames and atlases. Coupled with the incredible advances of imaging modalities over the past four decades, dramatic enhancements have been achieved towards the diagnosis and visualization of human disease. In the past three decades, advances in registration and computational processing have fundamentally transformed surgical techniques to allow guidance feedback during surgery. In addition, the movement to frameless systems over the past two decades represents a fundamental shift towards real-time surgical guidance. The realization of co-registration among patient, operating room (OR) physical space, and the preoperative image series to navigate actively within the brain has become commonplace in the neurosurgical theater as reviewed by Galloway in Ref. 21. While Galloway reviews conventional guidance systems, largely still in use today (i.e. optical tracking technology with registration driven by point-based or surface registration techniques), Galloway explains the slow adoption of image ‘volume-based’ registration methods due to a lack of need in the operating room (OR), i.e. if high-resolution images are produced intraoperatively, preoperative images are no longer needed. While that may have been the case at the emergence of intraoperative magnetic resonance (iMR) and contemporary computed tomographic (iCT) imaging units, it has become clear since, that registering the extensive wealth of imaging data acquired preoperatively is still highly desirable and preop-to-intraop ‘volume-based’ registration has become a necessity. In addition, it seems clear that preoperative resolution quality, while achievable with these intraoperative systems, comes at the cost of intraoperative time. As a result, it has become somewhat common to register high resolution anatomic preoperative data to the lower-quality intraoperative data for the sake of conserving patient surgery times. Since Ref. 21 however, these environments have become widely commercially available for well over a decade now, yet adoption still has been quite slow. While much of the rationale for these systems was to account for soft-tissue changes, the infrastructural investment for these facilities is substantial especially when one realizes the data produced is not typically acquired to the same quality as the preoperative counterpart. Interestingly, computational biomechanical models are serving as one of the cornerstone methods for many of the intraoperative registration frameworks due to their realistic deformation behavior, e.g., Ref. 28. These developments further emphasize the role of biomechanical computational models within field of frameless IGNS.

REALITIES OF SOFT TISSUE SURGERY

The recognition that soft tissue brain deformations, i.e. “brain shift”, affected image guided procedures to a significant degree was quite rapid upon the development of these systems. Specifically, the common assumption in all conventional guidance systems that the process of aligning image to physical space could be characterized by rigid body mechanics was clearly not the experience of practicing neurosurgeons. Strategies to compensate for brain shift appear as early as 1986 by Kelly et al.³⁵ whereby 1-mm stainless steel balls were placed at 5-mm intervals throughout the tumor and monitored using anteroposterior and lateral teleradiographs to monitor the tumor position when using laser resection techniques. While this work recognizes the need for compensation, the first quantitative shift assessments using a conventional IGNS system were reported by Nauta in Ref. 51. Interestingly, Nauta was also concerned with measuring differences between frame-based and frameless systems. In the one case where they assessed the accuracy of both types of guidance systems by comparing image-based tumor centroid locations relative to the true tumor centroid as determined by intraoperative placement, neither frame-based or frameless matched the centroid location of the small tumor, and each was approximately 5 mm away in different locations. Nauta went on to describe the causes of shift which are still the primary mechanisms considered today, namely, “release of cerebrospinal fluid, air entry into the subdural spaces, tumor debulking”, abnormal intracranial pressures, drainage, and hydration changes.⁵¹ A short time later, a series of independent investigations came out revealing the considerable degree the brain can deform during surgery and while Nauta quite clearly stated IGNS systems were compromised, these subsequent studies began to quantify the problem in much more detail, e.g., Refs. ^{27, 56}. Insightful relationships regarding the predisposition for brain movement in the direction of gravity were reported with average cortical surface deformations of 1 cm with some cases having displacements as large as 2.7 cm.^27,56 In addition, with the advent and use of iMR systems, more detailed studies measuring both surface and subsurface shift were performed.^49,52 One of the first detailed studies documenting both surface and subsurface shift was reported by Nimsky et al.⁵² Their work revealed large cortical shifts (>7.0 mm), and moderate shifts (3.0–6.9 mm) in 64 and 14% of cases, respectively. Deep tumor margins experienced large, and moderate shifts in 24 and 42%, respectively. In an independent study, Nabavi et al. reported similar results using iMR technology.^26,49 It is perplexing that while brain tumor surgery is a tissue-sparing procedure, moderate to high shifts of the cortical surface and deep tumor margins occur in 75+% of cases routinely, yet guidance architectures are still employing rigid body assumptions.

Nevertheless, conventional systems with rigid assumptions do provide critical real-time navigation assistance. In a recent review paper by Schulz et al.,⁶² 1400 papers concerned with image guided neurosurgery over the past 25 years were reviewed with interestingly only 14 comparative trials concerning the effectiveness of image guided neurosurgery being identified for inclusion. Despite the small number of qualifying studies, many interesting results were reported across studies. In general, the use of conventional neuronavigation (rigid registration assumptions) lead to reduced residual tumor burden postoperatively, prolonged survival, reduced neurological deterioration, and negligible additional intraoperative timing (longer setups were experienced however). The review went on to look at studies where the conventional guidance systems were integrated into platforms with iMR imaging units to account for soft-tissue changes due to surgery. With respect to these enhanced neuronavigational iMR environments, in general, more complete resections were reported with no additional deficits. In approximately 30% of cases, additional tumor resections resulted with no added neurological deterioration. While results varied, iMR enhanced neuronavigation also improved survival times over conventional neuronavigation. The only considerable intraoperative compromises were with respect to limiting patients to those without ferromagnetic implants, additional procedural time (noted time increases as much as 50% over conventional IGNS), the economic expense of the device to include in some instances considerable infrastructural costs, the encumbrance and upkeep of these devices, the special instrumentation needed, and the potential impact to surgical workflow. Nevertheless, an important finding across enhanced neuronavigation iMR studies, albeit sparse, is the agreement that taking into account soft-tissue anatomical changes intraoperatively provides quantitative additional benefit over conventional IGNS approaches. This provides considerable impetus for pursuing technologies to accommodate soft-tissue changes.

INTRAOPERATIVE CONVENTIONAL IMAGING

In order to understand the role of computational modeling towards accounting for soft tissue deformation in IGNS, it would be remiss to not briefly review imaging solutions as they represent the only commercially available options at this time. With respect to soft tissue deformation in frameless IGNS, as alluded to above, iMR technology was clearly focused at this problem and began almost two decades ago, e.g., Refs. ^{5, 72}. Many of the early investigations were concerned with the MR configuration, e.g. dual-bay configurations with dynamic patient beds, open configurations with long translating beds, or moving magnets with stationary beds. Commercially these approaches have matured such that medical centers with the resources can construct commercially driven iMR suites. While a compelling solution, for reasons already alluded to above, it has had limited adoption. It should also be mentioned that iMR investments by medical centers are largely in their first generation; studies regarding updating, and maintenance of these suites have not yet been forthcoming.

With iCT, some solutions were explored in the 1980s.⁴⁰ With the improvement in detectors since, investigation still continues²⁴ as well as some very interesting work using cone-beam CT (CBCT) technology in surgery.⁷⁹ iCT continues to lack soft-tissue contrast, and radiation concerns leave its role somewhat narrower at this time. However, highly mobile integrated solutions are available (e.g. Airo, Brainlab AG, Feldkirchen, Germany) and it is likely that this technology will play a role in neurosurgery in the future; but, the likelihood of widespread adoption and use in routine surgeries is still questionable.

Intraoperative ultrasound (iUS) for neurosurgical applications has been under continuous investigation since the earliest stages. While being the only conventional modality that is routinely used in many surgical/interventional applications, it has been of marked interest for neurosurgery. While a comprehensive accounting cannot be done here, a compelling review by one of the lead contributors was provided by Unsgaard et al. in Ref. 71 where they demonstrate the potential functionality of tracked intraoperative ultrasound for neurosurgical procedures. While guidelines are reported to generate quite exquisite image quality, adoption of these practices has been limited and this is largely due to the extent of workflow altering that ensues to ensure quality imaging. Nevertheless, efforts continue and include work towards soft-tissue deformation compensation, e.g., Ref. 55. While clearly iUS technologies will continue to become more adaptive to procedures, its reliance on as the sole source of feedback in neurosurgical procedures is not likely, but rather, its role could be as one source of data within the mechanics of building a comprehensive intraoperative updating system or as a source of complementary intraoperative imaging data.

The last conventional imaging modality to consider is the optical field. The surgical microscope has been an enormously valuable tool within the context of soft-tissue resection for neurosurgical applications. With respect to soft-tissue deformation in particular, quantitative measurements of tissue change in the optical field of view might be of particular value. Scope-enabled stereoscopic cameras (and unconventional laser range scanners) have been the most investigated sources of sparse optical data and have been used extensively to capture cortical surface deformations. With respect to the stereo method, the first proposed use within the neurosurgical context and brain shift was by Skrinjar et al.⁶⁶ and has been followed by others, e.g., Refs. ^{37, 69}. The advantage of using stereo pair brain surface reconstruction with the surgical microscope is the streamlined workflow whereby data can be collected with little interruption to the course of surgery. Other optical measurement methods such as laser range scanners, e.g., Ref. 44, have been explored to include comparative work between the two methods.³⁸ The data produced by these devices is typically a textured point cloud, i.e. a 3D point cloud capturing the surface of the brain which is then color-encoded with data from the visible field of view (FOV). Using non-rigid image alignment procedures in conjunction with these temporally sampled feature-rich textured point clouds, cortical shifts in 3D have been measured (first proposed in Ref. 64) and work continues, e.g., Ref. 31. While potentially providing highly resolved dynamic measurements of brain motion in a workflow friendly manner, the only drawback is the limited spatial extent of the data and that it only captures cortical changes in the FOV.

As can be observed throughout the above discussion, each form of intraoperative digitization seems to have advantages and disadvantages with respect to levels of information provided for correcting brain shift. Gantry-based technologies introduce significant cost and impact the patient environment while more workflow amenable technologies tend to produce incomplete data. Within this context, it is interesting to consider the role of computational models especially within the area of soft-tissue deformations for image guided surgery applications. In the past, computational modeling has been largely used for three functions: (1) to understand or provide analysis for data, (2) to provide an educational platform by simulation, or (3) to serve as a means to predict either before costly design or for feedback and control. Computational models can provide functionality among these for use in image guided surgery; however, in the realization discussed below, it is taking on a new role whereby models are serving as a constraining scaffold for sparse data which when of sufficient strength enable a functional purpose greater than the sum of contributing data. It is within this role that we discuss the use of models for the correction of soft tissue brain deformations due to surgery.

ROLE OF PATIENT-SPECIFIC COMPUTATIONAL MODELING TOWARDS BRAIN SHIFT

Computational techniques to nonrigidly register image data via modeling methods have a long precedent in the neurosurgical community. For example, elastic matching and related techniques have been employed by many to register multi-modality images.²⁵ Another example would be deformable templates for large deformation warping of images.¹² The first suggestion of subject-specific models to be used for accounting for soft tissue deformations within the context of stereotactic guided brain procedures appears in three independent early contributions from Edwards et al.,¹⁹ Miga et al.,⁴¹ and Miller and Chinzei.⁴⁶ It is interesting to look across these early works as each contributes to model realization differently and complementarily towards this new role. In Miller and Chinzei,⁴⁶ the investigators pursued classic quantitative mechanical compression in vitro tests of porcine tissue plugs and fit a viscoelastic constitutive model to their stress-vs-strain data. Their motivation was towards computer-assisted surgical applications within robotics. While an important contribution towards mechanical properties, the work does not speak to in vivo work, geometric constraints, or a realistic clinical environment. In the work by Miga et al.⁴¹ which was motivated by stereotactic neurosurgery techniques, the investigators use a 3D biphasic subject-specific finite element model of a porcine brain simulating intra-parenchymal loading conditions and compare directly to in vivo counterpart data with subsurface tissue targets provided by small stainless steel markers similar to those used in Ref. 35. The work offers in vivo conditions with realistic constraints and model geometry within a cranial environment but lacks human geometric specificity and extensive constitutive model testing. In Edwards et al.,¹⁹ a 2D finite element model from a human axial MR brain slice was created and employed a 3-component rigid/elastic-solid/fluid model for predicting the shift based on surface deformations, in this case provided by shifting intracranial electrodes. In this work, preoperative MR ‘normal’ anatomic images were modified to match a postoperative CT image set whereby shifting electrodes could be measured relative to the preoperative image set. The procedure was to use the electrode position changes as boundary conditions to the 2D model and then communicate deformation to the remaining MR imaging domain. While not offering a strong constitutive model or quantitative data as by Miller and Chinzei, or the 3D geometric effects and in vivo sub-surface data as by Miga et al., the work does offer an approach towards the use of cortical data to simulate volumetric shift and correct the image database within the context of a human anatomical configuration. Each individually is interesting; but taken together, one can see a quite complete picture of how computational models would move into this novel application role. More from the same investigators and several other investigative teams appeared quite rapidly in the literature and still continues. We also should note that early simulation work relevant to surgical conditions was already taking place too, e.g., Ref. 50.

Many of these early investigations were focused at the selection of modeling physics and their validation within an environment similar to surgical applications. The models largely reflected point-mass models (e.g., Ref. 65), elastic (e.g., Ref. 20), biphasic (e.g., Ref. 53), and viscoelastic (e.g., Ref. 45) realizations. Since these early publications, more work on methods of solution (e.g., Ref. 34), the use of linear and nonlinear models (e.g., Ref. 78), enhancements to constitutive equations (e.g., Ref. 48), and extensions to human systems for comparisons (e.g., Ref. 17, 63) evolved from that early work.

While these efforts towards predictive biomechanical modeling of neurosurgical applications are important, it is the integration with available intraoperative data that is the necessary ingredient for translating models to more clinically relevant roles associated with patient care. Interestingly and appropriately, the first codified alternative to the whole-brain imaging solution to the brain shift problem in IGNS comes from an investigator generally considered to be the ‘father’ of frameless stereotaxy.⁵⁸ Roberts et al. proposes an elegant methodology that integrates low-cost readily available intraoperative data, so called “sparse” data, into a framework that corrects for intraoperative deformations within the preoperative image database using biomechanical model-based techniques.⁵⁷ Briefly described, the framework involves the collection of readily available cortical surface information in the visible field, the incorporation of other forms of low-cost amenable intraoperative imaging data such as ultrasound, and the constraining of brain shift predictions using a physics-based model. While the work itself does not provide a closed loop feedback strategy per se, the components and the suggestion as a direction for the field is provided quite succinctly.

While no commercial model-based correction strategy is currently approved within the neurosurgical guidance environment, the field has moved forward with several realizations that are very promising. As suggested herein that computational models are taking a new role as a constraining scaffold for sparse data integration, the variations among approaches have been driven largely by the types of data used in these novel correction pipelines. The most straight forward approach is to measure deformations of the cortical surface (as suggested in Edwards et al.¹⁹) using optical methods and possibly subsurface features using intra-operative imaging technology and simply drive a predictive model in a somewhat direct configuration. With respect to model prediction specifically, this would be the equivalent of applying 3D displacements to a geometric grid and propagated by some constitutive physics. Several investigations reflect this approach with the first comprehensive framework provided by Skrinjar et al.⁶⁵ In, Ref. 65 Skrinjar et al. describe a lumped mass approach where the dynamics reflect a mass-spring-damper like reaction. Displacements acquired during surgery through optical imaging are directly applied to the model and a correction is imparted. The solutions reflect a temporally evolving calculation with a brain-skull interaction condition. DeLorenzo et al. expanded upon the work using a similar model but adopting a more automated approach to estimate cortical surface deformation using information game theory which allowed both camera calibration errors and cortical surface deformations to simultaneously be addressed during the deformation tracking.^15,16 Similar approaches with respect to data integration have been proposed using continuum models of elastic and nonlinear viscoelastic models and have moved this direction forward.^77,82,83 In Wittek et al.,⁷⁷ investigators demonstrate dramatic increases in computational speed for nonlinear models with results reflecting good deep structure alignment over several surgeries with a subsequent study showing similar results over many retrospective patient data sets.²³

Interestingly, these cortical surface direct displacement driven methods were quickly expanded upon when iMR data became available at medical centers, i.e. the recognition that intermediate registration of high resolution preop MR to incomplete iMR data was still quite necessary. The work by Ferrant et al. is an excellent example whereby they used an elastic finite element model in conjunction with iMR data to predict volumetric shift.²⁰ With respect to the above correction work driven by cortical surface data only, the iMR allowed the tracking of ventricle movements to be added as internal boundary conditions to the model correction framework which improved the fidelity of their registrations. These techniques continue to mature and are characterized by continued use of the data-rich intraoperative imaging environment to drive calculations, e.g., Ref. 3. In these new realizations, while underpinned with biomechanical models to constrain deformation realism, volumetric image-derived forces and intensity matching conditions have become commonplace to assist in the registration.

Given the adoption challenges associated with iMR-based approaches, investigators continued to look to other low-cost readily available intraoperative data as a driving source for correction and in so doing relied on novel algorithmic developments utilizing significantly sparser data. One strategy developed by Lunn et al. was to reconstruct model-based boundary conditions iteratively using an adjoint formulation to best fit cortical surface and subsurface data.³⁹ While this methodology continues to advance, one aspect that does distinguish it from the work above is, while driven by surface and subsurface data, the investigators limited themselves to internal ultrasound acquired tumor contour data, a source of data that is quite sparse in practical neurosurgical procedures. While an interesting approach, other challenges do remain. Just as technology integration has changed dramatically, in today’s neurosurgical procedures, the extent of craniotomies is changing with visible cortical surfaces becoming smaller and perhaps not as amenable to frequent ultrasound probe use. Despite these changes to operative characteristics, there is also a considerable effort by investigators towards a complete ultrasound-guided solution, typically reflects vessel-based registration approaches with nonrigid effects supplied by more nonphysical interpolative methods.⁵⁴ Initial results have been promising and it is exciting to speculate the continuing role of ultrasound within neurosurgical applications.

The last framework to be discussed addresses the problem uniquely in three manners: (1) the approach does not focus on direct boundary condition application, or the reconstruction of direct boundary condition distributions, (2) limits the extent of driving data to the visible cortical surface displacements only, i.e. most restricted in sparsity but most amenable to workflow, and (3) uses precomputing to improve speed and flexibility. The approach was inspired by frameworks in registration using predictive modeling as a tool to characterize shape differences for segmentation purposes¹⁴ and was reported by Dumpuri et al. in Ref. 18 and extended by Chen et al. in Ref. 8. Briefly described, based on a minimum of planning variables that include approximate craniotomy size, resection approach to tumor, and head orientation, Dumpuri et al. in Ref. 18 describes an algorithm to generate a distribution of orientation-sensitive boundary conditions that can simulate gravitational sag,⁴² hyperosmotic drug interaction, swelling, and resection cavity effects. These sets of driving conditions are used pre-operatively to compute a suite of solutions, a so-called ‘deformation atlas’. Intraoperatively, a combination of the solutions is generated to best fit cortical surface deformation measurements as provided through laser range scanning or stereo pair. This work has been realized into a registration pipeline⁶⁸ and initial validation work has been completed.^17,63 Interesting extensions incorporating subsurface structures arising from the dural septa and aspects suggesting a more considerable role for hyperosmotic drugs than previously reported has been forthcoming.⁸ The focus of this work has been somewhat different whereby it attempts to model the mechanisms of shift (sag, swelling, hyperosmotic-drugs, and debulking) rather than applying surface-based boundary conditions to simulate these mechanisms directly. In addition, the concept of generating an atlas is intriguing because it allows for patient changes in the OR to be captured by a distribution of possible solutions (e.g. change in patient orientation by rotation of the bed, or extent of cerebrospinal fluid drainage when simulating sag conditions) rather than needing measurement technologies per se. Lastly, the atlas resticts the solution thus constraining solution behavior while allowing for very fast calculation speed. While an interesting approach, it does require additional preoperative planning, and will have difficulty with conditions outside the distribution of boundary conditions that generate the atlas.

The above realizations toward solving the “brain shift” problem represent some of the first efforts to correct the guidance and delivery of therapy through mathematical models rather than pursuing infrastructural enhancement (e.g. installation of large scale imaging devices and equipment). To date, the results of many of these studies are promising and have reflected an ability to compensate for the majority of bulk brain shift with accuracy dependent on the extent of shift (typically about 70% of shift extent can be compensated although performance does vary). However, we should note that these examples do reflect significant levels of computation occurring within the course of a procedure which could have future implications for intraoperative analysis. While in the late 1990s, these implications were largely seen as negative with the community believing models were too slow to be practical for the surgical theater. Those concerns have been largely mitigated by computing hardware in the last decade. For example, improved speed of computation with recent implementations on graphical processing units³³ has shifted calculations from minutes to seconds. As is a common experience by investigators in the field today, the times associated with model-compensation computations have become secondary to other workflow aspects within the operating room.⁶⁸ However, it important to recognize that there are still hurdles to be overcome with the patient-specific modeling approach. These models often require significant preoperative processing (e.g. segmentation, mesh generation, and in the case of Ref. 68 model pre-computations). Automation such that highly skilled labor is not needed can be difficult to achieve also. Strides have been made in areas of segmentation (e.g. Refs. ^{59, 81}), alternative numerical methods (e.g., Refs. ^{47, 74}), and reconstructive approaches (e.g., Refs. ^{10, 32}). However, these will remain important challenges to be addressed as we consider frameworks in the future. In addition, we must realize that before any of these capabilities can be realized, important aspects of validation must be addressed.

MODEL CONSIDERATION AND VALIDATION

In order to adopt these frameworks, considerations regarding the appropriate model and validation methods must be thoughtfully reviewed and often require careful design. For example, even among the first “brain shift” modeling reports in Refs. ^{19, 41, 46} we see significant model sophistication differences that span lumped, linear, and nonlinear, both static and temporally dynamic differential equation formulations. We also see different initial validation strategies that span highly instrumented and controlled ex vivo material testing, controlled in vivo implanted marker tracking with intraprocedural imaging, and clinical imaging with somewhat indirect measurements. Adopting modeling and testing environments to best validate systems that are relevant to the most important context, patient care and outcome, is extremely challenging.

For example, with respect to the selection of appropriate soft-tissue models, there is clear consensus that the constitutive nature of tissue is highly complex. Unlike conventional materials, soft tissue has intrinsic properties that include self-adaptation and self-repair, age and hydration related dependencies, highly heterogeneous, multiphysics in nature, often, anisotropic, and viscoelastic, to name several among many more. When one combines the above intrinsic complexity with the many approaches to representing and solving computational models, the difficulty in blending multiple length scale effects, and the variability induced by the testing environments themselves, there will be inherently model-data misfit. When we consider this environment of development for the field and within the context of clinical goals for correction during surgery, the models adopted by investigators has certainly not been uniform and to a degree has been somewhat controversial.

With respect to the testing environments, equally perplexing is the use of inexact measurement techniques for the purpose of validation. While iMR is the only commercial technique used for brain deformation compensation and is generally considered a gold standard for human-based validation, in its routine practical use, the quality of images make its reliance upon as a comparator for validation of model-based registration approaches somewhat compromised. More specifically, as iMR is often used as a sparse data source to register high resolution preoperative images to the intraoperative environment, this is subject to error. If special care is not taken in the acquisition of iMR data, it is likely that the veracity of the results will need to be qualified by the resolution of data. These are important considerations for the field and have yet to be resolved.

BRAIN SHIFT AND BEYOND

While challenges remain, the future of computational modeling within surgical and interventional processes is bright. Even with the considerable strides made with the brain deformation compensation work above, we should also not forget that teams have been working on other effects such as retraction and resection.^43,75 We also have seen some exciting work in intraoperative imaging and guidance instrumentation for real-time brain tumor margin detection to include Raman spectroscopy, fluorescence, and mass spectrometry.^4,30,61,73 Robotic platforms are also on the horizon.⁷⁰ In some respects, we may be seeing early glimpses of the next evolution in intraoperative neurosurgical systems—robotic master–slave guidance platforms coupled to better predictive models with intraoperative sensing end effectors. While several decades ago, this may have seemed science fiction, we are there now.

Equally encouraging is the propagation of image guided surgery techniques to extracranial environments. While commercial platforms for spinal surgery already exist (e.g. BrainLab AG, Feldkirchen, Germany & Medtronic Plc, Minneapolis, Minnesota, USA) to include robotic platforms (e.g. MAZOR Robotics Inc., Orlando, Florida, USA), within the research community, we are seeing the appearance of other soft-tissue surgical guidance frameworks. Emerging work on image guided liver and prostate surgery from several groups is underway to include approaches to nonrigid registration to account for surgical deformations reflecting the use of biomechanical models.^2,60 Very preliminary efforts towards kidney,¹ lung,⁶ and breast¹³ are appearing that also incorporate biomechanical models as part of their guidance frameworks. Computational models in surgery are also being extended past deformation mechanisms into other areas of mechanics associated with therapeutics. For example, thermal and ionic ablative techniques to include radio frequency,⁹ microwave,¹¹ cryoablation,³⁶ and irreversible electroporation²² are employing models in design and prediction of therapies. In the area of biotransport, mechanics based models are emerging for estimating convection-enhanced drug delivery within soft tissue.⁶⁷ Some very novel extensions within different interventional domains looking at predictive outcomes in the application of neoadjuvant chemotherapy, and radiation therapy toward tumor growth mechanics have been forthcoming as well.^29,80 Lastly, we would be remiss not to mention the incredible strides in patient specific modeling for electrophysiological purposes such as deep brain stimulation.⁷

These are parting allusions to a growing rich literature and are but a small sample of the possibilities opening up with fast computing and computational models within the role of patient-specific precision surgery and intervention. While the diagnostic information of patients has expanded dramatically with modern medical imaging, and continues to grow with contemporary informatics research, the advancement of procedural integration to surgery has lagged; but this review paints a picture towards that rectification. As we move forward, we will move away from the current paradigm of technologies being injected to advance capabilities in focused problematic areas of surgical execution while neglecting workflow impact. More specifically, this review suggests that surgical procedures of the future will not merely involve the flow of diagnostic information to standardized technology-enhanced operating rooms. Rather, diagnostic information will initiate a process whereby procedure-specific, and patient-specific modular technologies and computational approaches are assembled into novel, perhaps never before been realized, systems that optimize therapy delivery. Similarly, as these more customized and reconfigurable approaches to treatment evolve, patient procedural teams will also evolve with physicians being joined by engineers, scientists, informaticists, and statisticians to help him or her facilitate treatment options. In many medical centers, these team approaches are already beginning; but, it is very exciting to speculate what these may become in the era of incredible advances in device design, and computational capabilities.