Using UncertainSCI to Quantify Uncertainty in Cardiac Simulations

Abstract

Cardiac simulations have become increasingly accurate at representing physiological processes. However, simulations often fail to capture the impact of parameter uncertainty in predictions. Uncertainty quantification (UQ) is a set of techniques that captures variability in simulation output based on model assumptions. Although many UQ methods exist, practical implementation can be challenging. We created UncertainSCI, a UQ framework that uses polynomial chaos (PC) expansion to model the forward stochastic error in simulations parameterized with random variables. UncertainSCI uses non-intrusive methods that parsimoniously explores parameter space. The result is an efficient, stable, and accurate PC emulator that can be analyzed to compute output statistics. We created a Python API to run UncertainSCI, minimizing user inputs needed to guide the UQ process. We have implemented UncertainSCI to: (1) quantify the sensitivity of computed torso potentials using the boundary element method to uncertainty in the heart position, and (2) quantify the sensitivity of computed torso potentials using the finite element method to uncertainty in the conductivities of biological tissues. With UncertainSCI, it is possible to evaluate the robustness of simulations to parameter uncertainty and establish realistic expectations on the accuracy of the model results and the clinical guidance they can provide.

1. Introduction

Computational electrocardiographic models and simulations include many input parameters, such as the conductivities of the tissues, geometry of the organs and tissues, and their relative positions [1, 2]. Inherent in each model parameter are errors associated with the practical necessity to simplify model assumptions and details. Such errors can arise from natural variations over patient populations, simplification or abstraction of the geometry or underlying mechanisms, as well as a lack of precision when obtaining the parameters. Quantification and control of these errors through uncertainty quantification (UQ) provide sensitivity information, which is a critical component when evaluating the relative impact of parameter variation on the solution accuracy. [3,4]

Many methods exist for simulation-based assessment of sensitivity, and the implementation depends on the complexity of the underlying system. Monte Carlo, perhaps the best known and simplest approach, samples parameter space in order to explore the impact of parametric variation; however, this approach sometimes requires millions of simulations to achieve a forward UQ prediction of acceptable accuracy, and hence this strategy frequently has a prohibitive computational cost. Two simplifications of this approach are (1) brute-force methods, which under-sample the parameter space and interpolate between the resulting values; and (2) range-finding experiments, which evaluate the outer extremes of the parameter spaces, which are assumed to represent the total variation. The major weakness of the latter approaches is that their underlying assumptions of linearity is not always correct.[5] In contrast to the approaches described above, the generalized polynomial chaos-stochastic collocation (gPC-SC) method is an effective sampling method that exploits assumptions concerning the mathematical nature of the stochastic field or process of interest–—assumptions that are often justified mathematically and practically–—to minimize the number of samples that are needed for the computation of accurate statistics.[6]

There are existing software toolboxes that perform UQ tasks.[7, 8] However, these toolboxes are created specifically either as standalone packages or customized for nonbioelectric field applications. In order to use these tools for biomedical applications, the simulation environment needs to be coupled via a computer script. Such high-level scripting results in computational speed penalties and makes validation challenging as algorithms are built by third-party developers. Furthermore, current toolboxes often have limited capabilities and are missing the benefits of more modern and effective UQ techniques.

We developed UncertainSCI, a software that tightly couples recently matured UQ algorithms and paradigms by means of a noninvasive Python interface, resulting in a nimble, efficient, and robust set of UQ tools that are specialized for biolectric field simulations. To evaluate the implementation of UncertainSCI, we have applied the software to (1) quantify the variability in computed torso potentials using the boundary element method (BEM) in response to uncertainty in the heart position and orientation via MATLAB, and (2) quantify the variability in computed torso potentials using the finite element method (FEM) in response to uncertainty in the conductivities of each biological tissue via our custom, open-source problem solving environment, SCIRun.[9] These two implementations produced similar results to what was reported in our previous studies using gPC-SC, which did not utilize UncertainSCI.[5,10]

2. Methods

Sensitivity Quantification through gPC-SG:

We quantify forward parametric uncertainty in cardiac simulations using polynomial Chaos expansions (PCE). [6] Such an approach attempts to approximate the dependence of a quantity of interest (QoI) on a finite number of random parameters via a multivariate polynomial function of those parameters. This approximation is an emulator for the associated forward problem. Once this approximation is constructed, then statistics of the QoI, including the mean, variance, and parameter sensitivities, are computed via straightforward, computationally efficient manipulations of the polynomial. One non-intrusive strategy to construct this polynomial dependence is through least-squares approximation, where data for the least-squares problem is collected through an ensemble of simulations of the forward model. UncertaintSCI uses PCE methods to quantify forward uncertainty, and employs a particular experimental design on which to collect simulation data.

For a fixed number of data points, the stability and accuracy of the PCE emulator is known to depend on the experimental design. UncertainSCI constructs this design through the procedure of Weighted Approximate Fekete Points (WAFP), which computes a geometrically unstructured experiment as a special type of D-optimal (determinant-maximizing) design. More precisely, the design is computed through a greedy algorithm that iteratively adds parametric samples that maximize a matrix determinant. The maximization is computed over a discrete candidate set; generating this candidate set via Monte Carlo sampling from the distribution of the parameter is known to produce suboptimal sets. UncertainSCI computes the candidate set by sampling from the induced distribution, which exploits a concentration of measurable phenomena to provably increase the quality of the candidate set. Sampling from the induced distribution for independent parameters is computationally efficient, having complexity that is linear in the dimension (number of parameters).

Once the experimental design is created through the WAFP procedure with induced distribution sampling, an ensemble of forward simulations is collected from the simulation software, and UncertainSCI produces a PCE emulator through a (weighted) least-squares procedure. From this least-squares procedure, UncertainSCI also can compute residuals and cross-validation metrics, and can adaptively tune the expressivity of the PCE emulator based on a user-prescribed tolerance and/or computational budget.

Applications:

In order to assess the feasibility, utility, and performance of UncertainSCI in the context of biomedical simulation, we performed uncertainty quantification with two classic cardiac bioelectric forward models. The first example used UncertainSCI to quantify the effect of variability in cardiac position on the electrocardiographic potentials, the ECG’s. This approach was based on a previous study by Swenson et al..[5] Briefly, we parameterized the motion of the heart into six parameters according to the cardiac position scheme implemented by Coll-Font et al.:[11] (1) translation left and right, (2) translation front to back, (3) translation up and down, (4) pitch side to side, (5) yaw of the septal axis, and (6) rotation along the septal axis. We then utilized data and geometries collected from isolated-heart torso-tank experiments, and constructed an objective function in Python that took the six motion parameters, as well as their bounds and inputs, and called on a MATLAB library to move the cardiac geometry, calculate the bioelectric potentials on the tank surface using the BEM, and return the resulting time signals.

The second example was also based on research in cardiac forward problems, and relied on varying the conductivities of tissue using a finite element approach to solve for torso potentials, as described by Geneser et al..[10] Using data from similar experiments to the first study but from the intact thorax, we segmented the heart, torso, and lung volumes from MRI scans using Seg3D (www.seg3d.org). We then constructed a volume mesh and used the FEM to estimate torso potentials with SCIRun. Access to UncertainSCI came through a Python script that used SCIRun’s Python interface to modify the conductivity values of the lungs and torso, and provided the resulting torso surface potentials as the inputs to UncertainSCI.

3. Results

First, we present the results of the sensitivity analysis on variability in heart position on torso potentials (Figure 1). The sensitivity map for swinging left to right, yaw of the septal axis and rotation along the septal axis show large regions of variability, with maximum sensitivity values of 0.82, 0.52, and 0.60, respectively. Such motion could occur when the heart shifts during the respiratory cycle.[5,11] Translation of the heart, more typical of a patient turning in bed, produced smaller regions of variability, with maximum sensitivity values of 0.28, 0.37, 0.43, for translation left and right, translation front and back, and translation up and down, respectively. Second, we present the results of the sensitivity analysis on variability in conductivities (Figure 2). The sensitivity maps for the lung conductivities and torso conductivities show large sensitivity coefficients overall, with a minimum value of 0.46 and a maximum value of 0.97. The maximal sensitivity values are positioned on the lower region of the torso surface.

Figure 1.

Effect of the heart position on torso surface potentials. Sensitivity maps using bounds described previously (top left to right), of translation left and right, translation front and back, and translation up and down, respectively, and (bottom left to right), swinging left to right, yaw of the septal axis, and rotation along the septal axis, respectively.

Figure 2.

Effect of the conductivities on torso surface potentials. Sensitivity maps using bounds described previously, for torso conductivity (left) and lung conductivity (right).

4. Discussion

The implementation and execution of bioelectric simulations requires a number of assumptions for input parameters, and understanding the effect of these assumptions is critical when assessing the resulting model outputs. While there exist a range of methods for assessing such model responses, e.g., brute force, Monte Carlo, and range-finding methods, the emergence of statistical methods such as polynomial chaos expansion allows for a more robust assessment of uncertainty, while mitigating the high computational cost due to parsimonious sampling techniques. However, implementations of these UQ methods have their own challenges. UncertainSCI bridges the gap between these advanced UQ methods and bioelectric simulation and modeling software by packaging UQ implementations in a noninvasive Python toolkit that wraps the necessary techniques and facilitates testing of various models. Because of the noninvasive nature of UncertainSCI’s UQ implementation, it is able to interface with a variety of software packages and does not require labor-intensive custom implementations for each different modeling environment.

To demonstrate the utility of UncertainSCI, we performed two UQ analyses based on previously published research from our group [5, 10]. The first analysis was on variability in BEM solutions with changes in cardiac position, implemented in MATLAB. We were able to leverage the existing MATLAB-Python API to allow for easy wrapping of the MATLAB forward BEM model such that UncertainSCI could easily and automatically drive the sampling of the model. We were able to replicate previous findings that rotation of the heart resulted in larger sensitivity in the forward solution output.[5] In the second analysis, we implemented a forward model using the FEM in SCIRun, and assessed the response of computed torso potentials to variability in conductivity of the torso and the lungs. Again due to an existing interface between SCIRun and Python, we were able to easily create a wrapping script to run our forward models and return the results to UncertainSCI, without having to modify the modeling environment. This approach represents an substantial extension to the original research, which examined the forward model in only a 2D slice of tissue.[10] This extension into three dimensions was enabled by the efficient and validated algorithms in SCIRun along with the flexible interface in UncertainSCI. The results show a complex pattern of sensitivity in response to torso and lung conductivity changes that was not apparent from the original study. These two examples demonstrate the ability of UncertainSCI to enable UQ analysis in any modeling environment that is Python compatible.

UncertainSCI offers a highly efficient, portable implementation of UQ to a broad range of bioelectric simulation applications. In such problems, there is a pressing and increasing need to understand the uncertainty stemming from uncontrolled variability in model parameters. UQ enables the evaluation of deviations from average physiological conditions, informing researchers about the need for more accurate measurements or refinement of the underlying mechanisms implemented in the model system. With UQ support, simulation scientists, engineers, and physicians will be able to make innovative use of computer modeling capabilities that provide the statistical confidence necessary to apply them to patients.