A Common-Ground Review of the Potential for Machine Learning Approaches in Electrocardiographic Imaging Based on Probabilistic Graphical Models

Abstract

Machine learning (ML) methods have seen an explosion in their development and application. They are increasingly being used in many different fields with considerable success. However, although the interest is growing, their impact in the field of electrocardiographic imaging (ECGI) remains limited. One of the main reasons that ML has yet to become more prevalent in ECGI is that the published literature is scattered and there is no common ground description and comparison of these methods in an ML framework.

Here we address this limitation with a review of ECGI methods from the perspective of ML. We will use probabilistic modeling to provide a common ground framework to compare different methods and well known approaches. Finally, we will discuss which approaches have been used to do inference on these models and which alternatives could be utilized as the methods in ML become more mature.

1. Introduction

Electrocardiographic Imaging (ECGI) attempts to image the electrical activity in the heart from the electrical recordings on the body surface and a model of their relationship. This is a challenging problem due to the ill-posedness of the inverse problem being solved [1] and, thus, all methods have to introduce prior knowledge as assumptions about the solution, the noise and/or the model to help stabilize the inverse. A classical method to solve this problem and the reference for most approaches is Tikhonov regularization [2], which solves Equation 1:

$$\underset{x}{\text{min}}{\Vert y-\mathit{Ax}\Vert }_2^2+\lambda {\Vert x\Vert }_2^2$$ (1)

where x is a vector of heart surface potentials at a chosen set of nodes, y is a corresponding vector of body surface potentials, and the matrix A encodes a solution to the forward problem that relates x to y. For simplicity here we assume that are are only considering a single time sample here.

The balance between the residual ${\Vert y-\mathit{Ax}\Vert }_2^2$ and the regularization term $\lambda {\Vert x\Vert }_2^2$ —which enforces the assumed prior knowledge— in Equation 1 is typical for inverse solvers. Other regularization approaches modify the regularization term, introduce hard constraints to the objective function or project the solutions onto a “valid” set.

The field of Machine Learning (ML) has seen an explosion in the number of publications and corresponding technical development and are increasingly being used in many fields with considerable success. In particular, Bayesian ML approaches allow us to model the data through the dependence relationships between random variables (RVs) and their probability distributions. These tools could help researchers in ECGI to properly model their assumptions about the problem and easily test them. However, although the interest is growing, the impact of ML in the field remains limited. One of the main reasons that ML has yet to become more prevalent in ECGI is that the published literature is scattered and there is no common ground description and comparison of these methods.

Here, we describe both classic and newer ECGI methods through the lens of ML. We use the parallel between classical optimization methods and Maximum-A-Posteriori (MAP) inference to derive the assumed probabilistic models in the ECGI methods. This framework will elucidate the inherent modeling assumptions, allow for comparison across methods, and easily incorporate different types of assumed prior knowledge. Finally, we will provide a quick glance at some inference approaches that can be applied to the models described.¹

2. Theoretical background

The probabilistic model describing the most basic ECGI method is the following:

$$\begin{gathered} p(x)=\mathcal{N}(x\mid 0,\rho ) \\ p(y\mid x)=\mathcal{N}(\mathit{Ax}\mid \sigma ), \end{gathered}$$ (2)

where $\mathcal{N}(\cdot )$ is a Gaussian distribution and σ and ρ are known scalar variances. The objective function in Equation 1 is then equivalent to doing inference on this model with a maximum a posteriori (MAP) objective, with $\lambda =\frac{\sigma }{\rho }$. This is a very simple probabilistic model as it only includes the observed data y and the latent variable x and assumes known variances and forward matrix. However, it is the building block from which we will develop the rest of models we describe. We will do so by adding new assumptions about the relevant RVs as well as introducing assumed prior physiological knowledge about them.

3. Modeling Approaches

Even though the potentials on the heart have a strong spatial structure, Tikhonov regularization does not assume any spatial correlation across the heart. Exploiting this characteristic has been the common theme of many new ECGI methods in the literature. The simplest models are the higher order versions of Tikhonov, which introduce a fixed correlation structure between nearby nodes through a regularization matrix R. In those models, the prior used to describe the potential distribution is a multivariate Gaussian with covariance matrix given by Σ = R⁻¹². Tikhonov 2^nd order uses the Laplacian operator of the heart geometry to encourage higher correlation between neighboring nodes on the heart geometry [2, 3]. Changing the covariance matrix modifies the type of “ground truth” being enforced. For example, some groups have modified the Laplacian operator to enforce correlation between nodes on both sides of the myocardial wall [4–6] or introduced varying weights on the covariance matrix to favor more uncertainty in nodes on the epicardium than on the endocardium [7]. The previous methods assume known covariance matrices —i.e. regularization matrices— however, these can also be learned from the data. The covariance parameters can be treated as RVs and thus be learned jointly with the unknown heart potentials [8,9] or derived a priori from simulations or ex-vivo recordings [10].

Spatial modeling is not limited only to employing different covariance matrices with a Gaussian distribution. It is also possible to change the base probability distribution used. For example, it is possible to use the Laplace distribution, which is equivalent to an L1 norm constraint of the solution and used to favor the presence of a small number of large values. Total Variation uses this same distribution —with a covariance matrix equal to the inverse gradient operator— to allow for a small number of large gradients on the heart potentials [11–15]. Other groups have extended this idea to induce sparsity with generalized Gaussian distributions —equivalent to the Lp norm in classical optimization [16,17].

The heart potentials also have a strong temporal structure that is often leveraged to add prior knowledge. The simplest approach is to extend the dimensions of the latent variable x to include time and define the covariance matrix to introduce temporal structure. For example, stacking the time instances on top of each other into a block vector and employing a spatio-temporal covariance matrix with a block diagonal form assumes independence between time instances. Any non-zero entry outside of the main block diagonal introduces correlation in time. For example, if two regularization matrices R and T —in space and time—are used, the inverse covariance has the spatial regularization R repeated along the diagonal and the temporal regularization T repeating in off-diagonal positions [18]. Greensite and Huiskamp [19] introduced an optimal structure based on assumptions of separability of temporal and spatial correlations; for a comparative study of three such methods see [20]. However, the most common approach to model temporal behavior is to use deterministic restrictions of the solution space. This is done via projection of the data and solutions onto a feasible set [4,21–25] or through explicit constraints in the optimization [26]. These temporal restrictions correspond to setting probability zero to the unfeasible space and creating truncated probability distributions.

An even more restrictive approach is to assume a specific temporal shape for the potentials. Activation-based methods do this, solving for unknown activation times on each node of the heart [27]. This restriction can be refined with more complex propagation models for the heart potentials. These models are equivalent to Hidden Markov probabilistic models, typically of order 1 [28]. The complexity of the model can vary from a simple curve defining the wave-front [29] to a full propagation models [12].

The models described so far only introduce prior knowledge about the heart potentials. It is possible, however, to also characterize the uncertainty introduced by other parameters. For example, a set of parameters that is often assumed fixed but that also introduces uncertainty is contained in the forward solutions. Miss-specifications of the geometry such as position of the heart or conductivities of the organs introduce error into the forward model and thus the inverse solutions. The literature addressing this problem is still small, but there has been some work trying to characterize the uncertainty introduced by these model errors [30–32] and, more recently to correct for those from the data [33–35].

Other parameters that can be further characterized as an RV are those that describe properties of the physiological propagation models. These models invariably depend on underlying parameters, typically chosen from “healthy” priors, post-mortem hearts or animal models. This RV can be set a priori [36,37], learned outisde of the inverse solution with discriminative methods or learned jointly with the heart potentials [38–40].

It is important to note that the extension of these models with new variables increases the complexity of the model. Deep probabilistic models usually demand more resources to compute and, more importantly, more data for inference. The curse of dimensionality limits the number of parameters added and they should only be introduced if enough prior knowledge exists about them to learn them correctly with reasonable probability.

4. Inference

So far we have used the equivalence between classical optimization and MAP inference to develop a probabilistic framework for ECGI. A key advantage of translating standard classical optimization problems to probabilistic models is that it is then possible to test different inference methods from the ML literature. Variational Inference (VI), Markov Chain Monte Carlo (MCMC) methods or, more recently, disentanglement approaches in the deep learning literature, all provide estimates of the full posterior distribution and not just its mode or mean. Thus, one can include an estimate of the uncertainty of the results [12,28,31,40–42].

One significant limitation of these methods is that they require advanced knowledge in ML to design and code. Fortunately, there has been a considerable recent effort in ML research to develop probabilistic programming languages that offer readily usable black-box variational inference and MCMC algorithms [43]. With these tools, it is possible to describe the probabilistic model directly in a tailored programming language and then learn the posterior distributions without the limitation of having to develop the entire inference structure.

5. Conclusions

We have described some relevant methods in the ECGI literature in the context of probabilistic modeling. This framework helps contextualize efforts from different groups and may allow further development with incorporation of new prior knowledge about the problem. Finally, we have suggested how some inference techniques in the ML literature may facilitate the adoption of advanced ML methods in ECGI.