Diffusion Models To Predict 3D Late Mechanical Activation From Sparse 2D Cardiac MRIs

Abstract

Identifying regions of late mechanical activation (LMA) of the left ventricular (LV) myocardium is critical in determining the optimal pacing site for cardiac resynchronization therapy in patients with heart failure. Several deep learning-based approaches have been developed to predict 3D LMA maps of LV myocardium from a stack of sparse 2D cardiac magnetic resonance imaging (MRIs). However, these models often loosely consider the geometric shape structure of the myocardium. This makes the reconstructed activation maps suboptimal; hence leading to a reduced accuracy of predicting the late activating regions of hearts. In this paper, we propose to use shape-constrained diffusion models to better reconstruct a 3D LMA map, given a limited number of 2D cardiac MRI slices. In contrast to previous methods that primarily rely on spatial correlations of image intensities for 3D reconstruction, our model leverages object shape as priors learned from the training data to guide the reconstruction process. To achieve this, we develop a joint learning network that simultaneously learns a mean shape under deformation models. Each reconstructed image is then considered as a deformed variant of the mean shape.

To validate the performance of our model, we train and test the proposed framework on a publicly available mesh dataset of 3D myocardium and compare it with state-of-the-art deep learning-based reconstruction models. Experimental results show that our model achieves superior performance in reconstructing the 3D LMA maps as compared to the state-of-the-art models.

1. Introduction

Cardiac resynchronization therapy (CRT) is widely used to treat cardiac conduction system disorders, such as left bundle branch block (LBBB) and intrinsic myocardial diseases (Abraham et al., 2002; Lindenfeld et al., 2007; Moss et al., 2009). However, standard CRT faces a substantial challenge with a high non-response rate, approximately around 40% (Chung et al., 2008; Exner et al., 2012). Implanting the CRT LV lead at an area with delayed activation may significantly decrease the non-response rate (Bilchick et al., 2014; Ramachandran et al., 2015). Cine displacement encoding with stimulated echoes (DENSE) MR imaging is an accurate and reproducible method for imaging regional myocardial displacement and strain (Aletras et al., 1999; Kim et al., 2004). Studies have shown that LMA can be effectively measured on circumferential strains of tissue displacements imaged by DENSE (Bilchick et al., 2014; Ramachandran et al., 2015). The LMA in heart failure patients can be detected by the Time to the Onset of circumferential Shortening (TOS) (Kvåle et al., 2019; Xing et al., 2023, 2021). The 3D LMA map is then obtained by projecting the TOS values to a reconstructed 3D volume of the myocardium from a stack of sparse 2D slices. Despite the success of reconstructing the 3D LMA maps, current approaches (Kim et al., 2004; Kvåle et al., 2019; Xing et al., 2023, 2021) do not achieve satisfactory results since they rarely consider the shape variability among patients’ hearts, resulting in unstable and poor performance of locating the late activating regions from reconstructed 3D LMA maps.

To address this challenge, recent advancements have leveraged deep learning techniques to substantially improve the performance of image reconstruction from a limited number of 2D slices (Cetin et al., 2023; Goodfellow et al., 2020; Maaløe et al., 2019). Extensive research has explored various network architectures for 3D image reconstruction, including UNets (Nguyen et al., 2021), transformers (Feng et al., 2021; Korkmaz et al., 2022), and state-of-theart generative diffusion models (Waibel et al., 2023). These works have significantly improved the reconstruction efficiency by learning intricate mappings between stacks of 2D images and their corresponding 3D volumes. While the deep learning-based approaches have achieved impressive results in reconstructing detailed 3D images, they often lack explicit consideration of shape information during the learning process. Consequently, important geometric structures of objects depicted in the images may not be well preserved. This may lead to the occurrence of artifacts, such as discontinuities, holes, or mismatched connections between different parts, that break the topology of the reconstructed objects.

In this paper, we propose to utilize a recently developed work, shape-aware diffusion model (Jayakumar et al., 2023), to reconstruct 3D volumes of the myocardium from sparse 2D slices. We will introduce a jointly trained atlas-building network to determine the atlas (Wang and Zhang, 2022) (i.e., the mean image), along with a stack of sparse 2D slices, used as the input prior to train the diffusion model. In this framework, the predicted 3D volume is considered to be a deformed variant of the myocardium atlas. This reconstructed 3D volume can then be used to interpolate the predicted TOS values to obtain the cardiac mechanical activation maps. Due to the lack of 3D ground truth cine DENSE cardiac MRI scans, we train and validate our model on a public dataset of 3D myocardium mesh data and evaluate the results of the TOS prediction and LMA detection on the clinical test data. Our method is generally applicable to similar clinical applications, where collecting a complete 3D volume with high resolution is challenging. For example, reconstructing intra-operative 3D brain MRIs in real-time during brain surgeries.

2. Background

This section provides a brief background on the computation of strain matrices used as input to the neural network and an unbiased atlas-building framework trained to estimate the Fréchet mean of group-wise images.

2.1. Quantification of Late Mechanical Activation Through Strain Analysis

Circumferential strain has been demonstrated to depict myocardial contraction along the circular out-line in the short axis, typically with a negative value (Budge et al., 2012). There are several advantages of analyzing strain values as it significantly reduces the computational complexity of the network input, shifting from vector-valued displacement data $V$ to real-valued strain images represented as functions $\left(S\right)$. Also, it provides more robustness by mitigating the impact of body motion artifacts commonly encountered in the original signals and it improves our ability to quantify dyssynchrony, as evidenced by studies such as (Budge et al., 2012). Following a recent work (Dandel et al., 2009), we calculate the activation time of the left ventricle using strains derived from the displacement maps. In Fig. 1a, we show examples of MR images overlaid with displacement maps over time.

Figure 1:

Example of (a) temporal DENSE magnitude MRIs overlaid with displacement maps; (b) myocardial sectors of left ventricles with strain values (blue color indicates contraction vs. red color indicates expansion); (c) strain matrix and its corresponding TOS curve. Blue circles in (a) and (b) indicate intersections of left and right ventricles (Xing et al., 2021).

Given a $d$-dimensional displacement map $u\left(x\right):\Omega \to {\mathbb{R}}^d$, a Jacobian matrix $D^{d\times d}$ of $u$ at each spatial location is

$$D=\left(\begin{array}{ccc}\Delta u_1^1& \cdots & \Delta u_1^d\\ ⋮& \ddots & ⋮\\ \Delta u_d^1& \cdots & \Delta u_d^d\end{array}\right),$$

where $\Delta u_i^j=\frac{\text{d}{u}_i}{\text{ d}{x}_j}$, with $i,j\in \{1,\cdots ,d\}$ and $\Omega ={\mathbb{R}}^d/{\mathbb{Z}}^d$ be a $d$-dimensional torus domain with periodic boundary conditions, obtained from cine images. A strain matrix $E$ is then computed as $E=\frac{1}{2}\left(D^{T}D-I\right)$, where $T$ denotes a matrix transpose and I is a $d\times d$ identity matrix. This paper focuses on the circumferential strain, which is the component of strain along the myocardium. In practical clinical applications, the muscle area of the left ventricle is often divided into a variable number of sectors (Cerqueira, 2002). In this work, we divide each of the 2D slices in the volumetric stack into 18 sectors which helps in understanding the motion of the myocardial wall in detail. Then we compute an 18 – dimensional strain vector from the full-resolution strain vector of the myocardium, where the strain value is extracted from the center of each of the 18 sectors for each slice. To facilitate the automatic estimation of TOS, we then rearrange the slices into a 2D matrix consisting of all strain vectors over a temporal dimension of the cardiac phases ((Xing et al., 2021)). An example of a spatiotemporal strain matrix overlaid with a circumferential TOS curve is shown in Fig. 1c.

2.2. Atlas Building Network

The deformation-based shape features of objects in an image can be utilized for image generation and registration tasks to ensure that the generated objects preserve the topological structures (Wang and Zhang, 2021; Zhang et al., 2013; Wu et al., 2022). Here, we briefly explain the notion of atlas-building ((Joshi et al., 2004)), which computes a Fréchet mean of a group of images as a deformation-based shape representation. For a given group of $N$ images $\left\{{𝒳}_1,{𝒳}_2,\cdots ,{𝒳}_N\right\}$, each image can be represented as a deformed version of the mean image. The problem of atlas building is to find the template or mean image $𝒮$ and deformation field $\left\{{\varphi }_1,{\varphi }_1,\cdots {\varphi }_N\right\}$ with derived initial velocity fields $v_1,v_2,\cdots v_t$. This is achieved by minimizing the following energy function:

$$E\left(𝒮,{\varphi }_n\right) =\sum _{n=1}^N\frac{1}{{\sigma }^2}\text{Dist}\left[𝒮\circ {\varphi }_n\left(v_t\right),{𝒳}_n\right]+\text{Reg}\left({\varphi }_n\left(v_t\right)\right),$$ (1)

where ${\sigma }^2$ is the noise variance, $\text{Dist}[\cdot ,\cdot ]$ is a metric of dissimilarity such as local or global normalized cross-correlation, mean squared error, cosine distance, and mutual information. The ∘ denotes an interpolation operator that deforms image ${𝒳}_n$ with an estimated transformation ${\varphi }_n$. The $\text{Reg}(\cdot )$ is a regularizer that guarantees the smoothness of the transformation which can be denoted as

$$\text{Reg}\left[{\varphi }_n\left(v_t\right)\right]={\int }_0^1\left(L{v}_t,v_t\right)dt, \text{with }\frac{d{\varphi }_n\left(t\right)}{dt}=-D{\varphi }_n\left(t\right)\cdot v_n\left(t\right),$$ (2)

where $L:V\to \tilde{V}$ is a symmetric, positive-definite differential operator that maps a tangent vector $v_t\in V$ into its dual space as a momentum vector $m_t\in \tilde{V}$. This can be represented as $m_t=L{v}_t$, or $v_t=K{m}_t$, with $K$ being an inverse operator of $L$. The operator, $D$, denotes a Jacobian matrix and · represents the operation of element-wise matrix multiplication. For the metric $L$, we consider $L={\left(-\alpha {\nabla }^2+\gamma \text{I}\right)}^3$ in this paper, where ${\nabla }^2$ is the discrete Laplacian operator, $\alpha$ is a positive regularity parameter that controls the number of continuous derivates over the domain of the deformation fields, $\gamma$ is a weighting coefficient, and I denotes an identity matrix.

With a given initial condition of velocity fields $v_n\left(0\right)$, the minimum of Eq. (2) is uniquely determined by solving a Euler-Poincaré differential equation (EPDiff) (Arnold, 1966; Miller et al., 2006) which is known as the geodesic shooting algorithm (Vialard et al., 2012) which can be represented as

$$\frac{\partial v_t}{\partial t}=-K\left[{\left(D{v}_t\right)}^T\star m_t+\nabla \cdot \left(m_t\otimes v_t\right)\right],$$ (3)

where $\star$ is the truncated matrix-vector field auto-correlation. The operator $\nabla$. is the discrete divergence of a vector field and $K$ is a smoothing operator with its inverse $L$. The computation of solving the EPDiff is time-consuming and expensive, particularly when images are with high dimensions. An efficient reparameterization of the velocity fields was then developed to drastically decrease the computational cost of LDDMM with geodesic shooting without losing the model accuracy (Zhang and Fletcher, 2015, 2019).

We are now ready to minimize the atlas building energy function in Eq. (1) as

$$E\left(𝒮,{\varphi }_i\right) =\sum _{n=1}^N\frac{1}{{\sigma }^2}\text{Dist}\left[𝒮\circ {\varphi }_n\left(v_n\left(t\right)\right),{𝒳}_n\right]+\left(L{v}_n\left(0\right),v_n\left(0\right)\right),\text{ s.t. Eq. }\left(2\right)\&\left(3\right).$$ (4)

For the simplicity of notation, we will drop the temporal index in the upcoming equations.

3. Our Method

In this section, we present a novel LMA prediction framework based on SADIR, a recently developed diffusion models (Jayakumar et al., 2023), from a limited number of sparse 2D cardiac MRIs. First, we introduce shape-constrained diffusion models for myocardium reconstructions via atlas building. Here, we discuss two sub-modules, including an atlas building network, which enables us to learn shape priors from a given set of 3D myocardium images, and a forward-backward diffusion process leveraging the atlas or mean image generated from the atlas building network. Lastly, we discuss a pre-trained LMA quantification network to predict 3D LMA maps of the left ventricle using the reconstructed myocardium.

3.1. Diffusion Models For 3D Myocardium Reconstruction Via Atlas Building

Given a number of $N$ myocardium training images ${\left\{J_n,{𝒳}_n\right\}}_{n=1}^N$ where $J_n$ represents a collection of sparse 2D images paired with its corresponding full 3D volume ${𝒳}_n$. We design two sub-modules to reconstruct 3D myocardium from sparse 2D slices:

• (i) An atlas building network ($M_{\theta }$) that provides a mean image $𝒮$ of ${𝒳}_n$. Followed by the network architecture of Geo-SIC (Wang and Zhang, 2022), we develop a learning-based atlas building network characterized by an initial velocity field $\left(v_n\left(0\right)\right)$.

• (ii) A diffusion reconstruction network $G_{\theta }$, considering each reconstructed myocardium ${\widehat{𝒳}}_n$ as a deformed variant of the obtained atlas, i.e., ${\widehat{𝒳}}_n\triangleq 𝒮\circ {\varphi }_n\left(v_n\left(G_{\theta }\right)\right)$. We use the mean image from $M_{\theta }$ as the conditional shape feature for the myocardium reconstruction network $G_{\theta }$ to predict the velocity field $v_n$. Similar to the existing diffusion models (Ho et al., 2020; Waibel et al., 2023), we develop a forward process to generate an isotropic Gaussian distribution and a backward diffusion process to predict the velocity fields (Jayakumar et al., 2023) associated with the pair of input training myocardium images and an atlas image. For notation simplicity, we will omit the index $n$ for each subject in the following sections.

An overview of our network architecture is shown in Fig. 2.

Figure 2:

An overview of our proposed LMA prediction model from sparse 2D cardiac MRIs.

Forward diffusion process. For a given 3D volume of the myocardium $x^0$ and the time point of the diffusion process ${\rm Y}$, we assume the data distribution of $x^{{\rm Y}}$ is a normal distribution which can be defined as $x^{{\rm Y}}~𝒩(\mu ,\eta )$, where $\mu$ is the mean and $\eta$ is the variance. The forward diffusion process for time steps $T$ is recursively defined via the posterior

$$q\left(x^{{\rm Y}}\mid x^{{\rm Y}-1}\right)=𝒩\left(x^{{\rm Y}};\sqrt{1-{\eta }^{{\rm Y}}}{x}^{{\rm Y}-1},{\eta }^{{\rm Y}}\mathbf{I}\right),$$ (5)

where $\mathbf{I}$ denotes an identity matrix and ${\eta }^{\tau }\in \left[\mathrm{0,1}\right]$ denotes the variance scheduler increased along the time steps with ${\eta }^1<{\eta }^2<\cdots <{\eta }^{{\rm Y}}$. We repeat the forward diffusion process for a fixed, predefined number of steps.

Following (Ho et al., 2020; Waibel et al., 2023), we obtain the result of the forward diffusion process as a one-step forward transition from $x_0$ to $x_T$ as

$$q\left(x^{{\rm Y}}\mid x^0\right)=𝒩\left(x^{{\rm Y}};\sqrt{{\bar{\psi }}^{{\rm Y}}}{x}^0,\left(1-{\bar{\psi }}^{{\rm Y}}\right)\mathbf{I}\right).$$ (6)

where ${\psi }^{{\rm Y}}=1-{\eta }^{{\rm Y}}$ and ${\bar{\psi }}^T={\prod }_{i=1}^{{\rm Y}}{\psi }^i$. Hence, we can write $x^{\tau }$ in terms of $x^0$ as

$$x^{{\rm Y}}=\sqrt{\bar{\psi }{\rm Y}}{x}^0+\sqrt{1-{\bar{\psi }}^{{\rm Y}}}ϵ,\text{ with }ϵ~𝒩(0,\mathbf{I})\text{.}$$ (7)

Reverse diffusion process. In the reverse process, we employ a conditional diffusion model, where the output mean 3D myocardium $𝒮$ from the atlas building network $M_{\theta }$, along with a stack of sparse 2D slices of the volume to be reconstructed $J_n$ is concatenated with $x^{\tau }$ from the forward diffusion process as the condition $c$ to form the input of the network $G_{\theta }$. This network is trained as the reverse process of the diffusion model.

The joint probability distribution of the reverse process is given by $q\left(x^{{\rm Y}-1}\mid x^{{\rm Y}}\right)$, that is learned by an Attention-UNet (Ronneberger et al., 2015), which is a type of Convolutional Neural Network with skip connections, residual blocks and excitation layers (Hu et al., 2019).

Defining the network parameter as $G_{\theta }$, we can formulate the reverse process as

$$q_{G_{\theta }}\left(x^{{\rm Y}-1}\mid x^{{\rm Y}}\right)=𝒩\left(x^{{\rm Y}-1};{\mu }_{G_{\theta }}\left(x^{{\rm Y}},j_i,\text{ S},{\rm Y}\right),\left.{\Sigma }_{G_{\theta }}\left(x^{{\rm Y}},j_i,\text{ S},{\rm Y}\right)\right)\right.$$ (8)

As seen in (Jayakumar et al., 2023), the corresponding deformation field $\varphi$ is then generated from the initial velocity field $v_0$, obtained from the network $G_{\theta }$, to deform the atlas $𝒮$, thus giving us a reconstructed myocardium $\widehat{𝒳}=𝒮\circ \varphi \left(v_0\left(G_{\theta }\right)\right)$ for a given stack of sparse 2D myocardium slices $J_n$.

3.2. LMA Prediction using Reconstructed Myocardium Volumes

To predict the 3D LMA map, we start by mapping the predicted TOS curves from each sparse 2D slice to their corresponding sectors in the myocardium segmentation of these slices. This produces a collection of sparse 3D vertices, each assigned a specific TOS value. To ensure consistency between the sparse slices and the overarching 3D model, we introduce an alignment procedure, refining the spatial positioning of these slices to fit well with the reconstructed 3D surface (Xing et al., 2021). Since the sparse slices are sampled from the dense slices, and the dense slices are the ground-truth of our model, we automatically know the spatial relationship between the sparse slices and reconstructed 3D model. The alignment procedure mainly builds a 3D coordinate system to hold both the sparse slices and the reconstructed 3D model.

The sparse slices in our datasets (particularly breath-hold MRIs) are set to be sampled at the same location of the subject’s heart. A TOS curve comprises TOS values from specific sectors evenly spaced along the middle line of the left ventricle myocardium in the corresponding sparse slice. During the alignment process, the spatial positions of these TOS sectors adjust along with the sparse slices while maintaining their relative positions. For the alignment, we first build a 3D coordinate for the sparsely sampled 2D slices, based on their spatial information (including the slice spatial location and pixel resolution). The coordinates of the 3D mesh face vertices are automatically determined, as the 3D volume should occupy the same room as the sparsely sampled slices. With this data, we can conduct interpolation in the 3D space, enabling us to calculate the interpolated TOS values for each mesh face. For TOS prediction, we adopt a pre-trained model developed by (?). After the network predicts the TOS curve for all 2D sparse slices from the same subject, we create a 3D activation map by interpolating TOS values onto the left ventricles of the reconstructed 3D myocardium obtained $G_{\theta }$.

3.3. Network Loss and Joint Training.

The network loss of the 3D myocardium reconstruction is a joint loss of the atlas building network $M_{\theta }$ and the reconstruction network $G_{\theta }$. First, we can define the atlas building loss as

$$ℒ\left(M_{\theta }\right) =\sum _{n=1}^N\frac{1}{{\sigma }^2}{∥𝒮\left(M_{\theta }\right)\circ {\varphi }_n\left(v_n\right)-{𝒳}_n∥}_2^2+\left(L{v}_n,v_n\right)+\text{reg}\left(M_{\theta }\right),$$ (9)

where reg(⋅) denotes a regularization on the atlas building network parameters.

Then we define the loss function of the diffusion reconstruction network from 2D sparse slices as a combination of sum-of-squared differences and Sørensen–Dice coefficient (Dice, 1945) loss between the predicted reconstruction and ground-truth in the following

$$ℒ\left(G_{\theta }\right) =\sum _{n=1}^N{∥𝒮\circ {\varphi }_n\left(v_n\left(G_{\theta }\right)\right)-{𝒳}_n∥}_2^2+\lambda \left[1-\text{Dice}\left(𝒮\circ {\varphi }_n\left(v_n\left(G_{\theta }\right)\right),{𝒳}_n\right)\right]+\text{reg}\left(G_{\theta }\right),$$ (10)

where $\lambda$ is a weighting factor that obtains the extent of overlap between the predicted and ground-truth volumes. We can define the dice similarity score $\text{Dice}(\cdot ,\cdot )$ as

$$\text{Dice}\left(\widehat{𝒳},{𝒳}_n\right)=2\left(\left|\widehat{𝒳}\right|\cap \left|{𝒳}_n\right|\right)/\left(\left|\widehat{𝒳}\right|+\left|{𝒳}_n\right|\right),$$ (11)

where ${\widehat{𝒳}}_n\triangleq 𝒮\circ {\varphi }_n\left(v_n\left(G_{\theta }\right)\right)$. Defining $\alpha$ as the weighting parameter, we are now ready to define the joint loss of the reconstruction networks as

$$ℒ=ℒ\left(M_{\theta }\right)+\alpha ℒ\left(G_{\theta }\right).$$ (12)

Alternative optimization for the joint network learning. We employ an alternative optimization scheme (Nocedal and Wright, 1999) to minimize the network loss defined in Eq. (12). Here, we jointly optimize all network parameters by alternating between the atlas building network $M_{\theta }$ and reconstruction network $G_{\theta }$ without sharing weights. We summarize the joint training of our reconstruction model in Alg. 1.

4. Experimental Evaluation

We validate our proposed model to predict LMA through reconstructed 3D myocardium from 2D sparse cardiac MRIs.

Dataset. For 3D myocardium reconstruction network, we use 215 publicly available 3D myocardium mesh data from MedShapeNet data repository (Li, 2023). Using 3D slicer, we convert the mesh data to binary label maps (Fedorov et al., 2012). All the images were preprocessed as 222 × 222 × 222 and pre-aligned with affine transformation.

For LMA prediction, we obtain cine DENSE volumes of 57 subjects along with 282 strain matrices. All the scans were acquired using a 1.5T MR scanner with a four-channel phased array radio-frequency coil. The cine DENSE imaging was performed in four short-axis planes at the basal, two ventricular, and apical regions with a resolution of $2.65\times 2.65{mm}^2$, and the temporal resolution $17ms$. The field of view was $240\times 240{mm}^2$, displacement encoding frequency $k_e=0.1\mathit{\text{cycles}}/mm$, $\text{echo time}=1.08ms$, and flip angle 15°. Displacement was encoded in two orthogonal directions and a spiral k-space trajectory was used with six interleaves per image. The ground-truth TOS curves were manually labeled by experts.

We augment the dataset by shifting the strain matrices and their corresponding labels along the sector dimension. The last 5 frames of the strain matrices are trimmed due to noisy image signals. Also, we employ mixup (Zhang et al., 2017) to augment the dataset yielded 3844 strain matrices together with the ground truth. All datasets used in our work were publicly released or pre-collected. As such, no ethical considerations are necessary.

4.1. Experimental Settings.

We first validate our proposed model for reconstructing 3D myocardium from a sparse stack of eight 2D slices. We compare the performance of our model with three state-of-the-art deep learning-based reconstruction models: 3D-UNet (Çiçek et al., 2016b); DDPM, a probabilistic diffusion model (Ho et al., 2020); and DISPR, a diffusion model based shape reconstruction model with the consideration of geometric topology (Waibel et al., 2023). We visualize and compare the reconstructed 3D myocardium of all models. We evaluate all the reconstruction models using three evaluation metrics, including, the Sørensen–Dice coefficient (DSC) (Dice, 1945), Jaccard Similarity (Jaccard, 1908), and RHD95 score (Huttenlocher et al., 1993), validating the prediction accuracy of 3D myocardium for all methods. Lastly, we show the predicted 3D activation maps and compare them with the clinical reference model, concentrating on the LMA region in the reconstructed myocardium.

Parameter Setting. We set the mean and standard deviation as 0 and 0.1, respectively, for the forward diffusion process. We use linear scheduling for the noising process and is scaled to reach an isotropic Gaussian distribution irrespective of the value of $T$. In the reverse diffusion process, we set the depth of the 3D attention-UNet backbone as 6, introducing an attention mechanism via spatial excitation channels (Hu et al., 2019) with ReLU (Rectified Linear Unit) activation. For the atlas building network, we set the depth of the UNet architecture as 4. We set the number of time steps for Euler integration in EPDiff (Eq. (3)) as 10, and the noise variance $\sigma =0.02$. We use a kernel map valued [0.5, 0, 1.0] for the shooting operation. Besides, we set the parameter $\alpha =3$ for the operator $L$. Similar to (Waibel et al., 2023), we set the batch size as 1 for all experiments.

We split the dataset into 70% training, 15% validation and 15% testing, utilizing cosine annealing learning rate scheduler that starts with a learning rate of $\eta =1{\text{e}}^{-3}$. We run all models on training and validation images using the Adam optimizer and save the networks with the best validation performance. We conduct all experiments on one $80GB$ NVIDIA A100 GPU for ~18 hours. For both training and testing, we downsample the volumes to 64 × 64 × 64.

4.2. Experimental Results

Fig. 3 shows a comparison of the 3D myocardium between the ground truth and all reconstruction models, including such as 3D-UNet (Çiçek et al., 2016a), DDPM (Ho et al., 2020), DISPR (Waibel et al., 2023), and our model. It shows that our method outperforms other baselines in maintaining the topology of the myocardium. Besides, compared to the SOTA models, our reconstruction framework produces little to no artifacts, preserving the original shape of the organ. Our work uses sparse 2D binary label-maps of the myocardial walls to reconstruct the 3D volume by deforming the 3D atlas with the predicted deformation fields. This ensures that our model performs with similar scores when there are significant variations in image quality since it relies on the segmentations obtained from these scans. Moreover, the stack of sparse 2D slices provides sufficient information on patient-specific differences in the anatomy of the myocardium.

Figure 3:

A comparison of reconstructed 3D myocardium between ground truth, 3D-UNet, DDPM, DISPR, and our model over four different views.

Tab. 1 reports the average DSC (Dice, 1945), Jaccard similarity (Jaccard, 1908), and Hausdorff distance (RHD95) (Huttenlocher et al., 1993) compared between the reconstructed myocardium from the ground truth and all models. Comparing the results of all models, our method outperforms all the baselines, achieving the best performance in DSC, Jaccard similarity, and RHD95 with the lowest standard deviations across all metrics.

Table 1:

A comparison of 3D myocardium reconstruction for our framework compared to other SOTA models

Model	DSC ↑	Jaccard similarity ↑	RHD95 ↓
3D-Unet	0.870 ± 0.0158	0.771 ± 0.024	0.840 ± 0.202
DDPM	0.823 ± 0.014	0.668 ± 0.019	1.027 ± 0.093
DISPR	0.950 ± 0.017	0.906 ± 0.031	0.347 ± 0.032
Ours	0.978 ± 0.016	0.957 ± 0.031	0.341 ± 0.023

Fig. 4 shows the generated 3D late mechanical activation maps using the reconstructed myocardial meshes for two subjects over three different views. The regions in red show larger TOS values, representing the late mechanical activation area while the regions in blue show areas containing lower TOS values, representing as the are normal regions. The 3D LMA maps provide intuitive visual information of LMA regions (highlighted in red in Fig. 4) location on the subject’s left ventricle. Compared with previous methods, our models have much more accurate anatomical structures, which allows the doctors to more precisely identify the anatomical structure of the LMA regions. Additionally, it shows the differences between anatomical features of the myocardium interpolated using the current reference model used by clinicians, which does not provide information on patient-wise variability in the anatomy of the myocardium, and the myocardium reconstructed by our framework. It is evident that our model efficiently preserves the shape information of the organ. This plays a vital role in an improved and clinically significant detection of LMA regions, hence allowing for personalized surgical plans as opposed to the traditional one-size-fits-all model. While our method effectively preserves the geometrical features of the myocardium with an improved performance of reconstruction, a thorough evaluation of the method on a variety of large datasets is needed. This future work could further improve the model’s generalizability across various image modalities with larger thicknesses of input slices.

Figure 4:

A comparison of 3D late mechanical activation maps using reconstructed myocardium volume of two subjects between our model and the clinical reference model.

5. Conclusion

This paper presents a shape-constrained diffusion model to obtain the 3D late mechanical activation of the myocardium given sparse 2D slices of DENSE MRI. The introduction of the mean image as a prior during the diffusion process helps the model to efficiently learn and preserve the inherent anatomical structure of the myocardium. We train and evaluate the performance of our reconstruction model on 3D myocardium mesh data. Using this pre-trained network, we predict the 3D volumes of the myocardium for two subjects and compare them to the current 3D reference model used in the clinical setting. Compared to the state-of-the-art models, our proposed network performed substantially well in predicting 3D late mechanical activations through myocardium reconstruction from sparse 2D slices. The clinical application of our framework would be pivotal since it can help in the accurate detection of LMA regions, thus allowing for effective CRT procedures.