Medical image alignment based on landmark- and approximate contour-matching

Abstract.

Purpose: Our goal is to propose a landmark- and contour-matching (LCM) registration method that uses both landmark information and approximate point correspondences to boost the similarity between image pairs with sparse landmark information.

Approach: A model for registering two-dimensional (2D) medical images with landmark information and contour-approximating landmarks was proposed. The model was also extended to accommodate the registration of three-dimensional (3D) cardiac images. We validated the LCM method on 2D hand x-rays and 3D porcine cardiac magnetic resonance images. The following metrics were used to assess the quality of specific aspects of the registered images: Dice similarity coefficient for the overall image overlap, target registration error for pointwise correspondence, and interior angle for local curvature.

Results: Target registrations were reduced from 27.12 to 0.01 mm post-LCM registration. Implementing the proposed algorithm also led to a 112% average improvement in image similarity in terms of Dice coefficients. In addition, interior angle measurements indicate that the proposed method preserved the local curvature at major reference landmarks and mitigated the appearance of deformities in the registered images.

Conclusions: The proposed method addressed several issues associated with purely landmark-based techniques, such as iterative closest point registration and thin plate spline interpolation. Furthermore, it provided accurate registration results even in the presence of landmark localization errors.

1. Introduction

Image registration can generally be classified under two main categories: landmark-based and intensity-based. Intensity-based registration matches corresponding structures between images by minimizing a distance measure that quantifies voxel similarity over the entire image domain without the need for guiding landmarks.¹ As such, these methods tend to be more robust but computationally expensive.

On the other hand, landmark-based registration can be viewed simply as either an interpolation or a data-fitting problem. However, defining landmarks in medical images can be prone to errors since the selection of landmarks highly depends on the ability of the physician to mentally integrate information from different images.^2–5 In addition, some organs such as the heart only have few spatially accurate and repeatable anatomical landmarks and, hence, insufficient information to guide the transformations.⁶ Consequently, landmark information is typically used only to match the scaling and orientation as a pre-registration step, after which intensity information is utilized to further maximize the post-registration image overlap.

Several approaches combining landmark- and intensity-based registration have already been proposed. Eriksson and Astrom introduced in Ref. 7 an intensity-based approach that focused on minimizing the sum of squared differences while restricting the solution space to thin plate spline (TPS) mappings.^8–10 In Ref. 11, the normalized gradient field similarity measure was employed together with the elastic regularizer. Similar restrictions were imposed on the solution space.

However, even with these hybrid landmark- and intensity-based techniques, minimizing the amount of misregistration can still be tricky. In our previous work,¹² we tried to address several of the issues associated with processing medical data with sparse landmark information by proposing a method to automatically detect interest points and an intuitive partitioning of the contours in an image. This effectively reduced the need for an expert to manually delineate points of interest (POIs) and the possibility of introducing landmark localization errors.

In this paper, we present a model for intra-modality registration that builds on one of the most commonly used landmark-based methods—TPS^13–16—by simultaneously using landmark and approximate contour information. Through a comprehensive assessment of the registration results, we demonstrate that the landmark- and contour-matching model (LCM) boosts image similarity and landmark-matching and preserves the local curvature at control points better than TPS and other landmark-based techniques even in the presence of landmark localization errors.

2. Background: Thin Plate Spline Registration

TPS are a spline-based class of interpolating functions widely used in medical imaging. Introduced by Duchon in Ref. 9, TPS interpolates over scattered data while minimizing the bending energy of a thin metal plate on point constraints.¹⁰ The idea of TPS was later used in point-based registration.

Let $\mathcal{R}$ and $\mathcal{T}$ be the reference and template images defined on an image domain $\mathrm{\Omega }$ and suppose that ${\{r_j\}}_{j=1}^K$ and ${\{t_j\}}_{j=1}^K$ are $K$ landmark pairs from $\mathcal{R}$ and $\mathcal{T}$, respectively. The transformation that minimizes the bending energy of a TPS while simultaneously satisfying the landmark constraints minimizes following optimization problem:

$$\begin{gathered} \underset{\theta }{\min }{\mathcal{S}}^{\mathrm{TPS}}[\theta ]≔\underset{\theta }{\min }{\int }_{\mathrm{\Omega }}⟨{\nabla }^2\theta (x),{\nabla }^2\theta (x)⟩\mathrm{d}x \\ \text{subject to}\sum _{j=1}^K{(\theta (t_j)-r_j)}^2=0. \end{gathered}$$ (1)

It has been shown that the unique solution of Eq. (1) can be represented by a linear combination of radial basis functions and an affine part.^8–10,17 That is, for a $d$-dimensional registration problem, $\theta$ is of the form

$${\theta }^i(x)=\sum _{j=1}^K{c}_j^i\rho (\Vert x-t_j\Vert )+\sum _{j=0}^d{w}_j^i\text{for}i=1,\dots ,d,$$ (2)

where $\{c_j^i\},\{w_k^i\}\subset \mathbb{R}$. In the presence of localization errors, it is possible to relax the interpolation condition in Eq. (1) by replacing the interpolation by approximation as follows:

$$\underset{\theta }{\min }\sum _{j=1}^K{(\theta (t_j)-r_j)}^2+\beta {\int }_{\mathrm{\Omega }}⟨{\nabla }^2\theta (x),{\nabla }^2\theta (x)⟩\mathrm{d}x.$$ (3)

The parameter $\beta \ge 0$ in the objective function above controls the smoothness of the minimizer and the mismatch of the data points. Note that setting $\beta =0$ yields the interpolation problem in Eq. (1).

3. Proposed Landmark and Contour-Matching Model

Let $\mathcal{R}$ and $\mathcal{T}$ be the reference and template images with exact (major) landmarks ${\{r_j\}}_{j=1}^K$ and ${\{t_j\}}_{j=1}^K$, respectively. Given that ${\{{\stackrel{*}{r}}_j\}}_{j=1}^L$ and ${\{{\stackrel{*}{t}}_j\}}_{j=1}^L$ are ordered sets of sampling points that trace the contours of a region of interest present in $\mathcal{R}$ and $\mathcal{T}$, we aim to solve the optimization problem

$${\theta }^{*}=\underset{\theta }{\arg \min }\stackrel{{\mathcal{D}}^{\mathrm{LM}}[\theta ]}{\overbrace{\sum _{j=1}^K{\Vert \theta (t_j)-r_j\Vert }^2}}+\alpha \stackrel{\mathcal{C}[\theta ]}{\overbrace{\sum _{j=1}^L\frac{1}{2}[1-{(v[\theta ({\stackrel{*}{t}}_j)]·v[{\stackrel{*}{r}}_j])}^2]}},$$ (4)

in which $\theta$ is a TPS transformation [Eq. (2)]. In the LCM model [Eq. (4)],

• a.the first term ${\mathcal{D}}^{\mathrm{LM}}$ denotes the sum of squared landmark distances,
• b.the second term $\mathcal{C}$ denotes the contour matching term, where
• c.$v[\theta ({\stackrel{*}{t}}_j)]·v[{\stackrel{*}{r}}_j]$ signifies the cosine of the angle between corresponding normalized (unit) secant vectors $v[\theta ({\stackrel{*}{t}}_j)]$ and $v[{\stackrel{*}{r}}_j]$ formed by connecting consecutive contour-approximating points in the transformed template and reference images, and
• d.
  every component of the contour-matching term is bounded. In particular, for $j\in \{1,\dots ,L\}$,

  $$0\le {\mathcal{C}}_j[\theta ]=\frac{1}{2}[1-{(v[\theta ({\stackrel{*}{t}}_j)]·v[{\stackrel{*}{r}}_j])}^2]\le \frac{1}{2}.$$

Minimizing the contour-matching term $\mathcal{C}$ is equivalent to maximizing the similarity in the orientation of corresponding unit vectors without any constraints on scaling. Thus, solving the registration problem in Eq. (4) relaxes the interpolation conditions and balances the overlap of the exact landmarks and the overall similarity between the orientation of the image contours.

An example of the setup required in the proposed LCM model is shown in Fig. 1. An extension of the model is discussed in Sec. 4.3 to accommodate three-dimensional (3D) images.

Fig. 1.

LCM model requisites. (a) Image, (b) major landmarks, (c) exact landmarks (in pink), contour-approximating points (dark gray), and vectors (light gray). Here, the number of exact landmarks is $K=11$ and the number of contour-approximating points is $L=33$.

4. Methods

4.1. Data

Hand x-rays from Refs. ¹⁷ to ¹⁹ were used to validate the proposed LCM registration method for two-dimensional (2D) images.

Eight explanted healthy porcine hearts were used for the 3D experiments. The hearts were placed in a Plexiglass phantom filled with Fluorinert to avoid susceptibility artifacts at air–tissue interface and scanned on a 1.5T GE Signa Excite using a high-resolution head coil, as described in a previous study.²⁰ Specifically, here, we obtained the heart anatomy from T2-weighted images (3D FSE MR pulse sequence, $\mathrm{TR}=700\mathrm{ms}$, $\mathrm{TE}=35\mathrm{ms}$, $\mathrm{NEX}=1$, $\mathrm{ETL}=2$, $\mathrm{FOV}=10$ to 12 cm) which were reconstructed and interpolated at $0.5\times 0.5\times 1.6{\mathrm{mm}}^3$ spatial resolution.

4.2. Landmark Detection

Prior to registration, the landmark detection method proposed in our previous work¹² was used to generate a discretization of the contours present in the hand images and in the sampling short-axis slices of the cardiac volumes.

We defined the POIs as points along the edges in an image where high curvature occurs. Since the curvature measures how sharply a curve bends,²¹ the task of locating the POIs was simplified by instead finding where the minimum interior angle occurs [see Fig. 2(a)].

Fig. 2.

Interest point detection using interior angles. (a) The points of high curvature (POIs) along the contour of the hand have narrower interior angles. (b), (c) Interior angles in endocardial segments. When a cross-section of the RV is present, the interest points are taken to be the RV cusps. In slices where only the elliptical contours of the epicardium and LV endocardium are present, the POIs are defined to be the ends of the major axis of the LV ellipse (vertices in red and green).

In hand x-ray images, the POIs were the fingertips and the cusps in between adjacent fingers. Meanwhile, the POI within an axial cardiac slice would depend on the presence (or absence) of a cross-section of the right ventricle (RV). If a cross-section of the RV was visible like in basal and mid-cavity slices, then the POIs were given by the RV insertion points. On the other hand, in apical slices where only the LV and epicardial contours were present, curvature was maximal at the ends of the major axis of the endocardial LV ellipse [see Figs. 2(b) and 2(c)].

Finally, a discretization of each arc connecting adjacent POIs was obtained by partitioning the ordered set of connected pixels into shorter arcs of equal length. For more details, we refer the reader to Ref. 12.

4.3. Registration

For the 2D experiments, no initial registration was used. Owing to the larger deformations involved in inter-subject 3D cardiac registration, an affine pre-registration using the three repeatable landmarks was performed prior to LCM to correct the scaling and orientation of the template image.

The constrained minimization problem in Eq. (4) was solved using Newton’s method to obtain the optimal transformation parameters. At every iteration, the distances between the transformed major template landmarks and their target locations were calculated. In addition, the vectors connecting adjacent contour-approximating landmarks were normalized, and the cosine of the interior angles formed by corresponding unit vector pairs in the reference and transformed template were calculated to measure the overall similarity between the orientation of the contours present in the two images.

The exact Hessian of both the LCM terms in Eq. (4) were used in the implementation of Newton’s method.

4.4. Validation

Iterative closest point (ICP)^22–25 and TPS registration were performed to provide benchmarks for the LCM technique.

4.4.1. 2D LCM registration

In experiment 1a, we observed the effect of blindly performing TPS registration by using only 11 POIs as exact landmarks. Then, in experiment 1b, we observed whether using the same number of major landmarks as in experiment 1a but adding extra contour information to be used only in the contour-matching term $C$ lessens the occurrence of unnatural bending and improves the image overlap away from the exact landmarks.

In the second set of experiments, a landmark localization error was introduced to the major template landmarks to determine the accuracy of the registration methods in the presence of a landmark localization error.

Dice coefficients, target registration errors (TRE), and interior angles of the registered images resulting from the application of ICP and TPS were then compared against those of LCM-registered images. The use of these three measures of accuracy aims to provide a comprehensive assessment of the quality of the registered images.

The Dice coefficient describes the amount of overlap between the co-registered images, whereas TRE provides a more localized measure of how well the optimal transformation obtained from each method aligns the control points. In the 2D case, TRE was computed as the distance between the location of the transformed landmark with localization error from its correct location.

A robust image registration method not only provides good image overlaps but also preserves the shape (curvature) of the reference image. Recall that the central idea behind the landmark detection method discussed in Sec. 4.2 used interior angles to characterize the curvature at a point: a small interior angle signified a point of high curvature. For this reason, we calculated the difference in the interior angle measurement at each transformed template point where a localization error was introduced and its corresponding interior angle on the reference image. A smaller difference implied that the registration method closely captured the target local curvature.

4.4.2. 3D Analog of LCM registration for cardiac volumes

The proposed model was modified to accommodate the registration of the same set of 3D cardiac images used in Refs. ¹² and ²⁶.

Surface-approximating landmarks were detected prior to LCM registration. In every sampling slice, points along the free wall of the RV, the interventricular septum, and the left ventricular myocardium were chosen based on their location with respect to points of maximum curvature within the short-axis slice as discussed in Sec. 4.2 [see also Figs. 2(b) and 2(c)].

Note that connecting adjacent landmarks resulted to the formation of approximate myocardial contours that resembled the latitude and longitude of the Earth’s lower hemisphere [Fig. 3(a)]. This meant that each approximate landmark could be associated with two vectors: one parallel to the short axis of the heart ($v_{\mathrm{Hor}}$), and the other extending from the approximate landmark to its corresponding point in the next sampling slice ($v_{\mathrm{Vert}}$). Thus, given $L$ surface-approximating landmarks, we defined the 3D analog of the contour-matching term $\mathcal{C}$ of the proposed LCM model [Eq. (4)] specifically for our cardiac dataset as

$$\begin{gathered} \mathcal{C}[\theta ]=\sum _{j=1}^L\frac{1}{2}[1-{(v_{\mathrm{Hor}}[\theta ({\stackrel{*}{t}}_j)]·v_{\mathrm{Hor}}[{\stackrel{*}{r}}_j])}^2] \\ +\frac{1}{2}[1-{(v_{\mathrm{Vert}}[\theta ({\stackrel{*}{t}}_j)]·v_{\mathrm{Vert}}[{\stackrel{*}{r}}_j])}^2]. \end{gathered}$$ (5)

Aside from calculating the improvement of image overlap in terms of Dice coefficients, we computed the average TRE with respect to the three repeatable cardiac landmarks, namely the left ventricular apex ${\mathrm{A}}_{\mathrm{LV}}$, and the two right ventriculo-septal junctions ${\mathrm{P}}_1$ and ${\mathrm{P}}_2$ [Figs. 3(b)–3(c)]. Similar to the 2D experiments, we compared the quality of LCM- versus ICP- and TPS-registered images.

Fig. 3.

Landmark and surface-matching (LCM) model requisites. (a) Details of the required setup for 3D LCM registration: heart with longitudinal and latitudinal vectors at each surface point. (b), (c) Actual LCM setup for two hearts in the dataset with latitudinal and longitudinal segments connecting adjacent surface points with the repeatable landmarks ${\mathrm{P}}_1$, ${\mathrm{P}}_2$, and ${\mathrm{A}}_{\mathrm{LV}}$.

5. Results

5.1. 2D LCM Registration

The results of the first set of experiments are displayed in Fig. 4(a). Imposing only a few hard constraints in TPS registration produced images where the fingers were slightly bent. $K=11$ exact landmarks were used in experiment 1.

Fig. 4.

Experiment 1. Comparison of TPS and LCM registration accuracy when no errors are present in the landmark data. (a) Results of experiment 1a, (b) results of experiment 1b. (c) LCM registration results. Pre-registration Dice = 0.64. (First column) Reference image with exact and contour-approximating landmarks, (second column) registered image, (third column) post-registration subtraction image $|R-T[\theta ]|$, (fourth column) optimal transformation. Bending, ridges along the finger contours, and bone deformities are present in the TPS-registered images in (a) and (b). Notably, the LCM-registered image in (c) does not suffer from such deformities.

Next, 55 contour-approximating landmarks were used as exact landmarks in the TPS interpolation problem to determine whether an increase in the number of exact landmark correspondences results to an improvement in the registration accuracy. While the Dice similarity coefficient did increase, irregularities were present in the transformed images. Notice that the metacarpophalangeal joints of the middle and ring fingers were distorted by the registration transformation [Fig. 4(b)].

In contrast, LCM registration led to an improved Dice similarity coefficient compared to experiment 1a and a more visually accurate transformed template [Fig. 4(c)]. More specifically, there were no apparent deformities such as bent fingers and distorted bones in the LCM-registered images.

Exemplary results from experiment 2 are provided in Fig. 5. Each column in said figure corresponds to a set of experiments where a localization error was introduced to one of the 11 of the major landmarks. Displayed in the third and fifth rows are ICP registration results with different numbers of landmarks, while the fourth and sixth rows are the TPS results for the same landmark sets—this time treated as hard constraints. The last row shows LCM-registered images for each experiment.

Fig. 5.

ICP versus TPS versus LCM comparisons for experiment 2, where a localization error was introduced to $t_1,\dots ,t_4$. (a) Template image; (b) and (d) ICP-registered images, and (c) and (e) TPS-registered images with $K=11$ and $K=55$ exact landmarks, respectively; (f) LCM-registered images with $\alpha =1\times {10}^4$, $K=11$ exact and $L=54$ contour-approximating landmarks. Direct application of ICP and TPS interpolation led to misregistrations. LCM mitigates the effect of the LM error.

ICP failed to sufficiently align the images. The optimal ICP transformation was significantly impacted by the introduction of a localization error to one of the major landmarks. Of the three methods, it yielded the lowest image overlaps and the largest TREs. This is not surprising due to the fact that ICP only provides rigid transformations, which are not suitable when the control points move locally.

The TREs associated with different configurations for experiment 2 (e.g., registration method used, location of major landmark error, number of contour-approximating points used, etc.) can be found in Table 1. Observe that while increasing the number of exact landmark-matching conditions for TPS sometimes led to smaller TREs, it did not necessarily guarantee a visually accurate transformed template as demonstrated in row 6 of Fig. 5.

Table 1.

TREs resulting from experiment 2, where a localization error was introduced to one of the major LMs ${\{t_i\}}_{i=1}^{11}$.

	Experiment 2 TREs
	Major landmark with error
	$t_1$	$t_2$	$t_3$	$t_4$	$t_5$	$t_6$	$t_7$	$t_8$	$t_9$	$t_{10}$	$t_{11}$	Mean TRE
$\mathrm{ICP}(K=11,L=0)$	14.85	13.21	25.51	6.81	19.43	0.98	15.88	8.55	11.62	39.32	12.44	15.33
$\mathrm{TPS}(K=11,L=0)$	13.39	11.21	8.51	11.56	9.27	7.15	5.85	6.07	5.87	10.03	13.64	9.32
$\mathrm{ICP}(K=55,L=0)$	21.30	5.35	21.95	7.81	16.66	6.73	16.61	8.76	15.36	28.63	14.52	14.88
$\mathrm{TPS}(K=55,L=0)$	4.18	3.97	5.12	5.59	7.13	2.80	3.84	4.70	2.01	3.72	3.98	4.28
$\mathrm{LCM}(K=11,L=54)$	4.51	4.66	1.55	3.29	5.19	4.20	4.67	4.10	5.52	8.90	5.51	3.38

When analyzing the results of experiment 2 (with one landmark localization error), it is important to note that TPS registration, at its core, is just an interpolation technique. Therefore, blindly applying the technique would naturally result to misregistrations.

One way to quantify the amount of misregistration at every major landmark is by measuring the interior angle at the transformed major landmark and comparing it against the interior angle at its corresponding reference landmark (Fig. 6). A small difference in the angle measurements implies that the local radius of curvature at the reference landmark was induced by the registration method.

Fig. 6.

Interior angle (in yellow) as a measure of the local curvature. (a) Reference, (b) ICP-registered, (c) TPS-registered, and (d) LCM-registered images. A small difference in interior angles with respect to the reference indicates that the local radius of curvature at the major landmark was preserved by the registration method despite the presence of errors. The average post-registration differences in interior angle measurement were $\overline{|{\angle }_R-{\angle }_{\mathrm{ICP}}|}=16.9211\mathrm{deg}$, $\overline{|{\angle }_R-{\angle }_{\mathrm{TPS}}|}=23.6723\mathrm{deg}$, and $\overline{|{\angle }_R-{\angle }_{\mathrm{LCM}}|}=16.5685\mathrm{deg}$.

The average post-registration differences in interior angle measurement from the three methods were $\overline{|{\angle }_R-{\angle }_{\mathrm{ICP}}|}=16.9211\mathrm{deg}$, $\overline{|{\angle }_R-{\angle }_{\mathrm{TPS}}|}=23.6723\mathrm{deg}$, and $\overline{|{\angle }_R-{\angle }_{\mathrm{LCM}}|}=16.5685\mathrm{deg}$. These results indicated that our method preserved the local curvature at the major landmarks and mitigated the appearance of deformities in the transformed template better than the benchmarks did.

In summary, the proposed LCM method outperformed the ICP and TPS methods. It resulted in better post-registration Dice (PRD) image similarities, lower average TREs, and transformed images without abnormalities even in the presence of a landmark error.

5.2. 3D Analog of LCM Registration for Cardiac Volumes

We now present the results of the modified version of the LCM registration method specifically designed for 3D cardiac registration.

Exemplary results are shown in Figs. 7–8, along with cross-sections of the template, registered templates, and their corresponding difference images with respect to the reference image. Once again, rigid ICP transformations failed to properly register cardiac images from different subjects [Figs. 7(g) and 8(g)]. TPS-registered images manifested deformities similar to those observed in the 2D hands experiments. Such deformities were more noticeable when viewing short-axis slices of the registered images, whose edges exhibited ridges even when the original template edges were smooth [Figs. 7(h) and 8(h)]. In contrast, the registered images obtained through LCM registration [Figs. 7(i) and 8(i)] were free from such deformities and instead had smoother edges.

Fig. 7.

3D registration. (a)–(e) 3D view of the reference, template, ICP-, TPS-, and LCM-registered images, (f) short-axis slices of the template and pre-registration difference images, (g) short-axis slices of the ICP-transformed template and difference images with respect to the reference, (h) short-axis slices of the TPS-transformed template and difference images with respect to the reference, and (i) short-axis slices of the LCM-transformed template and difference images with respect to the reference, $\alpha =0.1$.

Fig. 8.

3D registration. (a)–(e) 3D view of the reference, template, ICP-, TPS-, and LCM-registered images, (f) short-axis slices of the template and pre-registration difference images, (g) short-axis slices of the ICP-transformed template and difference images with respect to the reference, (h) short-axis slices of the TPS-transformed template and difference images with respect to the reference, and (i) short-axis slices of the LCM-transformed template and difference images with respect to the reference, $\alpha =0.1$.

TREs were calculated and summarized in Table 2 to measure the performance of the 3D LCM model when coupled with the proposed setup. The average pre-registration TRE decreased from 27.12 to 0.01 mm after LCM registration. The maximum post-LCM TRE across all possible reference-template pairings was 0.0977 mm. We report a 112%-average improvement in Dice similarity coefficients after implementing the proposed method.

Table 2.

Comparison of PRD similarity and TREs. TREs are expressed in terms of the voxel locations of the three repeatable landmarks shown in Fig. 3. Mean improvement refers to the percentage change (in PRD/TRE) across all possible reference-template image pairings. For PRD, a higher mean improvement is better. It means that image overlap increased after registration. On the other hand, a smaller post-registration TRE implies a better alignment of the repeatable landmarks.

	Post-3D cardiac registration Dice and TRE
	PRD (higher is better)			Average post-reg. TRE (lower is better)
	Mean	SD	Mean improvement	Mean	SD	Mean improvement
ICP	0.54	0.22	0.22	23.46	13.29	−0.10
TPS	0.91	0.04	1.37	0	0	−1.0
LCM	0.83	0.08	1.12	0.01	0.01	−0.87

6. Conclusions and Future Work

We proposed a new registration model that used contour-approximating landmarks to supplement missing edge information in between defined landmarks. We demonstrated that the model was able to circumvent drawbacks associated with the straightforward application of the ICP and TPS registration techniques.

The LCM method was shown to increase the PRD similarity between the reference and registered template by improving the image overlap away from major landmarks. Consequently, this reduced the appearance of the unnatural bending in image regions bordered by the data interpolation points (major landmark locations).

We then showed that naively increasing the number of interpolation conditions did not always guarantee a clinically accurate registration result. Doing so resulted in an ill-conditioned problem,²⁷ made the ICP and TPS techniques computationally more expensive, and caused visual deformities in the transformed template. Meanwhile, solving the LCM registration problem given a small number of exact landmarks and supplementary approximate contour information provided accurate results.

The cardiac interest point and surface-approximating method proposed in Ref. 12, together with a 3D extension of the 2D LCM model, provided a complete framework for cardiac image registration that reduced the need to manually delineate anatomical landmarks and supplied a dense collection of reference-template point clouds to facilitate image alignment.

Interior angle measurements were also employed to assess the performance of the registration methods in terms of preserving the local curvature at landmarks with localization errors. We found that TPS failed this last test, while the proposed method induced interior angles that were closest to those in the reference image. This meant that it further mitigated the appearance of deformities in the registered images and captured the desired local curvature.

Overall, the LCM model increases the flexibility of the TPS approach especially when only a few repeatable landmarks can be defined, when defining too many landmarks leads to high oscillations in the registration transformations, or when the identification of exact landmarks is susceptible to human error.

We remark that our methods are currently limited to high-contrast medical images that can be easily segmented through intensity thresholding. Future work includes the improvement of the segmentation and classification steps (as in Refs. ^28–30) to allow the proposed pipeline to accommodate the co-registration of ex-vivo and in-vivo cardiovascular MR images. For instance, such steps would be useful for computational applications employing predictive simulations that would require the registration of our previously developed high-resolution ex-vivo DTI-based cardiac fiber atlas²⁶ to in-vivo MRI-based 3D heart models. In line with this, we also aim to make landmark detection and segmentation in 3D cardiac images more robust by including an LV chamber ellipsoid-fitting as a preprocessing step. This would allow the extension of the interest point detection method to accommodate cardiac volumes that were imaged with a significant tilt relative to the $z$-axis.

Biographies

Mia Mojica is a data analyst at the Slaight Family Centre for Advanced MRI, Toronto Western Hospital. She received her MSc in applied mathematics from the University of the Philippines in 2013 and has recently received her PhD in modeling and computational science (Ontario Tech University). Her research interests lie in the intersection of numerical analysis, mathematical imaging, and its applications. Since 2009, she has been working on inverse problems in image reconstruction.

Mihaela Pop is currently a visiting scientist at Inria (France) and Sunnybrook Research Institute Toronto (Canada). She received her MSc and PhD degrees in medical biophysics from the University of Toronto. Her preclinical research is focused on identifying arrhythmia foci in infarcted hearts using high resolution cardiac MR imaging methods combined with electrophysiological modeling. She is co-founder and chair of the STACOM workshop and on the board of directors of FIMH (Functional Imaging and Modelling of Heart) forum.

Mehran Ebrahimi is an associate professor in the Faculty of Science at Ontario Tech University, Oshawa, Ontario, Canada. He received his PhD in applied mathematics from the University of Waterloo, Canada, in 2008. His research interest includes medical image analysis and inverse problems in imaging. His research is supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC).

Disclosures

The authors have no conflicts of interest to disclose.

Data, Materials, and Code Availability

The code used to generate the results and figures is available in a Dropbox repository.