Multiattribute probabilistic prostate elastic registration (MAPPER): Application to fusion of ultrasound and magnetic resonance imaging

Abstract

Purpose:

Transrectal ultrasound (TRUS)-guided needle biopsy is the current gold standard for prostate cancer diagnosis. However, up to 40% of prostate cancer lesions appears isoechoic on TRUS. Hence, TRUS-guided biopsy has a high false negative rate for prostate cancer diagnosis. Magnetic resonance imaging (MRI) is better able to distinguish prostate cancer from benign tissue. However, MRI-guided biopsy requires special equipment and training and a longer procedure time. MRI-TRUS fusion, where MRI is acquired preoperatively and then aligned to TRUS, allows for advantages of both modalities to be leveraged during biopsy. MRI-TRUS-guided biopsy increases the yield of cancer positive biopsies. In this work, the authors present multiattribute probabilistic postate elastic registration (MAPPER) to align prostate MRI and TRUS imagery.

Methods:

MAPPER involves (1) segmenting the prostate on MRI, (2) calculating a multiattribute probabilistic map of prostate location on TRUS, and (3) maximizing overlap between the prostate segmentation on MRI and the multiattribute probabilistic map on TRUS, thereby driving registration of MRI onto TRUS. MAPPER represents a significant advancement over the current state-of-the-art as it requires no user interaction during the biopsy procedure by leveraging texture and spatial information to determine the prostate location on TRUS. Although MAPPER requires manual interaction to segment the prostate on MRI, this step is performed prior to biopsy and will not substantially increase biopsy procedure time.

Results:

MAPPER was evaluated on 13 patient studies from two independent datasets—Dataset 1 has 6 studies acquired with a side-firing TRUS probe and a 1.5 T pelvic phased-array coil MRI; Dataset 2 has 7 studies acquired with a volumetric end-firing TRUS probe and a 3.0 T endorectal coil MRI. MAPPER has a root-mean-square error (RMSE) for expert selected fiducials of 3.36 ± 1.10 mm for Dataset 1 and 3.14 ± 0.75 mm for Dataset 2. State-of-the-art MRI-TRUS fusion methods report RMSE of 3.06–2.07 mm.

Conclusions:

MAPPER aligns MRI and TRUS imagery without manual intervention ensuring efficient, reproducible registration. MAPPER has a similar RMSE to state-of-the-art methods that require manual intervention.

1. INTRODUCTION

Transrectal ultrasound (TRUS)-guided needle biopsy is the gold standard for prostate cancer diagnosis.¹ In TRUS-guided biopsy, the prostate is divided into six regions and two cores per region are taken,² additional cores may be taken from cancer suspicious regions. Up to 40% of prostate cancer lesions is isoechoic on TRUS making lesions difficult to target.³ Due to these limitations, 30% of men who has a prostate biopsy will undergo a repeat procedure.⁴ Magnetic resonance imaging (MRI) is better able to visualize prostate cancer lesions.⁵ However, MRI-guided biopsy requires specialized equipment and technicians and is expensive and time-consuming.⁶

MRI-TRUS fusion, where MRI is spatially aligned to TRUS, enables both modalities to guide biopsy. Combining MRI and TRUS substantially increases the positive yield of prostate biopsies.^7–10 Labanaris et al.⁷ demonstrated improved prostate cancer detection in 260 patients divided into two groups: (1) a 18-core TRUS-guided biopsy had a detection rate of 19.4% and (2) a 18-core biopsy with additional cores taken from cancer suspicious regions on MRI had a detection rate of 74.9%.

There are several challenges that must be overcome in MRI-TRUS fusion. Traditional intensity-based metrics are inappropriate due to poor correlation between MRI and TRUS intensities.¹¹ Prostate shape differences exist between MRI and TRUS due to deformations induced by the TRUS probe and, when present, the MRI endorectal coil.¹² Registration should be near real-time (<5 min) to minimize procedure time and maximize patient comfort. Manual intervention to guide MRI-TRUS fusion may increase procedure time; hence, reducing manual intervention is important.

In this paper, we present multiattribute probabilistic prostate elastic registration (MAPPER) to align MRI and TRUS images of the prostate without the need for manual intervention during biopsy. MAPPER involves the following: Module 1: prior to biopsy, segmenting the prostate on MRI; Module 2: during biopsy, calculating a multiattribute probabilistic map of the prostate on TRUS; Module 3: maximizing overlap between prostate segmentation on MRI and multiattribute probabilistic map on TRUS, to align MRI onto TRUS. MAPPER is well suited for MRI-TRUS fusion as it automatically determines prostate location on TRUS (Module 2). Elastic registration (Module 3) enables MAPPER to account for differences in prostate deformation on MRI and TRUS.

The remainder of the paper is organized as follows. This section describes previous work on MRI-TRUS fusion and the novel contributions of MAPPER. Section 2 details the MAPPER algorithm. Section 3 provides results and Sec. 4 provides discussion of MAPPER. Section 5 gives concluding remarks.

1.A. Previous work in MRI-TRUS fusion

Most state-of-the-art MRI-TRUS fusion methods require manual intervention to locate the prostate.^9,11,13–16 Manual intervention may involve prostate delineation or selection of fiducials on MRI and TRUS. MRI-TRUS fusion methods can be grouped into (a) fiducial,^11,13–15 (b) surface,⁹ and (c) model-based.¹⁶

Fiducial-based methods minimize the distance between corresponding fiducials on MRI and TRUS.^11,13–15 Bubley et al.¹³ manually selected fiducials to determine a rigid transformation between MRI and TRUS imagery in 30 previously diagnosed prostate cancer patients. Sixteen patients (53%) had a positive biopsy core obtained from a cancer suspicious region on MRI. Mitra et al.¹¹ extracted fiducials from the surface and internal regions of the prostate to determine a diffeomorphic transformation between MRI and TRUS. Xu et al.¹⁴ used fiducials extracted from the prostate surface on MRI and TRUS to determine an affine transformation. Pinto et al.¹⁷ used this method to diagnose prostate cancer in 55 out of 101 patients. Reynier et al.¹⁵ used fiducials extracted from a manual prostate segmentation to calculate an elastic transformation. Using this method, prostate cancer was detected in 54 of 80 patients with highly suspicious MRI findings.¹⁰

Surface-based methods minimize the distance between the prostate surface on MRI and TRUS.⁹ Natarajan et al.⁹ used thin plate splines (TPS) to align prostate surfaces, where surfaces were obtained via semiautomated segmentation requiring manual selection of 4–6 fiducials. In 56 patients, this method had a cancer detection rate of 23% compared to 7% for nontargeted biopsies.⁹ A follow-up study found a detection rate of 53% in 171 men; 15 of 16 patients with highly suspicious MRI findings had positive biopsies.¹⁸

Hu et al.¹⁶ performed model-based MRI-TRUS fusion where a finite element model (FEM) of the prostate on MRI was used to align MRI to TRUS. Model initialization on TRUS required manual selection of two fiducials, on the prostate base and apex. Dickinson et al.¹⁹ used a variation of this method, requiring the manual placement of 10–20 fiducials on the prostate surface, to guide ablation of localized prostate cancer in 26 patients. Registration took 3–16 min.

State-of-the-art MRI-TRUS fusion methods rely on manual interaction to identify the prostate on TRUS and MRI by selecting fiducials or delineating boundaries.^9,11,13–16 Manual intervention during biopsy may increase procedure time, patient discomfort, and registration variation. Interobserver variability of prostate delineation on MRI is reported to be 2.5 ± 1.2 mm.¹² Variability in selecting fiducials or delineating boundaries may introduce registration error; however, we are unaware of any study that has explicitly studied this issue.

1.B. Novel contributions of multiattribute probabilistic prostate elastic registration

MAPPER is an improvement over state-of-the-art MRI-TRUS fusion methods as it requires no manual interaction during biopsy. MAPPER provides two novel contributions: (1) a method to estimate prostate location on TRUS and (2) a registration metric to align a segmentation (on MRI) to a probabilistic map (on TRUS).

MAPPER estimates prostate location on TRUS by calculating a probabilistic map combining texture and spatial information pertaining to prostate appearance and location. The method is motivated by the utility of texture and spatial information to segment the prostate.^20,21 Similarly, in this work, a probabilistic map of prostate location on TRUS is leveraged for registration. The spatial probability calculated from a set of training images describes the likelihood of a pixel being prostate according to location relative to the TRUS probe. The texture probability, calculated as a Gaussian distribution from a set of texture features, describes the likelihood of a pixel being prostate according to appearance.

The use of texture features makes MAPPER sensitive to TRUS appearance. Hence, consistent TRUS appearance, in terms of pixel intensity and texture characteristics, is important. TRUS may have attenuation artifacts caused by signal loss as ultrasound waves propagate through tissue²² resulting in pixels closer to the TRUS probe appearing brighter than pixels far away. As the TRUS probe is circular, attenuation will be along radial lines from the probe. TRUS attenuation correction, to account for signal loss, has been demonstrated to improve cardiac segmentation.²² In this work, attenuation correction is utilized for improved registration.

MAPPER leverages a novel registration metric to align a prostate segmentation onto a probabilistic map of prostate location. The motivation behind the metric is twofold: (1) aligning prostate surfaces on MRI and TRUS provides accurate registration⁹ and (2) TRUS texture and spatial information can accurately segment the prostate.²¹ Hence, aligning MRI and TRUS may be possible by leveraging TRUS image features directly. Our novel metric returns high values for transformations that align the prostate segmentation and pixels likely to be prostate, while returning low values where the segmentation aligns with pixels unlikely to be prostate.

2. METHODS

2.A. Notation

A 3D MRI volume 𝒞_M = (C_M, f_M) is defined by a set of 3D Cartesian coordinates C_M and an image intensity function f_M(c) : c ∈ C_M. The corresponding 3D prostate segmentation is defined as ${\mathcal{M}}_M=(C_M,g_M)$, where g_M(c) = i for a pixel c of class i. Class i = 1 indicates prostate and i = 0 background. Similarly, a 3D TRUS volume is defined as 𝒞_T = [C_T, f_T]. From 𝒞_T, a probabilistic map 𝒞_P,i = (C_T, P_i(d)) is calculated, where P_i(d) : d ∈ C_T is the probability of pixel d being in class i. Table I lists notation used in this paper. Figure 1 displays a flowchart of MAPPER comprising the following:

TABLE I.

Description of notation used in this paper.

Notation	Description	Notation	Description
𝒞M	3D MRI image scene	Pi[FT(d)]	Probability of FT(d) being class i ∈ {0, 1}
CM	3D pixel grid of 𝒞M	ΩT,i	Collection of pixels from class i in CT
${\mathcal{M}}_M$	3D MRI prostate segmentation	μF,i	Mean vector of FT(d) for ΩT,i
𝒞T	3D TRUS image scene	ΣF,i	Covariance matrix of FT(d) for ΩT,i
CT	3D pixel grid of 𝒞T	${\hat{\mathcal{M}}}_T$	Estimated TRUS prostate segmentation
${\mathcal{M}}_T$	3D TRUS prostate segmentation	T	Transformation function
fT(d)	TRUS image intensity function for d ∈ CT	S[T(ℳM), 𝒞T]	Similarity metric for $T\left({\mathcal{M}}_M\right)$ and 𝒞T
Pi(d)	Probability of d ∈ CT being in class i ∈ {0, 1}	R(T)	Regularization metric for T
(r, θ, z)	Polar coordinates of d ∈ CT	ΩM,i	Collection of pixels from class i in CM
${\tilde{f}}_T\left(d\right)$	Attenuation corrected intensity function for d ∈ CT	p	B-spline knot location in CT
Pi(d)	Spatial probability for d ∈ CT	E[p]	Expected location of p
FT(d)	Texture feature set for d ∈ CT	$\mathcal{N}\left(p\right)$	B-spline neighbor knots of p

FIG. 1.

Flowchart of the MAPPER algorithm. (1) Segment prostate on MRI using a minimally interactive algorithm (Ref. 20) (segmentation in pink); (2) construct multiattribute probabilistic map of prostate on TRUS (blue corresponds to pixels least likely to be prostate, red to pixels most likely to be prostate). The probabilistic map combines (a) prostate location (spatial probability) and (b) prostate appearance (texture probability). (3) Register MRI segmentation to TRUS probabilistic map by (a) affine (translation, rotation, scale) then (b) elastic transformation.

• •Module 1: Segment the prostate on MRI via a minimally interactive algorithm.²⁰
• •Module 2: Calculate a multiattribute probabilistic map of prostate location on TRUS. As an initial step, attenuation correction is performed. The probabilistic map is calculated by (a) determining a spatial probability to describe prostate location, (b) calculating a texture probability to describe prostate appearance, and then combining spatial and texture probabilities.
• •Module 3: Register MRI segmentation and TRUS probabilistic map. Registration comprises (a) an affine transformation to account for translation, rotation, and scale then (b) an elastic transformation to account for prostate deformations.

2.B. Module 1: Prostate segmentation on MRI

The prostate is segmented with a semiautomated multifeature appearance (MFA) algorithm.²⁰ We briefly summarize the algorithm here.

• (1)Select bounding box: A bounding box containing the prostate is manually selected.
• (2)Calculate segmentation: The MFA algorithm calculates a segmentation in the bounding box using shape and appearance as described in the approach of Toth and Madabhushi.²⁰
• (3)Refine segmentation: Fiducials are manually selected on the prostate surface. The MFA algorithm is constrained to ensure the fiducials are on the prostate surface.
• (4)Iterative refinement: Steps 2 and 3 repeated until segmentation of the prostate is accurate.

2.C. Module 2: Probabilistic map of prostate location on TRUS

Attenuation correction²² is performed on 𝒞_T to account for spatial variation in image intensities. A probabilistic map of prostate on TRUS is calculated by (1) extracting texture features defined as F_T(d) from 𝒞_T and (2) estimating prostate location (spatial probability) and appearance (texture probability).

2.C.1. Attenuation correction

Attenuation correction is performed using an unsupervised algorithm similar to Cohen et al.²³ in the polar coordinate frame to account for TRUS probe geometry. Each pixel d ∈ C_T is defined by 3D Cartesian coordinates (x, y, z) such that the probe center is (0, 0, 0). Corresponding polar coordinates are calculated as,

$${\displaystyle r=x^2+y^2,}$$

$${\displaystyle \theta ={tan}^{-1}\left(\frac{x}{y}\right),}$$

$${\displaystyle z=z.}$$ (1)

Image attenuation is modeled in polar coordinates as

$${\displaystyle f_T(r,\theta ,z)=\beta (r,\theta ,z){\tilde{f}}_T(r,\theta ,z)+\eta (r,\theta ,z),}$$ (2)

where ${\tilde{f}}_T(r,\theta ,z)$ is the true, unknown signal at location (r, θ, z). η(r, θ, z) is modeled as additive white Gaussian noise assumed to be independent of ${\tilde{f}}_T(r,\theta ,z)$ as described in Xiao et al.²² β(r, θ, z) is estimated by convolving a Gaussian kernel with the image, i.e., a low-pass filter (lpf) of the image. The true signal is then recovered by

$${\displaystyle {\tilde{f}}_T(r,\theta ,z)=exp\{log\left[f_T(r,\theta ,z)\right]-\hspace{0.17em}\text{lpf}(log\left[f_T(r,\theta ,z)\right])\}.}$$ (3)

Finally, ${\tilde{f}}_T(r,\theta ,z)$ is converted to 3D Cartesian coordinates, ${\tilde{f}}_T\left(d\right)$. Figure 2 illustrates a study where attenuation correction improved registration by over 1 mm.

FIG. 2.

The panels demonstrate the importance of attenuation correction. TRUS (a) without and (a) with attenuation correction. (b) and (f) median texture feature; (c) and (g) probability maps, where blue corresponds to pixels least likely prostate and red to pixels most likely prostate; (d) and (h) checkerboard images of aligned MRI and TRUS. In (d) and (h), light gray arrows show aligned regions and dark gray arrows show misaligned regions. Red ellipses in (b) and (f) show a region where attenuation correction increased the difference between prostate and background pixels. Black ellipses in (c) and (g) show the corresponding region on the probability maps.

2.C.2. Feature extraction

For each pixel ${\tilde{f}}_T\left(d\right):d\in C_T$, a set of texture features F_T(d) are calculated for a neighborhood region 𝒩(d) : d∈𝒞_T. Texture features may include (a) intensity (intensity, mean, median), (b) intensity spread (range), (c) intensity variation (variance, Rayleigh, or the Nakagami m-parameter), and (d) edge information (Gabor wavelet).

The mean feature is calculated as $f_m\left(d\right)=(1/|\mathcal{N}\left(d\right)\left|\right){\sum }_{n\in \mathcal{N}\left(d\right)}{\tilde{f}}_T\left(n\right)$. Similarly, the median feature f_d(d) is calculated by applying the median filter operator over $\mathcal{N}\left(d\right)$. The range feature f_r(d) is calculated as $f_r\left(d\right)={max}_{n\in \mathcal{N}\left(d\right)}\left({\tilde{f}}_T\left(n\right)\right)-{min}_{n\in \mathcal{N}\left(d\right)}\left({\tilde{f}}_T\left(n\right)\right)$.

Features which measure intensity variation assume a specific distribution to calculate intensity variation within $\mathcal{N}\left(d\right)$. For instance variance assumes a Gaussian distribution, and is calculated as

$${\displaystyle f_v\left(d\right)=\sqrt{\frac{1}{\left|\mathcal{N}\left(d\right)\right|}\sum _{n\in \mathcal{N}\left(d\right)}{({\tilde{f}}_T\left(n\right)-f_m\left(d\right))}^2}.}$$ (4)

Rayleigh, assumes the distribution to described well formed ultrasound scatter and is estimated by

$${\displaystyle f_y\left(d\right)=\sqrt{\frac{1}{2\left|\mathcal{N}\left(d\right)\right|}\sum _{n\in \mathcal{N}\left(d\right)}{\tilde{f}}_T{\left(n\right)}^2}.}$$ (5)

The Nakagami m-parameter defined as f_n describes the shape of a distribution that is generalizable across different ultrasound scatter conditions. The Nakagami m-parameter is calculated using the iterative method of Greenwood and Durand.²⁴

Edge information is calculated from a set of Gabor wavelets.²⁵ Gabor wavelets convolved with TRUS imagery return high values for strong edges and low values for weak edges. The feature set F_T(d) is a subset of [f_m, f_d, f_r, f_v, f_y, f_n, f_g].

2.C.3. Calculating probability map of prostateon TRUS

P_i(d), the probability of pixel d being in class i, depends on the spatial location d and the feature set F_T(d). The probability of a location d belonging to class i is defined as P_i(d). Similarly, the probability of the feature set F_T(d) being in class i is P_i[F_T(d)]. We assume P_i(d) and P_i[F_T(d)] are independent. Hence, P_i(d) is calculated as

$${\displaystyle {\mathbf{P}}_i\left(d\right)=P_i\left[F_T\left(d\right)\right]\times P_i\left(d\right).}$$ (6)

The calculation of P_i(d) and P_i[F_T(d)] is described below.

2.C.3.a. Spatial probability.

P_i(d), the likelihood of pixel d being in class i based on spatial location, is calculated as the frequency of pixel d being in the prostate across J training studies 𝒞_T,j : j ∈ {1, …, J}. For each training study, an expert radiologist manually delineated the prostate yielding the 3D prostate segmentation ${\mathcal{M}}_{T,j}$. A prostate segmentation ${\mathcal{M}}_{T,j}=(C_T,g_{T,j})$ is defined such that g_T,j(d) = i for pixel d in class i. Each study is defined such that the center of the TRUS probe is the origin to ensure the location of pixel d is consistent relative to the probe over all studies. P_i(d) is then estimated as

$${\displaystyle P_i\left(d\right)=\frac{1}{J}\sum _{j=1}^J{g}_{T,j}\left(d\right).}$$ (7)

2.C.3.b. Texture probability.

The probability P_i[F_T(d)] is the likelihood of the feature set F_T(d) being in class i. F_T(d) is modeled as a multivariate Gaussian distribution with a mean vector μ_F,i and covariance matrix Σ_F,i for class i. Given the distribution parameters μ_F,i and Σ_F,i, the probability P_i[F_T(d)] is calculated as

$${\displaystyle P_i\left[F_T\left(d\right)\right]=\frac{1}{2{\pi }^{k/2}{\mathrm{\Sigma }}_{F,i}^{1/2}}{e}^{{(F_T\left(d\right)-{\mu }_{F,i})}^{\prime }{\mathrm{\Sigma }}_{F,i}^{-1}(F_T\left(d\right)-{\mu }_{F,i})},}$$ (8)

where k is the number of features in F_T(d).

The parameters μ_F,i and Σ_F,i are unknown and therefore must be estimated. First, the prostate location on TRUS is estimated by assuming an initial rigid transformation T_r (Sec. 2.D) to determine the estimated prostate segmentation defined as ${\hat{\mathcal{M}}}_T=T_r\left({\mathcal{M}}_M\right)$, where ${\hat{\mathcal{M}}}_T=(C_T,{\hat{g}}_T)$ and ${\hat{g}}_T\left(d\right)=i$ for a pixel d estimated to be in class i. Then, μ_F,i and Σ_F,i are calculated by ${\mu }_{F,i}=(1/|{\mathrm{\Omega }}_{T,i}\left|\right)\sum _{d\in \underset{T,i}{\mathrm{\Omega }}}{F}_T\left(d\right)$, where Ω_T,i is the collection of pixels in C_T in class i according to ${\hat{g}}_T\left(d\right)$. Similarly, Σ_F,i is the covariance matrix of F_T(d) for Ω_T,i.

2.D. Module 3: Registration of MRI segmentationand TRUS probabilistic map

A transformation T is found to spatially align 𝒞_M, using ${\mathcal{M}}_M$, onto 𝒞_T via the equation

$${\displaystyle T=\underset{T}{\text{argmax}}[S\left[T\left({\mathcal{M}}_M\right){,\mathcal{C}}_T\right]-\alpha R\left(T\right)],}$$ (9)

where S(⋅, ⋅ ) is a similarity metric between $T\left({\mathcal{M}}_M\right)$ and 𝒞_T. R(T) is a regularization metric that penalizes nonsmooth T. The parameter α controls the weight of R(⋅) relative to S(⋅, ⋅ ). The similarity metric S(⋅, ⋅ ) is calculated as

$${\displaystyle S\left[T\left({\mathcal{M}}_M\right){,\mathcal{C}}_T\right]={\mathrm{\Pi }}_{i=0}^1{\mathrm{\Pi }}_{d\in C_T}\left[P_i[F\left(d\right),d]\right|T\left({\mathcal{M}}_M\right)={\mathrm{\Omega }}_{M,i}],}$$ (10)

where Ω_M,i is the collection of pixels in C_M that belong to class i. Registration is initialized with a rigid transformation T_r that maximizes overlap between ${\mathcal{M}}_M$ and P₁(d). T_r is calculated as

$${\displaystyle T_r=\underset{T_r}{\text{argmax}}\left[{\mathrm{\Pi }}_{d\in C_T}[P_1\left(d\right)\times T_r\left(g_M\left(c\right)\right)]\right].}$$ (11)

Next an affine transformation T_a is calculated with Eq. (10) setting α = 0. Regularization is unnecessary as T_a is by definition smooth. Finally, an elastic B-spline transformation T_e (Ref. 26) is calculated. R(T) is defined as described in Sec. 2.D.1.

2.D.1. Regularization metric

R(T) constrains T_e to transformations which are likely to occur and is calculated as

$${\displaystyle R\left(T\right)=\sum _{p\in T}(1-e^{-\Vert p-E\left[p\right]\Vert }),}$$ (12)

where p ∈ C_M is the location of a B-spline knot and E[p] is the maximum likelihood estimate of where p should be located. In this work, E[p] is estimated as

$${\displaystyle E\left[p\right]=\frac{1}{\left|\mathcal{N}\left(p\right)\right|}\sum _{q\in \mathcal{N}\left(p\right)}q,}$$ (13)

where $\mathcal{N}\left(p\right)$ is the set of knots which neighbor p. E[p] is the mean over the set of neighbor knots for p. Figure 3 gives a 2D illustration of the regularization metric. For MAPPER the regularization metric is calculated in 3D.

FIG. 3.

An illustration of the regularization metric R(T). Black points correspond to B-spline knots. (a) The initial location of a knot of interest p¹ (red square) determines the neighborhood knots $\mathcal{N}\left(p^1\right)$ (green circles). The expected location E[p¹] (blue triangle) is calculated for p¹. (b) Example where R(T) returns a high value as p¹ is far from E[p¹]. (c) and (d) return low R(T) values as p¹ is near E[p¹]. (d) Deformations far from p¹ are not taken into account when considering E[p¹], other knots may contribute to a higher R(T).

If p = E[p], then p will not contribute R(T). As p moves farther from E[p], the value of (1 − e^{−‖p−E[p]‖}) increases, and R(T) increases. Hence R(T) is lower for evenly spaced, smoothly varying knots compared to randomly spaced, erratically varying knots. During registration, deformations that are not evenly spaced and smoothly varying will only occur if the increase in S(⋅, ⋅ ) is greater than the increase in R(T).

3. RESULTS

3.A. Experimental design

3.A.1. Dataset description

MAPPER was evaluated on two datasets described below. For all studies, one or more expert radiologist manually selected corresponding fiducials on MRI and TRUS. Fiducials included the urethra, verunmontanum, ejaculatory ducts, regions suspicious for prostate cancer, and calcifications.

3.A.1.a. Dataset 1 (${\mathcal{D}}_1$): Side-firing transrectal probe.

${\mathcal{D}}_1$ was obtained at University College London Hospital prospectively from human research subjects with IRB approval. T2-weighted (T2w) MRI was acquired using a Siemens 1.5 T scanner and a pelvic phased-array coil for 6 patients. TRUS imagery was acquired with a B-K Profocus probe that obtains 2D transverse B-mode images. The TRUS probe was attached to a mechanical stepping device that translates the probe perpendicular to the axial plane at 2 mm intervals. The TRUS origin was set to the center of rotation for the transverse direction and the middle image acquired along the mechanical stepper. For each patient, one TRUS volume was acquired consisting of a set of parallel B-mode slices. An expert radiologist E₁ selected corresponding fiducials on all 6 studies.

3.A.1.b. Dataset 2 (${\mathcal{D}}_2$): Volumetric end-firing transrectal probe.

${\mathcal{D}}_2$ was obtained from Boston Medical Center prospectively, from human research subjects with IRB approval. T2w MRI was acquired using a General Electric 3.0 T scanner and an endorectal coil for 7 patients. TRUS imagery was acquired using a GE 4DE7C probe that acquires 3D data in a single, multiplane fan-beam sweep. The TRUS origin was set to the center of rotation for both the transverse and sagittal directions. For each patient, 1–3 volumes were acquired, each volume consisting of a single volumetric image. A total of 13 MRI-TRUS pairs were acquired for 7 patients. Two expert radiologists selected corresponding fiducials, E₁ for 10 studies and E₂ for 5 studies.

3.A.2. Performance evaluation: Root-mean-square error (RMSE)

RMSE measures how well two sets of fiducials align; a RMSE of 0 represents perfect alignment. A set of fiducials on MRI is defined $p_M^i:i\in \{1,\dots ,N\}$. Similarly, a set of fiducials on TRUS is defined as $p_T^i:i\in \{1,\dots ,N\}$, such that $p_M^i$ corresponds to $p_T^i$. RMSE is calculated as $\sqrt{(1/N)\sum _{i=1}^N{(\underset{M}{\overset{i}{p}}-\underset{T}{\overset{i}{p}})}^2}$.

3.A.3. Implementation details

All methods were implemented using the Insight Segmentation and Registration Toolkit (ITK) version 4.5.²⁷ Texture features were calculated using $\mathcal{N}\left(d\right)$ of a spherical neighborhood of 1 mm³, determined empirically to accurately capture local image statistics. Training studies for the spatial probability were chosen from the same dataset as the images undergoing registration. A patient-based leave-one-out cross validation scheme was employed. Hence, all images acquired from the patient being used for registration were excluded from the training set in order to reduce bias in the registration algorithm. T_a and T_e were found via a Powell optimization scheme using a single resolution.²⁸

3.B. Experiment 1: Effect of attenuation correction

Spatially inconsistent TRUS appearance may lead to the probabilistic map P_i(d) inaccurately estimating the prostate location and result in registration errors. MAPPER registration accuracy with and without attenuation correction was assessed by RMSE for ${\mathcal{D}}_1$.

Figure 4 provides RMSE for MAPPER with and without attenuation correction for five texture features: intensity (f_T), median (f_d), variance (f_v), Rayleigh (f_y), and Gabor wavelets (f_g). Attenuation correction has two effects: (1) it reduces RMSE variance between studies giving a more robust registration and (2) it lowers RMSE providing a more accurate registration. Effects of attenuation correction on registration occur for all features.

FIG. 4.

RMSE for five texture features [intensity (f_T), median (f_d), variance (f_v), Rayleigh (f_y), and Gabor wavelets (f_g)] with and without attenuation correction. Each dot represents RMSE for an individual study. From top to bottom, the horizontal lines denote 25%, 50%, and 75% RMSE and the vertical lines denote 5% and 95% RMSE.

3.C. Experiment 2: Effect of texture feature

P_i(d) is dependent on the features in F_T(d); features better able to distinguish prostate from background more accurately represent prostate location and result in a more accurate registration. Registration accuracy, assessed by RMSE, was evaluated for the seven features described in Sec. 2.C.2.

Figure 5 displays RMSE or (a) ${\mathcal{D}}_1$ and (b) ${\mathcal{D}}_2$ evaluated on six texture features: intensity (f_T), median (f_d), variance (f_v), Rayleigh (f_y), Gabor wavelets (f_g), and the three features with the lowest RMSE (F_T). Each dataset had different best performing features selected in F_T. For ${\mathcal{D}}_1$, the side-firing TRUS probe, the best performing features were f_d, f_g, and f_v. For ${\mathcal{D}}_2$, the end-firing TRUS probe, f_T, f_y, and f_v were the best performing features. Different best performing features for ${\mathcal{D}}_1$ and ${\mathcal{D}}_2$ likely reflect different TRUS imagery characteristics between the datasets and highlight the importance of feature selection for MAPPER. Representative studies are shown in Fig. 6 for ${\mathcal{D}}_1$ and Fig. 7 for ${\mathcal{D}}_2$. In both studies, MAPPER aligns the prostate surface and internal structures indicated by dotted lines.

FIG. 5.

RMSE for (a) ${\mathcal{D}}_1$ and (b) ${\mathcal{D}}_2$ evaluated on six texture features [intensity (f_T), median (f_d), variance (f_v), Rayleigh (f_y), Gabor wavelets (f_g), and the three features with the lowest RMSE (F_T)]. Each dot represents RMSE for an individual study. From top to bottom, the horizontal lines denote 25%, 50%, and 75% RMSE and the vertical lines denote 5% and 95% RMSE.

FIG. 6.

An example MRI-TRUS registration for a study in ${\mathcal{D}}_1$ with a RMSE of 3.57 mm. Prostate apex for corresponding (a) MRI, (b) TRUS, (c) and checkerboard overlay of the two modalities. Dotted lines delineate the central gland on MRI and TRUS. Similarly, for the prostate midgland, (d) MRI, (e) TRUS, and (f) checkerboard overlay and for the prostate base on (g) MRI, (h) TRUS, and (i) checkerboard overlay.

FIG. 7.

An example MRI-TRUS registration for a study in ${\mathcal{D}}_2$ with RMSE of 2.69 mm. Prostate apex where dotted lines delineate the central zone on (a) MRI, (b) TRUS, (c) and a checkerboard overlay of the two modalities. Similarly, for the prostate midgland, dotted lines delineate the lateral lobe on (d) MRI, (e) TRUS, and (f) checkerboard overlay. Finally, the prostate base where dotted lines delineate the central zone on (g) MRI, (h) TRUS, and (i) checkerboard overlay. TRUS imagery for (b) apex and (h) base appears cropped due to fan-beam shaped probe.

T_e improved RMSE compared to T_a for ${\mathcal{D}}_1$, where MRI was acquired with a pelvic phased-array coil. For ${\mathcal{D}}_2$, where MRI was acquired with an endorectal coil, T_e did not substantially improve RMSE over T_a. Differences in RMSE improvement between the datasets are indicative of ${\mathcal{D}}_1$ having larger differences in prostate deformation between MRI and TRUS compared to ${\mathcal{D}}_2$.

Additionally, for ${\mathcal{D}}_2$, we compared RMSE between fiducials selected by two expert radiologists to evaluate interobserver variability. Figure 8 displays the RMSE for the two expert radiologists. Differences in RMSE between the two expert radiologists is ≈0.3 mm for the best performing features (f_y and f_T).

FIG. 8.

RMSE for six texture features [intensity (f_T), median (f_d), variance (f_v), Rayleigh (f_y), Gabor wavelets (f_g), and the three features with the lowest RMSE (F_T)] calculated from fiducials selected by two expert observers for ${\mathcal{D}}_2$. Each dot represents the RMSE for an individual study, the horizontal lines from bottom to top denote the 25%, 50%, and 75% RMSE for that feature set. Vertical lines denote the 5% and 95% of the RMSE for that feature set.

3.D. Experiment 3: Effects of MRI Segmentation

To evaluate the effect prostate segmentation has on MAPPER registration accuracy, the prostate was segmented with different levels of manual interaction via the following strategies.

• •Bounding box (B): MFA model with manual bounding box selection.
• •Fiducials (F): MFA model with manual bounding box selection and fiducial selection.
• •Delineation (D): Expert radiologist manual delineation of the prostate.

Figure 9 displays registration accuracy, in terms of RMSE, for different segmentation strategies. Manual prostate delineation, the most accurate segmentation, resulted in the lowest RMSE. Manual selection of the bounding box and fiducials lowered RMSE compared to manual selection of only the bounding box.

FIG. 9.

RMSE evaluated for different prostate segmentation schemes for (a) ${\mathcal{D}}_1$ and (b) ${\mathcal{D}}_2$ using f_T. Each dot represents RMSE for an individual study. From top to bottom, the horizontal lines denote 25%, 50%, and 75% RMSE and the vertical lines denote 5% and 95% RMSE.

3.E. Experiment 4: Computational time

The time required to register MRI onto TRUS is important, increased registration time may lead to a longer biopsy procedure. For each dataset, the time to perform each registration steps initial rigid alignment (T_r), affine alignment (T_a), and elastic alignment (T_e) was recorded.

Experiments were run on a computer with a 3.0 GHz Xeon Quad-Core processor and 1 GB of RAM. Table II reports the time in minutes to register MRI onto TRUS. ${\mathcal{D}}_1$ and ${\mathcal{D}}_2$ have comparable times, with T_e taking the longest time.

TABLE II.

Time in minutes to register MRI onto TRUS reported as mean ± standard deviation.

Registration step	${\mathcal{D}}_1$ time (min)	${\mathcal{D}}_2$ time (min)
Tr	0.26 ± 0.05	0.45 ± 0.15
Ta	0.13 ± 0.02	0.30 ± 0.04
Te	69.75 ± 22.86	68.31 ± 23.75

4. DISCUSSION

MAPPER has a RMSE of 3.14–3.36 mm for the two datasets considered. Comparing the registration accuracy of MAPPER against state-of-the-art methods^9,11,14–16 is difficult due to differences in image acquisition, ground truth determination, and evaluation strategies. The RMSE of MAPPER is similar to the RMSE reported for state-of-the-art methods.^9,11,14–16 However, making conclusive statements on the relative accuracy of any MRI-TRUS fusion algorithms would require evaluating each algorithm on the same dataset. To the best of our knowledge, no such comparative study has been performed.

The current implementation of MAPPER takes roughly 1 h for registration, too long to be used clinical for biopsy guidance. In this work, MAPPER was implemented with a single resolution Powell optimization scheme. By leveraging multiresolution registration and a faster optimization scheme, such as gradient descent, registration time may be greatly reduced.

Figure 10 displays a prostate surface rendering for one study showing regions of MRI misalignment external and internal to the TRUS. For this study, there are two regions of misalignment: (1) near the rectal wall and (2) near the bladder. Figure 10 shows a 2D axial TRUS image with a cross section of the surface rendering shown in Fig. 10 and the true prostate surface. The hyperechoic region distal to the TRUS probe results in P_i(d) being unable to model prostate location and causing a RMSE of ≈4 mm. Similarly, Fig. 10(c) displays another 2D axial TRUS image with the surface rendering cross section and the true prostate boundary. For this study, misalignment is less pronounced, a RMSE of ≈1 mm. The misalignment near the rectal wall is caused by T_e being unable to recover the differences in prostate deformation.

FIG. 10.

(a) Prostate surface rendering with prostate base facing right; shaded according to misalignment external and internal to the TRUS prostate surface. 2D axial TRUS image with a region of misalignment (b) distal to the TRUS probe and (c) near the TRUS probe. The thin line shows the manual prostate surface delineation.

Poor TRUS image quality negatively impacts the registration accuracy of MAPPER due to the reliance on TRUS image appearance. Figure 11 shows two example studies, one from each dataset, where poor TRUS image quality resulted in inaccurate registration. Both of these studies were outliers, in terms of poor image quality and/or large deformation in the prostate.

FIG. 11.

Two studies where poor TRUS image quality resulted in inaccurate registration. (a)–(c) A study from ${\mathcal{D}}_1$ with a RMSE of 5.47 mm where (b) TRUS intensity appears blurry with poor definition of the prostate boundary. The artifacts result in MAPPER being unable to determine prostate location on TRUS, leading to (c) misalignment (ellipse). (d)–(f) A study from ${\mathcal{D}}_2$ with a RMSE of 4.27 mm where (e) TRUS has abnormal prostate deformation on the right side of the image. MAPPER cannot account for this deformation resulting in (f) registration errors (ellipse).

The spatial probability in MAPPER controls the variation of prostate deformation able to be recovered. MAPPER most likely requires the inclusion of pathologic prostate imagery in the training set to register highly pathologic prostate images (e.g., protrusion of the prostate into surrounding tissue, extracapsular spread of prostate cancer). However, a thorough investigation of the effect of the training set is necessary to determine the generalizability of MAPPER.

5. CONCLUSION

In this work, we present MAPPER, a novel prostate MRI-TRUS fusion algorithm. MAPPER was evaluated on 13 patient studies from two datasets. Dataset 1 had six studies with a side-firing TRUS probe and 1.5 T pelvic phased-array coil MRI. Dataset 2 had seven studies with a volumetric end-firing TRUS probe and 3.0 T endorectal coil MRI. RMSE for MAPPER was 3.36 ± 1.10 mm for Dataset 1 and 3.14 ± 0.75 mm for Dataset 2. MAPPER uses a semiautomated segmentation scheme on MRI and a probabilistic map of prostate location on TRUS to perform registration without manual intervention during the biopsy procedure. In comparison, state-of-the-art methods require manual intervention to delineate the prostate or select fiducials on MRI and TRUS to guide registration.^9,11,14–16

A limitation of this work is the use of a B-spline transformation in Module 3 (Sec. 2.D) to account for differences in prostate deformation between MRI and TRUS. In this work, a regularization metric ensured the underlying deformation in the prostate was smoothly varying. However, other transformations such as FEMs, which allow for explicit modeling of tissue physics, could enable a more realistic deformation between MRI and TRUS.¹⁶ Future work will evaluate other transformations and regularization metrics to model prostate deformation.

MAPPER is reliant on an accurate prostate segmentation on MRI as demonstrated in Experiment 3. In this work, the prostate segmentation is performed offline prior to biopsy using a MFA model of prostate appearance as described in Toth and Madabhushi.²⁰ Future work will be directed toward evaluating in detail the performance of MAPPER for (a) independent manual prostate delineations and (b) different prostate segmentation algorithms.