Collimator optimization in myocardial perfusion SPECT using the ideal observer and realistic background variability for lesion detection and joint detection and localization tasks

Abstract

In SPECT imaging, collimators are a major factor limiting image quality and largely determine the noise and resolution of SPECT images. In this paper, we seek the collimator with the optimal tradeoff between image noise and resolution with respect to performance on two tasks related to myocardial perfusion SPECT: perfusion defect detection and joint detection and localization. We used the Ideal Observer (IO) operating on realistic background-known-statistically (BKS) and signal-known-exactly (SKE) data. The areas under the receiver operating characteristic (ROC) and localization ROC (LROC) curves (AUC_d, AUC_d+l), respectively, were used as the figures of merit for both tasks. We used a previously developed population of 54 phantoms based on the eXtended Cardiac Torso Phantom (XCAT) that included variations in gender, body size, heart size and subcutaneous adipose tissue level. For each phantom, organ uptakes were varied randomly based on distributions observed in patient data. We simulated perfusion defects at six different locations with extents and severities of 10% and 25%, respectively, which represented challenging but clinically relevant defects. The extent and severity are, respectively, the perfusion defect's fraction of the myocardial volume and reduction of uptake relative to the normal myocardium. Projection data were generated using an analytical projector that modeled attenuation, scatter, and collimator-detector response effects, a 9% energy resolution at 140 keV, and a 4 mm full-width at half maximum (FWHM) intrinsic spatial resolution. We investigated a family of eight parallel-hole collimators that spanned a large range of sensitivity-resolution tradeoffs. For each collimator and defect location, the IO test statistics were computed using a Markov Chain Monte Carlo (MCMC) method for an ensemble of 540 pairs of defect-present and -absent images that included the aforementioned anatomical and uptake variability. Sets of test statistics were computed for both tasks and analyzed using ROC and LROC analysis methodologies. The results of this study suggest that collimators with somewhat poorer resolution and higher sensitivity than those of a typical low-energy high-resolution (LEHR) collimator were optimal for both defect detection and joint detection and localization tasks in myocardial perfusion SPECT for the range of defect sizes investigated. This study also indicates that optimizing instrumentation for a detection task may provide near-optimal performance on the more challenging detection-localization task.

1. Introduction

In Single Photon Emission Computed Tomography (SPECT), the collimator establishes a relationship between positions in the image projection plane and directions in space that largely determines the spatial resolution. This directional information is only imperfectly obtained, and at the cost of a loss of sensitivity and a resulting increase in quantum noise in the images. The tradeoff between resolution and noise has a significant impact in determining the diagnostic accuracy of readers interpreting the images.

Collimator properties including hole length, hole diameter and septal thickness determine the tradeoff between resolution and sensitivity. Techniques for selecting collimator parameters for a given geometric resolution are well established (Keller 1968, Swann et al 1976, Beck and Redtung 1985). Typically, a desired spatial resolution and maximum permissible single-septal penetration probability at some specified energy are specified a priori, and the collimator parameters giving the highest sensitivity are then determined. There are, however, important limitations to this approach regarding how to optimize the resolution–sensitivity trade-off for a given task, such as estimation, i.e. quantifying one or more parameters of interest using the given image data (Lau et al 2001, Inoue et al 2004, Moore et al 2005, McQuaid et al 2011), or classification, i.e. deciding to which class an image belongs (Tsui 1978, Tsui et al 1983, Myers et al 1990, Moore et al 1995, 2005, Narayanan et al 2002, Zeng and Gullberg 2002, Gross et al 2003, Zhou and Gindi 2009). In this work, we focus on two particular classification tasks: binary signal detection and the arguably more clinically realistic joint detection and localization task. In a binary detection task, an observer is asked to classify a given image as either containing or not containing a defect. In the joint detection and localization task, the observer is required to jointly detect the presence of a signal and estimate its location.

Performing task-based system optimization requires specification of a task, an observer, and a scalar figure of merit (FOM) that quantifies the performance of the observer on the task. One obvious way to perform task-based assessment of medical image quality is with a human observer study. However, human observer studies are time consuming and expensive. Thus, in addition to (and frequently instead of) using human observers, model observers are often used to evaluate image quality (Barrett et al 1993, Barrett and Myers 2004).

Two important categories of model observers are ideal and anthropomorphic observers. Anthropomorphic observers, such as the Channelized Hotelling Observer (CHO) (Myers and Barrett 1987) and the recently developed visual search (VS) model observer (Gifford 2013), are designed to mimic human observer performance or performance rankings for various tasks, including detection (Yao and Barrett 1992, Wollenweber et al 1999, Gifford et al 2000, Abbey and Barrett 2001, Oldan et al 2004, Gilland et al 2006) and localization (Farncombe et al 2004, Gifford et al 2005, Gifford 2013, 2014). Anthropomorphic observers, especially the CHO, have shown good agreement with human observers, but applying them to reconstructed images is challenging because of the difficulty of estimating the noise properties of reconstructed images analytically, and the computational expense of estimating them using ensemble techniques. In this regard, the VS model observer for localization tasks is appealing, as it has been shown to predict human observer performance rankings without the need to estimate the noise covariance of the reconstructed images (Gifford 2013, 2014). However, both the CHO and VS are applied to reconstructed images, which means there is a need to optimize image reconstruction, even when only comparing, for example, different collimators. For example, the regularization parameters optimal for one collimator are not, in principle, optimal for another collimator.

The Ideal Observer (IO) has full knowledge of the imaging system and object statistics and makes optimal use of all the information in the image. IO performance can be estimated directly from the projection data, where noise statistics are well known and easy to characterize. Subsequent conventional (invertible) image processing or reconstruction steps do not improve its performance. In optimizing instrumentation and acquisition parameters (i.e. factors affecting task performance that precede reconstruction), it is helpful to use the IO because it identifies the parameter settings that maximize information about the task contained in the measured raw data. The role of the subsequent reconstruction, regularization, and compensation steps can then be conceptualized as matching the information in the raw data for optimal performance by a human observer. In (Ghaly et al 2015), we showed that sequentially optimizing acquisition parameters in the projection domain and then optimizing the reconstruction algorithm and compensation methods in the image domain yielded similar optimal operating ranges compared to optimizing the acquisition parameters and the reconstruction algorithm jointly, provided that good models of the image formation process are incorporated into the reconstruction algorithm. In that study, we used the IO and CHO to optimize Tc-99m acquisition energy window width and evaluate various scatter modeling and compensation methods, including the dual and triple energy window and the Effective Source Scatter Estimation (ESSE) method, in the context of a myocardial perfusion SPECT defect detection task. We compared between the optimal acquisition parameters obtained using the IO and those obtained using the CHO when applied to images reconstructed with scatter compensation using various scatter modeling methods with various degrees of fidelity to the true process. We found that when good scatter models, such as the ESSE method, were used in the reconstruction the optimal energy window for the IO and CHO were similar.

Despite the potential advantages of using the IO to evaluate and optimize imaging systems, it is very difficult to estimate IO performance for realistic tasks that model realistic objects and background variability arising from variations in patient anatomy and biokinetics. This is because the background model statistics are complex and difficult to analyze analytically. Kupinski et al (2003) developed a method to estimate the performance of the IO for realistic and general background-known-statistically and signal-known-exactly (BKS/SKE) paradigms. Park et al (2003) extended the method to estimate the performance of the IO in the case where both backgrounds and signals are random with known statistical properties (BKS/SKS). These methods use Markov Chain Monte Carlo (MCMC) techniques to estimate the IO test statistic, i.e. the likelihood ratios, for, in principle, arbitrarily complex objects and imaging systems. However, these methods require generating very large numbers of noise-free projections of the background for arbitrary values of background parameters. Thus, computation time considerations practically limit these methods to simplified system models and backgrounds.

He et al (2008) adapted Kupinski's method to backgrounds parameterized by a combination of discrete and continuous values. Discrete values were used to specify phantom anatomies and continuous values to specify organ uptake. In this method, projections of all organs for all the discrete anatomies are precomputed and stored in a projection library from which projections for any particular value of the background parameters can be obtained very rapidly by scaling and summing appropriate members of the library. This method was applied in myocardial perfusion SPECT imaging using a realistic background model consisting of a para meterized torso phantom and projections generated using a realistic analytical SPECT projector. In this work, we extended that technique by using a more complete and realistic phantom library. We then used the MCMC technique to estimate the IO test statistic, i.e. the likelihood ratio, for both binary detection and joint detection and localization tasks in the context of MPS operating on realistic simulated backgrounds including anatomical and uptake variability.

2. Methods

2.1. Background variability

To provide clinically relevant optimization of imaging system parameters, it is important that the objects be as realistic as possible and include anatomical and uptake variability that model a clinical population. We have previously designed, developed, and reported on a digital phantom population based on the 3D XCAT phantom (Ghaly et al 2014). The simulated population included 3 variations each in body size, heart size, and level of subcutaneous adipose tissue (fat level), resulting in a total of 27 phantoms for each gender. In addition to the anatomical variation, we modeled uptake variation by separately generating projection data for the heart, liver and body (including other organs). We could then scale and sum the individual projections to model any arbitrary uptake ratio and noise level. The projections were simulated using the attenuation distribution of the entire object, and the summing and scaling is simply a linear combination. The projections for each organ contained scatter, spillover and partial volume effects into the regions of other organs. Variations in organ uptake were modeled by randomly sampling the organ uptake activities from distributions derived from a set of 34 clinical studies of patients who underwent MPS (Ghaly et al 2014). In this study, we simulated a stress study of a standard MPS one-day stress/rest clinical protocol (Cerqueira et al 2010). We modeled an injected activity of 10 mCi of Tc-99m and a total acquisition time of 26 min.

2.2. Simulated perfusion defects

We simulated perfusion defects at six different locations, each with extent and severity of 10% and 25%, respectively. The extent and severity are, respectively, the perfusion defect fractional volume and reduction of uptake relative to the normal myocardium. This combination of extent and severity was chosen to be challenging but clinically relevant. For each phantom, we varied the axial and angular extents of the defects to achieve the aforementioned volume extent. Figure 1 shows each of the six defect locations (labeled as d₁ to d₆ from left to right) in the short axis slices containing the defect centroid. The perfusion defects, d₁ to d₅, were uniformly distributed over a mid-ventricular slice of the heart between the anterior and inferior walls through the lateral wall. The center of each successive defect was separated by 45°. We simulated an additional perfusion defect, d₆, centered at a mid-ventricular slice in the septal wall.

Figure 1.

Sample short axis images showing a normal heart (left) and a defective heart at six different locations of the myocardial wall (d₁ to d₆ from left to right). For illustrative purposes, defects shown have 100% severity.

To study the effect of the defect size on the observer performance, we also simulated perfusion defects at locations d₂ and d₅ with 5% and 25% extents (labeled as d_2,5%, d_2,25%, d_5,5% and d_5,25% respectively) in addition to the 10% extent defects, as shown in figure 2. For each defect, the product of the extent and severity was kept constant to model a constant overall reduction of the activity in the myocardium. Parameters of the simulated defects are reported in table 1. Since the extent was fixed and the heart sizes were different for the various phantoms, the axial length of the defect was different for the different heart and body sizes. Thus, table 1 gives the mean and standard deviation of the axial length.

Figure 2.

Sample short axis images showing a heart with a perfusion defect at two different locations (d₂ (top row) and d₅ (bottom row)) in the heart and extents of 5%, 10% and 25%, from left to right.

Table 1.

Parameters of simulated defects.

				Length of defect (cm)
Defect	Location	Extent (%)	Severity (%)	Mean	Std. dev.
d1	Anterior	10	25	3.6	0.5
d2	Antero-lateral	10	25	3.2	0.6
d2,5%	Antero-lateral	5	50	2.5	0.4
d2,25%	Antero-lateral	25	10	5.3	0.8
d3	Lateral	10	25	3.4	0.6
d4	Infero-lateral	10	25	2.8	0.5
d5	Inferior	10	25	2.2	0.4
d5,5%	Inferior	5	50	1.7	0.3
d5,25%	Inferior	25	10	3.6	0.6
d6	Septal	10	25	5.4	0.7

2.3. Projection data simulation

We generated projection data using an analytical projector that modeled attenuation, scatter, the collimator-detector response (CDR), a 9.5 mm thick NaI(Tl) crystal with an energy resolution of 9% at 140 keV (which we varied as one over the square root of energy), and a 4 mm intrinsic spatial resolution. Projection images were simulated at 60 equally spaced angles over a 180° acquisition arc extending from 45° right anterior oblique (RAO) to 45° left posterior oblique (LPO). We simulated a non-circular body-contouring orbit specific for each phantom where the collimator face was 5 cm away from the body surface at each projection view. This included the distance for the contact pad in front of the collimator as well as physical spacing that results from the use of automatic contouring systems. Projection images were formed in a 128 × 114 matrix with a pixel size of 0.442 cm. To model scatter in the object, we used the effective source scatter estimation (ESSE) method (Frey and Tsui 1997). The ESSE method estimates the object-dependent spatially varying contribution of scattered photons to the projection data, based on the attenuation map and the activity distribution. The CDR was modeled using distance-dependent CDR functions (CDRFs). The CDRFs modeled hexagonal parallel-hole collimators and included the intrinsic, geometric, septal penetration and septal scatter response components. Scatter kernels used by the ESSE method and CDRF tables were pre-calculated using Monte Carlo simulations.

2.4. Collimator design

We investigated a family of eight parallel-hole collimators that spanned a wide range of resolution-sensitivity trade-offs. The collimator parameters, i.e. hole length, hole diameter (flat-to-flat distance) and septal thickness, were chosen to maximize the sensitivity for each selected resolution with the value of the septal penetration criterion (i.e. the probability that a photon would pass through the thinnest possible full septum based on the hole geometry) at 140 keV fixed at 0.3%, to match that of the GE-LEHR collimator. We used standard formulas to calculate the collimator parameters for each given resolution (Beck and Redtung 1985, Moore et al 2005). The parameters of the simulated collimators are listed in table 2. Sample images of the heart of a medium sized phantom acquired using the different collimators are shown in figure 3.

Table 2.

Parameters of the simulated family of collimators

	Hole diameter (mm)	Septal thickness (mm)	Resolutiona (mm) at 10 cm	Relative geometric sensitivityb
C1	0.566	0.081	5.2	0.208
C2c	1.242	0.178	6.9	1
C3	1.730	0.248	9.2	1.940
C4	2.136	0.307	11.0	2.956
C5	2.533	0.364	12.4	4.157
C6	2.925	0.420	13.8	5.544
C7	3.700	0.531	16.7	8.872
C8	4.468	0.642	19.7	12.933

^aFWHM of the combined intrinsic resolution and geometric collimator resolution.

^bSensitivity relative to that of the GE-LEHR collimator.

^cCollimator parameters were chosen to match GE-LEHR collimator's resolution and geometric sensitivity.

The hole-length was 32 mm for all collimators, which is the hole-length that maximized the sensitivity for the given penetration fraction.

Figure 3.

Sample noise-free (top row) and noisy (bottom row) projection images of the heart acquired at an anterior view using collimators C₁ to C₈, respectively, from left to right. From left-to-right note the decreasing noise (in the bottom row) and sharpness of the images, as expected.

2.5. Ideal Observer for detection and joint detection and localization tasks

In SPECT imaging, the projection data, g, can be represented by the following imaging equation:

$$\mathbf{g}=\mathbf{Pf}+\mathbf{n},$$ (1)

where P, the system matrix, is a continuous-to-discrete projection operator that maps the object f to the projection space and n is the Poisson-distributed measurement noise. The object f consists of a random background, f_b, and a signal, f_sj, if a signal is present at location j (j = 1 to L). The background and signal projection images can then be expressed as b = Pf_b and s_j = Pf_sj, respectively.

In a binary detection task, the goal is to determine whether or not a defect is present based on the measured data. The two hypotheses to be tested can be represented mathematically as:

$$\begin{array}{cc}\hfill & H_0:\mathbf{g}=\mathbf{b}+\mathbf{n},\hfill \\ \hfill & H_1:\mathbf{g}=\mathbf{b}+\mathbf{s}+\mathbf{n}.\hfill \end{array}$$ (2)

The ideal observer uses the likelihood ratio Λ(g|b, s) of defect-present (H₁) to –absent (H₀) as the test statistic t_d(g).

$$t_d\left(\mathbf{g}\right)=\Lambda (\mathbf{g}\mid \mathbf{b},\mathbf{s})=\frac{pr(\mathbf{g}\mid H_1)}{pr(\mathbf{g}\mid H_0)}.$$ (3)

However, for the joint detection and localization task, the observer is required to both detect and localize a defect among a set of L (in this work, L = 6) possible defect locations. In myocardial perfusion SPECT this would be, for example, detecting and specifying a segment of the heart that has a perfusion defect. The task can be extended to an L + 1 class classification task where L is the (finite) number of defects locations. When L equals 1, the problem becomes the binary detection task described above. The L + 1 hypotheses to be tested can be represented mathematically as:

$$\begin{array}{cc}\hfill & H_0:\mathbf{g}=\mathbf{b}+\mathbf{n},\hfill \\ \hfill & H_j:\mathbf{g}=\mathbf{b}+{\mathbf{s}}_j+\mathbf{n}.\hfill \end{array}$$ (4)

We define the hypothesis H_j that the defect s_j is located at location j and j = 1, 2 ... L. The optimal decision strategy for the joint detection and localization task, as derived in (Khurd and Gindi 2005), can be written as:

$$t_l\left(\mathbf{g}\right)=\underset{j\in \{1,2,\dots L\}}{\max }pr\left(j\right)\Lambda (\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j),$$ (5)

$$\begin{array}{cc}\hfill & l\left(\mathbf{g}\right)=\arg \underset{j\in \{1,2,\dots L\}}{\max }pr\left(j\right)\Lambda (\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)\hfill \\ \hfill & \text{Decide}l\left(\mathbf{g}\right)\text{if}{t}_l\left(\mathbf{g}\right)>\tau ,\text{else decide}{H}_0,\hfill \end{array}$$ (6)

where t_l(g) is the test statistic for the joint detection and localization task. In (5), pr(j) is the prior probability that a perfusion defect is present at location j. In this study, we assumed equal probability of defect presence at each of the six locations. In the above, l(g) is the location assigned to the defect if the test statistic t_l(g) is larger than a threshold value τ.

For both tasks, it is required to calculate one or more likelihood ratios. For the case where the background and signal are known exactly (BKE/SKE) and the only source of randomness is the measurement noise, the expression for the likelihood ratio is:

$${\Lambda }_{\mathrm{BKE}}(\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)=\prod _i{\left(1+\frac{s_j\left(i\right)}{b\left(i\right)}\right)}^{g\left(i\right)}\exp (-s_j\left(i\right)),$$ (7)

where i is a pixel index in the background, signal and raw projection images. However, for the BKS/SKE task, the expression for the likelihood ratio is given by:

$${\Lambda }_{\mathrm{BKS}}(\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)=\int {\Lambda }_{\mathrm{BKE}}(\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)\mathrm{pr}(\mathbf{b}\mid \mathbf{g},H_0)\mathrm{d}b.$$ (8)

Kupinski (Kupinski et al 2003) has shown that if the background, f_b, and thus the projection images, b, arising from them, can be completely characterized by a statistically-defined set of parameters, $\stackrel{\rightharpoonup }{\theta }$, and that, for any given $\stackrel{\rightharpoonup }{\theta }$, there exists a unique b, the expression for the likelihood ratio can be reformulated as an integral of the BKE likelihood ratio over the background, $\mathbf{b}=\mathbf{b}\left(\stackrel{\rightharpoonup }{\theta }\right)$, parameterized by the vector $\stackrel{\rightharpoonup }{\theta }$, given the image g under H₀ as:

$${\Lambda }_{\mathrm{BKS}}(\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)=\int {\Lambda }_{\mathrm{BKE}}(\mathbf{g}\mid \mathbf{b}\left(\stackrel{\rightharpoonup }{\theta }\right),{\mathbf{s}}_j)\mathrm{pr}(\stackrel{\rightharpoonup }{\theta }\mid \mathbf{g},H_0)\mathrm{d}\stackrel{\rightharpoonup }{\theta }.$$ (9)

In this work, we parameterized the background by the object parameter vector, $\stackrel{\rightharpoonup }{\theta }=[m,n,p,q,A_{\text{heart}},A_{\text{liver}},A_{\text{body}}]$, that is a combination of discrete and continuous parameters: m, n, p and q are integer parameters specifying the object gender, body size, heart size and fat level and A_heart, A_liver and A_body are scalar parameters representing the total activity in the heart, liver and body, respectively. Thus, the background $\mathbf{b}\left(\stackrel{\rightharpoonup }{\theta }\right)$ given the object parameter vector $\stackrel{\rightharpoonup }{\theta }$ can be represented as:

$$\mathbf{b}\left(\stackrel{\rightharpoonup }{\theta }\right)=A_{\text{heart}}{\mathbf{prj}}_{m,n,p,q}^{\text{heart}}+A_{liver}{\mathbf{prj}}_{m,n,p,q}^{\text{liver}}+A_{body}{\mathbf{prj}}_{m,n,p,q}^{\text{body}},$$ (10)

where ${\mathbf{prj}}_{m,n,p,q}^{\text{heart}}$, ${\mathbf{prj}}_{m,n,p,q}^{\text{liver}}$ and ${\mathbf{prj}}_{m,n,p,q}^{\text{body}}$ are the projections of the heart, liver and the body (including other organs), respectively, filled with unit activity for a given anatomy.

Markov Chain Monte Carlo

The integral in (9) can be estimated using standard Monte Carlo integration methods by generating many independent and identically distributed (i.i.d) samples of $\stackrel{\rightharpoonup }{\theta }$ from the posterior distribution $\mathrm{pr}(\stackrel{\rightharpoonup }{\theta }\mid \mathbf{g},{\mathrm{H}}_0)$, and computing the sample mean of ${\Lambda }_{\mathrm{BKE}}(\mathbf{g}\mid \mathbf{b}\left(\stackrel{\rightharpoonup }{\theta }\right),{\mathbf{s}}_j)$. However, it is difficult to draw i.i.d samples from the posterior distribution $\mathrm{pr}(\stackrel{\rightharpoonup }{\theta }\mid \mathbf{g},{\mathrm{H}}_0)$ because the density function is usually complex or unknown. Markov Chain Monte Carlo (MCMC) techniques offer an alternative way to draw samples from the posterior distribution. We adopted a Metropolis–Hastings approach to draw a set of samples ${\stackrel{\rightharpoonup }{\theta }}^{\left(0\right)}$, ${\stackrel{\rightharpoonup }{\theta }}^{\left(1\right)}$, ...,${\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}$, ..., ${\stackrel{\rightharpoonup }{\theta }}^{\left(N\right)}$, where N is the number of iterations. For each ${\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}$, a proposed $\widetilde{\stackrel{\rightharpoonup }{\theta }}$ is generated according to a symmetric and an easy to sample conditional distribution, $q(\widetilde{\stackrel{\rightharpoonup }{\theta }}\mid {\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)})$, and is accepted with probability $\rho (\widetilde{\stackrel{\rightharpoonup }{\theta }}\mid {\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)})$ given by:

$$\rho (\widetilde{\stackrel{\rightharpoonup }{\theta }}\mid {\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)})=\min \left\{\frac{pr(g\mid \widetilde{\stackrel{\rightharpoonup }{\theta }},H_0)pr\left(\widetilde{\stackrel{\rightharpoonup }{\theta }}\right)q(\widetilde{\stackrel{\rightharpoonup }{\theta }}\mid {\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)})}{pr(g\mid {\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)},H_0)pr\left({\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}\right)q({\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}\mid \widetilde{\stackrel{\rightharpoonup }{\theta }})},1\right\}.$$ (11)

We used a proposal function, q(.|.), like the one used in (He et al 2008). The function was designed as a symmetric and variable-at-a-time function: in each iteration, the function randomly choses one parameter from the parameter vector $\stackrel{\rightharpoonup }{\theta }$ to propose. The proposal density function for the continuous parameters was a Gaussian with mean assigned to the current parameter value and a proposed standard deviation that was set to achieve ~30% acceptance rate, as suggested in (Albert 2009). For the discrete parameters, we defined the proposal distribution by discretizing a standard Gaussian distribution.

The entire sampling procedure is described by the following.

Metropolis–Hasting algorithm for estimating Λ_BKS(g|b, s_j):

• Step 0 (initialization): Choose length of ‘burn-in’ period M and initial state ${\stackrel{\rightharpoonup }{\theta }}^{\left(0\right)}$. Set k = 0.

• Step 1 (candidate point): Generate a candidate point $\widetilde{\stackrel{\rightharpoonup }{\theta }}$ according to the proposal distribution $q(\widetilde{\stackrel{\rightharpoonup }{\theta }}\mid {\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)})$.

• Step 2 (accept/reject): Generate a random number U ~ u(0, 1). Set ${\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}=\widetilde{\stackrel{\rightharpoonup }{\theta }}$ if $U⩽\rho (\widetilde{\stackrel{\rightharpoonup }{\theta }}\mid {\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)})$ (Metropolis–Hasting criterion). Otherwise set ${\stackrel{\rightharpoonup }{\theta }}^{(k+1)}={\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}$.

• Step 3 (iterate): Repeat steps 1 and 2 until ${\stackrel{\rightharpoonup }{\theta }}^{\left(M\right)}$ is available. Terminate ‘burn-in’ process and proceed to step 4 with ${\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}={\stackrel{\rightharpoonup }{\theta }}^{\left(M\right)}$.

• Steps 4–6 (ergodic average): Repeat process in steps 2 and 3 and compute the ergodic average:

  $${\tilde{\Lambda }}_{\mathrm{BKS}}(\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)=\frac{1}{N-M}\sum _{k=M+1}^N{\Lambda }_{\mathrm{BKE}}(\mathbf{g}\mid \mathbf{b}\left({\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}\right),{\mathbf{s}}_j).$$ (12)

The ergodic average ${\tilde{\Lambda }}_{\mathrm{BKS}}(\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)$ is an estimate of Λ_BKS(g|b, s_j).

The Metropolis–Hasting algorithm is an iterative approach that requires thousands of iterations to estimate the IO test statistics. In this work we used 50 000 iterations. It is often suggested, as we have done here, to discard the early outputs of the chain and not include them in the sum of step 6. This is because the choice of the initial state ${\stackrel{\rightharpoonup }{\theta }}^{\left(0\right)}$ biases the subsequent likelihood ratios and the amount of bias decreases with iteration. We discarded the first 20 000 iterations, which are referred to as the burn-in iterations in MCMC. This ensures the stabilization of each Markov chain for the computation of ${\tilde{\Lambda }}_{\mathrm{BKS}}(\mathbf{g}\mid \mathbf{b},{\mathbf{s}}_j)$.

Step 2 requires generating the projection images $\mathbf{b}\left({\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}\right)$ and s_j. This is computationally demanding and is practically impossible to apply for realistic background distributions and system models using either analytic or Monte Carlo simulation approaches. To efficiently apply the MCMC IO estimation, we precomputed noise-free projection images ${\mathbf{prj}}_{m,n,p,q}^{\text{heart}}$, ${\mathbf{prj}}_{m,n,p,q}^{\text{liver}}$ and ${\mathbf{prj}}_{m,n,p,q}^{\text{body}}$. This constitutes a database of 2(genders) × 3(heart sizes) × 3(fat levels) × 3(body sizes) = 54 different anatomies. The database was stored in-memory during the calculations to improve computational efficiency. This puts practical limits on the number of anatomies that can be included in the population. For a given parameter vector, ${\stackrel{\rightharpoonup }{\theta }}^{\left(k\right)}$, we located the corresponding anatomy using the indexes m, n, p, and q and the background image was then generated by scaling and summing the three organ projections together according to the values of A_heart, A_liver and A_body.

Figure 4 shows the likelihood ratios, Λ_BKS(g|b, s_j) for one sample input projection image as a function of the iteration number for the 30 000 iterations after the burn-in process. The input projection image simulated here had a perfusion defect in the antero-lateral wall of the heart (d₂). For the binary detection task, the test statistic, t_d(g) to be used was the one corresponding to the d₂ defect. To compute the test statistic, t_l(g), for the joint detection and localization task, we picked the maximum from the six likelihood ratios and used the location corresponding to that maximum. For the input projection simulated in figure 4, the location would be correctly assigned to the anterolateral wall (d₂).

Figure 4.

A plot of the likelihood ratio with iteration number for an input projection image with a perfusion defect located at the anterolateral wall (location d₂).

To perform the IO study for both tasks, we generated an ensemble of 540 pairs of noisy defect-present and -absent images. For each collimator and each defect location, we computed the IO test statistic using the MCMC method for the 540 images. For the joint detection and localization task, for each input projection image we calculated six likelihood ratios: one for each of the different defect locations. We could then identify the largest among them, as described earlier, and select the corresponding defect location. The projection data contained a 64 × 24 × 60 pixel region of interest centered over the heart centroid. Using a larger region made the IO computation more computationally intensive without changing performance.

The IO test statistics were then analyzed using ROC and LROC methodologies for the detection and joint detection and localization tasks, respectively. We used the area under the ROC and LROC curves (AUC_d and AUC_d+l) as the figures of merit to evaluate observer performance. We used the ROCkit code (Metz et al 1998), which fits a binormal ROC curve to the input data, to estimate AUC_d. To calculate AUC_d+l, we constructed the LROC curve by sweeping the decision threshold across the distributions of defect-present and defect-absent test statistics. At each threshold, we calculated the percent correct localization and the false positive fraction pair. We then computed the area under the LROC curve using trapezoidal integration. We used bootstrapping methods to estimate the standard deviation of the AUC_d+l.

3. Results

3.1. Binary defect detection task

Figure 5 shows a plot of the AUC_d for the different collimators for the binary detection task. Note that the error bars are very small (~0.005). These data suggest that a collimator with a FWHM resolution of 0.9–1.1 cm (C₃–C₄) at 10 cm is optimal. We tested the statistical significance of the differences between the AUC_d values for the collimators C₂ to C₅ on a pair-wise basis. The tests indicated that the differences were statistically significant at 95% confidence level (all p-values were less than 10⁻¹⁵) except for the difference between AUC_d values for C₃ and C₄ with a p-value equal to 0.178. We note that optimal collimators determined in this study tended to have poorer resolution and higher efficiency than the collimator C₂, which had resolution characteristics typical of those used for myocardial perfusion imaging, an effect seen by others using different task criteria (Zeng and Gullberg 2002, Zhou and Gindi 2009).

Figure 5.

The ideal Observer performance measured in terms of AUC_d for the different collimators.

We also calculated the specificity for various sensitivities for the standard GE-LEHR (C₂) and the optimal collimator (C₄) (table 3). The range of sensitivities was selected based on measured sensitivities reported in various clinical trials (Hendel et al 1999, Hendel et al 2002, Masood et al 2005, Garcia 2007). The maximum difference between the specificities of the two collimators was 4% at the 90% sensitivity level.

Table 3.

Sensitivity/Specificity pairs for the standard and optimal collimators.

	Specificity (%)
Sensitivity (%)	Standard GE-LEHR (C2)	Optimal collimator (C4)
80	84	87
82.50	82	85
85	80	83
87.50	77	80
90	73	77

3.1.1. Effect of perfusion defect location

We estimated the observer performance for each of the six defects and the corresponding optimal collimator, as shown in table 4. It is clear that the optimal collimator was the same for individual defects and for the whole set of defects combined. The AUC values suggest that there was a great deal of difference in the amount of information about the absence or presence of a myocardial perfusion defect at the various defect locations. A likely reason for this is the difference in the number of detected photons from the different locations and the average distance of the defects to the face of the collimator, resulting in different image resolution and noise levels.

Table 4.

IO performance measured in terms of AUC_d for the different defect locations.

Defect location	AUCd	Std. dev.	Optimal collimator
d1	0.917	0.003	C3–C4
d2	0.947	0.006	C3–C4
d3	0.921	0.001	C3–C4
d4	0.921	0.003	C3–C4
d5	0.932	0.007	C3–C4
d6	0.909	0.007	C3–C5

3.1.2. Effect of perfusion defect size

In this experiment, we evaluated the IO performance on the detection of perfusion defects of various sizes. We simulated perfusion defects with 5%, 10% and 25% extents, located at locations d₂ and d₅, as shown in figure 2. As a reminder, the product of the extent and severity was kept constant to model constant overall reduction of the activity in the myocardium. For each collimator and defect, we computed the IO test statistics using the MCMC method for 1 080 images. As shown in figures 6 and 7, we observed significant and direct dependence of the optimal collimator on defect size: as the defect size decreased the optimal collimator shifted toward better resolution. This agrees with previous results as reported in (Zhou and Gindi 2009).

Figure 6.

Observer performance in terms of AUC_d for various defect sizes located at d₂.

Figure 7.

Observer performance in terms of AUC_d for various defect sizes located at d₅.

3.1.3. Observer performance for different phantom sub populations

To investigate whether the optimal collimator depended on patient anatomical characteristics, we calculated AUC_d for different sub populations, as shown in figure 8. The figure also indicates the optimal collimator range for each sub population. The sub populations included phantoms having small, medium and large heart sizes, phantoms having small, medium and large adipose tissue levels and finally phantoms with small, medium and large body sizes. We note that the IO performance improved as the number of detected counts from the heart increased as well as when the size of the heart increased. This applies to the cases when assessing the observer performance for the various heart sizes. In addition, the observer performance decreased as the adipose tissue thickness increased, resulting in fewer detected counts from the heart and a larger distance from the detector to the heart. For larger body sizes, increased attenuation and scatter inside the body and poorer resolution, due to the increased distance from the heart to the face of the collimator, would degrade image quality. It is worth noting that in the digital phantom population used here (Ghaly et al 2014), the heart size varied in proportion to body size. For example, a medium-sized heart in a medium sized phantom was larger than a medium-sized heart in a small sized phantom and smaller than that in a large sized phantom. This compensated for the increased attenuation in the larger phantoms and thus resulted in more counts acquired from the heart for larger phantoms than for smaller phantoms.

Figure 8.

AUC_d values for the different sub populations.

We note that fat thickness had an impact on the choice of the optimal collimator. For phantoms with the small adipose tissue thickness, the optimal collimator range was C₃ to C₅, while for the larger fat thicknesses the optimal collimator was C₃. Note that in this dataset the increasing adipose tissue thickness did not change organ sizes. Thus, unlike the change in body size, the increased attenuation and decreased resolution due to the increased radius of rotation led to poorer detection performance and tended to favor better collimator resolution.

3.2. Joint perfusion defect detection and localization task

Figure 9 shows the IO performance, measured in terms of AUC_d+l, for the different collimators. This suggests that a collimator with a resolution of 0.9 cm FWHM (C₃) at 10 cm was optimal. We tested the statistical significance of the differences between the AUC_d+l values for the collimators C₂ to C₄ on a pair-wise basis. The tests indicated that the differences were statistically significant at a 95% confidence level (all p-values were less than 10⁻¹⁴). This is similar to what was suggested by the observer for the binary detection task. This result suggests that the optimal collimator for both tasks was similar. This agrees with the results of Zhou et al (Zhou and Gindi 2009). However, for the joint detection and localization task, the IO performance was more peaked at C₃, and there was an increasing penalty for choosing collimators with poorer resolution. This same effect has been reported in (Zhou and Gindi 2009).

Figure 9.

The ideal Observer performance measured in terms of AUC_d+l for the different collimators.

Table 5 shows the contingency table for the assignment of the different defects to the different locations. The vertical direction denotes the true location of the defect and the horizontal denotes the assigned location. The IO had the greatest success rate for assigning perfusion defects at location d₃. We also notice that the maximum failure rate occurred for either the super or sub-diagonal entries of the contingency table. This indicates that incorrect localization was generally to one of the neighboring defect locations.

Table 5.

Contingency table in percent for collimator C₃.

	d1	d2	d3	d4	d5	d6
d1	89.6	5.2	2.2	0.9	2.0	0.0
d2	1.3	89.3	5.2	0.7	3.1	0.4
d3	0.2	1.1	95.4	1.3	1.3	0.7
d4	0.6	0.0	3.1	91.9	3.9	0.6
d5	0.4	0.0	1.3	7.2	88.5	2.6
d6	0.7	0.4	3.5	4.1	7.8	83.5

3.3. Computational cost

In this section, we report the computation times required to perform the experiments conducted in this paper. The times required to calculate the test statistics for one image were about 14 and 25 min for the detection and joint detection and localization tasks, respectively. (Note that for the joint task, we calculated six likelihood ratios, one for each of the different defect locations.) Thus, the calculations presented here required a total of 4.76 CPU years. A breakdown of the simulation times is as shown in table 6. The CPU times are for a single core of a 2.33 GHz Intel Xeon E5410 quad core processor. Not included in these times are the time required to generate the projection library database and the table of collimator detector response functions for each collimator.

Table 6.

A breakdown of the simulation times.

	Binary detection	Joint detection and localization
Noise realizations	540	540
Defect status	2 (present and absent)	2
Defects	10 (d1–d6 plus 2 additional defect sizes at locations d2 and d5)	6 (d1–d6)
Required time to calculate the likelihood ratio for one image	~14 min	~25 min
Total required time (years)	2.3	2.46

4. Discussion

This paper describes a collimator optimization procedure for defect detection, as well as defect detection plus localization tasks, in the context of MPS imaging. The phantom population used in this study included variability in gender, body size, heart size and adipose tissue level, with no modeling of the cardiac or respiratory motion included. Adding more anatomical variation such as heart angle or respiratory motion, while more realistic, would make the simulation prohibitively computationally expensive. In an attempt to acknowledge the fact that the heart changes shape during beating motion, we did generate phantoms at the point in the cardiac cycle that was closest to the average heart over the 8 gates of the cardiac cycle (Ghaly et al 2014). We anticipate that inclusion of cardiac and/or respiratory motion would affect observers performance, but it is not clear that it would affect our conclusions.

We used a simplified version of the joint detection and localization task in which the defect could appear only in one of six different locations distributed over a mid-ventricular slice of the heart. Since the defects were at a finite number of discrete locations, we did not apply the notion of search tolerance ordinarily used in LROC studies. The addition of more defect locations or search tolerance would be very computationally expensive. Of note, MPS images are usually interpreted in territories and sub-territories (Cerqueira et al 2002). In a clinical search scenario, the task is not to identify the precise position of the defect, but to localize it in some territory of the coronary artery structure. This is different than the clinical search in tumor imaging, in which accurate localization of the tumor is essential. For computational reasons, we also limited the number of defects. We chose a mid-ventricular slice and the six locations that mimic the six segments that are clinically used to interpret this plane, as depicted in figure 1.

We investigated a single injected activity of Tc-99m of 10 mCi. This corresponds to the usual clinical injected activity that is administered to a patient for the first (stress) component in a one-day stress/rest protocol. In this protocol, the diagnostic outcome of the low-dose stress scan determines the necessity of the high-dose rest scan. If the patient has a normal stress study, many practices forego the rest study and thus avoid the additional radiation exposure. However, a higher injected activity would certainly affect the noise levels of the projection and reconstructed SPECT images, and consequently could result in a different optimal collimator than that reported here.

We investigated a family of eight parallel-hole collimators that spanned a wide range of sensitivity and resolution tradeoffs. We chose the collimator parameters to maximize the sensitivity for each selected resolution, while fixing the single septal penetration (SSP) probability at 140 keV to 0.3%. Other septal penetration probabilities might give different optimal results than those reported here. Investigating the optimal septal penetration probability would, however, have been highly computationally expensive. Of note, a previous study of two collimators with vastly different septal penetration fractions indicated relatively little difference in image contrast (He et al 2005).

We used the IO operating on projection data to find the optimal collimator that would preserve the maximum possible information in the sinogram, regardless of the reconstruction method used to convert the sinogram into images suitable for human interpretation. We note that a given reconstruction method may or may not make optimal use of the sinogram information, and thus affect observer performance and the choice of the optimal collimator. It has been advocated that joint (simultaneous) optimization of collimator and reconstruction methods yields better detection/estimation task performance than the sequential approach, in which the collimator is optimized using the projection data, for various tasks such as defect detection, and estimation tasks (Zhou et al 2009, McQuaid et al 2011). In the cited studies, it was reported that the optimal collimators had higher resolution than those identified when optimizing using only the raw projection data (i.e. when not considering the reconstruction method). In a previous study (Ghaly et al 2013, 2015), we demonstrated that, for a detection task, when a good model of the image formation process is incorporated into the reconstruction algorithm, the IO yielded similar optimal operating ranges as those obtained using the CHO. In this study, we took advantage of the fact that accurate models of the collimator-detector response for Tc-99m are readily available, which permitted us to avoid the computationally expensive joint acquisition and reconstruction optimization, and performed the collimator optimization in this study using the raw projection data.

Various vendors have recently introduced several new dedicated hardware camera systems with optimized collimator design, acquisition geometry, and reconstruction software that allow for improvement in system resolution and sensitivity. One of these systems is the D-SPECT (Spectrum Dynamics, Haifa, Israel), which uses a parallel hole collimator. However, the sampling geometry is very different than the system that we modeled. As a result, due to the differences in sampling and collimation, the tradeoff between collimator resolution and image quality for these systems is likely to be very different, and requires further investigation.

Results reported in this paper suggest that collimators with somewhat poorer resolution than a commonly used SPECT collimator appear optimal for the task of defect detection in MPS. A difference of 0.02 in the AUC_d values of the reported optimal collimators and the standard collimator was statistically significant. While this change in AUC may seem small, to put it in clinical perspective, this is the same difference in the AUC_d for MPS with and without attenuation compensation reported in (He et al 2004). Moreover, the 4% increase in specificity at 90% sensitivity was similar to that reported in (Masood et al 2005). In that study, they evaluated the effect of CT-based attenuation compensation using clinical SPECT images acquired using a hybrid SPECT/CT dual-head gamma camera (GE Hawkeye; GE Medical Systems, Waukesha, Wisconsin) and GE-LEHR collimator.

5. Conclusions

We performed defect detection and defect detection and localization task-based collimator optimization in the context of MPS using the IO operating on a realistic simulated population of phantoms. This represents a clear step forward in the use of clinically realistic IO-based optimization. Our results indicate that poorer resolution collimators than standard collimators, such as the GE-LEHR collimator, are optimal for the range of defect sizes studied. The results showed that the optimal collimator was the same for the individual defects and for the whole set of defects combined together. We observed significant and direct dependence of the optimal collimator on defect size: as the defect size decreased, the optimal collimator shifted toward better resolution. It was also noted that the optimal collimator for both tasks was similar, though the detection-localization task tended to prefer a collimator with somewhat higher resolution. The improvement in the area under the ROC curve for the detection task was about 0.02, providing a 4% increase in specificity at 90% sensitivity compared to a typical LEHR collimator.