Accuracy and Reproducibility of Automated, Standardized Coronary Transluminal Attenuation Gradient Measurements

Abstract

Purpose

Coronary Computed Tomography Angiography (CCTA) contrast opacification gradients, or Transluminal Attenuation Gradients (TAG) offer incremental value to predict functionally significant lesions. This study introduces and evaluates an automated gradients software package that can potentially supplant current, labor-intensive manual TAG calculation methods.

Methods

All 60 major coronary arteries in 20 patients who underwent a clinically indicated single heart beat 320×0.5 mm detector row CCTA were retrospectively evaluated by two readers using a previously validated manual measurement approach and two additional readers who used the new automated gradient software. Accuracy of the automated method against the manual measurements, considered the reference standard, was assessed via linear regression and Bland-Altman analyses. Inter- and intra-observer reproducibility and factors that can affect accuracy or reproducibility of both manual and automated TAG measurements, including CAD severity and iterative reconstruction, were also assessed.

Results

Analysis time was reduced by 68% when compared to manual TAG measurement. There was excellent correlation between automated TAG and the reference standard manual TAG. Bland-Altman analyses indicated low mean differences (1 HU/cm) and narrower inter- and intra-observer limits of agreement for automated compared to manual measurements (25% and 36% reduction with automated software, respectively). Among patient and technical factors assessed, none affected agreement of manual and automated TAG measurement.

Conclusion

Automated 320×0.5 mm detector row gradient software reduces computation time by 68% with high accuracy and reproducibility.

Introduction

While coronary CT angiography (CCTA) utilization increases, it remains limited because of challenges in identifying the hemodynamic significance of individual lesions (1, 2). Growing evidence supports the initial observation that coronary blood flow information can be obtained from CCTA using contrast opacification variations (3–6). The contrast opacification gradient, or transluminal attenuation gradient (TAG) (7) computes a linear regression of the CT density (Hounsfield units [HU]) versus distance along the length of a coronary artery to summarize the proximal to distal variation in coronary contrast opacification in a standard CCTA acquisition.

The literature on TAG (3, 4, 6, 8, 9) has reported mixed results; some strongly support its use as a simple adjunct to CCTA while others suggest it has limited value. The TAG is influenced by scanner hardware, physiological parameters, as well as the measurement technique itself. This work addresses potential variability due to lack of a standardized measurement method. In the initial gradient technique, Steigner et al (7) used customized software to analyze vessels at 0.5 mm intervals along the coronary centerline to 2.5 mm luminal diameter distally. Choi et al (3, 8) and Yoon et al (9) instead manually measured opacification at 5 mm intervals but to a distal cutoff defined by 2 mm² lumen area. Wong et al (6) also followed the same 5 mm interval manual approach and 2 mm² cutoff, but defined coronary opacification using 1 mm² luminal regions of interest (ROI).

This study introduces new automated software for TAG measurement and investigates its accuracy against manual TAG measurement. We also test the hypothesis that automated calculation reduces inter-observer and intra-observer variability compared to manual TAG, and finally we investigate the effect of different patient-, artery-, and scan-related parameters on the agreement between automated and manual TAG, as well as on their reproducibility.

Methods

Image acquisition and image reconstruction

The institutional human research committee approved this HIPAA-compliant retrospective study; informed consent was waived. We assessed gradients in twenty randomly selected intermediate risk patients (Table 1) imaged with first generation single R–R 320 × 0.5 mm detector row (Aquilion ONE, Toshiba Medical Systems Corporation, Tochigi, Japan) CCTA at our institution between September 2012 and September 2013. Beta blockers were administered orally and/or intravenously with a target heart rate of ≤65 beats per minute. Contrast (60–70 ml iopamidol 370 mg iodine/ml, [Isovue 370, Bracco Diagnostics, Princeton, NJ]) followed by saline (40 ml) were injected at a rate of 6 ml/s with a dual-syringe injector (Empower Plus CTA, Bracco Diagnostics, Princeton, NJ). CCTA acquisition was timed by bolus tracking (180 HU) in the descending aorta. Most cases used prospective ECG triggering at 65–85% of the R–R interval (10). Radiation dose was estimated from the dose-length product using a conversion factor of 0.014 mSv/mGy-cm.

Table 1.

Study population characteristics

Demographics
Age (years, mean±SE)	59.2±2.2
Male (%, n)	50% (10)
Weight (kg, mean±SE)	90.1±6.1
Height (cm, mean±SE)	169.4±3.8
BMI (kg/cm2, mean±SE)	29.9±1.2
Hyperlipidemia (%, n)	60% (12)
Diabetes mellitus (%, n)	30% (6)
Hypertension (%, n)	60% (12)
Obesity (%, n)	25% (5)
Smoking (%, n)	0% (0)
Family history (%, n)	20% (4)
LV ejection fraction (mean±SE)	56±3%
CT indication
Chest Pain (%, n)	55% (11)
Abnormal/equivocal test (%, n)	25% (5)
Pre-op evaluation (%, n)	5% (1)
Coronary aneurysm (%, n)	5% (1)
Others (%, n)	10% (2)
CT contrast
Total volume (ml, mean±SE)	65.8±1.6
Contrast rate (ml/sec, mean±SE)	5.9±0.1
CT parameters
Peak tube voltage: 120 kV	85% (17)
100 kV	15% (3)
Tube current (mA, mean±SE)	503±17.9
Mean HR at acquisition (bpm, mean±SE)	57±1.7
Nitroglycerin
0.4 mg (%, n)	60% (12)
0.8 mg (%, n)	40% (8)
Beta blockade (n) *
None	4
Oral	4
<10mg iv	5
10–20mg iv	5
21–30mg iv	4
>30mg iv	1
CAD by CT
Normal	52% (31)
Non-obstructive CAD	25% (15)
Obstructive CAD	23% (14)
Plaque Characteristics by CT (n=29 vessels)
Non-calcified	55% (16)
Calcified	45% (13)
CAD by Reference Standard (n=42 vessels)**
Normal	48% (20)
Non-obstructive CAD	31% (13)
Obstructive CAD	21% (9)
DLP (mGy-cm, median, IQR)	296.8 (248.4–392.5)
Estimated dose (mSv, median, IQR)	4.2 (3.5–5.5)

SE- Standard Error, IQR- Interquartile Range

^*Three patients received both oral and IV medication β-blocker; numbers in parentheses represent treated patients.

^**Confirmed by either invasive catheter angiography, nuclear myocardial perfusion imaging or stress echocardiography, performed a median of 12 (IQR, 5–39) days of CCTA. Six patients did not have a reference standard test.

Images were reconstructed at 0.5 mm slice thickness with 0.25 mm overlap using an iterative (AIDR3D) algorithm (11) and default coronary reconstruction kernel (FC03). For those 15 patients with available “raw” sinogram data, images were additionally reconstructed using conventional filtered back-projection (FBP), keeping all other reconstruction parameters identical.

Gradient measurements

Four readers evaluated TAG in the major coronary arteries (left anterior descending [LAD], left circumflex artery [LCx], and right coronary artery to the posterolateral branch [RCA]) in AIDR3D reconstructions. One week later, each reader evaluated TAG in the 15 patients with FBP reconstructions. Finally, another week later, each reader revaluated TAG in the first 10 AIDR3D reconstructions. All readers were blinded to each other’s measurements. Two of the readers used the new automated gradient software (Toshiba Medical Systems Corporation, Tochigi, Japan) and the other two manually obtained the gradient using an image post-processing workstation (Vitrea 6.4, Vital Images, A Toshiba Medical Systems Group, Minnetonka, MN) following the method described by Wong et al (6). All readers timed the duration of analysis for each artery.

Automated gradient calculations

Each cardiovascular imager placed seed points for automated coronary centerline detection and performed manual correction in multi-planar reformations (MPR) as needed. The software then automatically generated cross-sectional images every 1 mm perpendicular to the centerline (Figure 1A). Next, each reader used sliders in the MPR view to place two gradient “landmarks”, one proximally at the coronary ostium and one distally where lumen diameter tapered to <2 mm². The software reported the TAG between landmarks using the average HU in 3×3-pixel blocks centered at the centerline in each cross-sectional image. Assuming typical 0.35 mm in-plane resolution, the 3×3-pixel block approximates a circular 1mm² ROI as described by Wong et al (6).

Figure 1.

(A) Snapshot of the software performing automated TAG calculations in an LAD with obstructive (>70%) calcified plaque proximally. User manually defines segment of interest from left main ostium to a 2 mm² distal landmark (white arrows), with access to centerline editing in curved MPR and luminal contour editing in cross-sectional images. (B) Methodology for manual TAG calculations in the same artery. User performs manual measurement of the luminal intensity in ROIs placed within the lumen beginning at the ostium every 5 mm along the length of the coronary artery, and statistical analysis software is used to perform the linear regression. Accuracy of the two techniques as defined by (C) Bland-Altman plot and (D) linear regression plot.

Manual gradient calculations

Each reader similarly defined and manually corrected the centerline, and identified the same two landmark locations (Figure 1B). Manually positioned 1 mm² ROIs in the coronary lumen were then placed in cross-sectional images perpendicular to the coronary artery centerline every 5 mm between the two landmarks. The location and average HU of each ROI were then manually transcribed and the TAG was calculated as the slope of the linear regression analysis.

Accuracy, inter-observer agreement and reproducibility of automated gradients

Using the manually-derived TAG as reference, automated TAG accuracy was assessed using all vessels in the 20 patients with AIDR3D reconstructions. Inter-observer reproducibility of manual and automated TAG was also assessed from this data. Intra-observer reproducibility for each reader independently was assessed from the repeat TAG measurements performed two weeks after the initial measurements. The time interval between the initial and repeat measurement was considered adequate to minimize recall bias. We further investigated five factors that can potentially influence TAG accuracy and inter- and intra-observer agreement: coronary artery, body mass index (BMI), presence and severity of CAD, plaque composition, and image reconstruction algorithm. CAD severity was classified by both CCTA, as well as a reference standard when available. Details are provided in Table 2.

Table 2.

Parameters affecting accuracy, inter-observer variability and intra-observer reproducibility of manual and automated TAG measurements.

		Accuracy		Manual				Automated
		Manual minus Automated		Inter-observer		Intra-observer*		Inter-observer		Intra-observer*
		Δ (Hu/cm)	p	Δ (Hu/cm)	p	Δ (Hu/cm)	p	Δ (Hu/cm)	p	Δ (Hu/cm)	p
All Patients		−1.0±1.1		1.6±1.4		0.05±1.2		0.4±1.0		1.0±0.4
BMI (kg/cm2)	<25	3.1±3.5	0.23	4.9±3.2	0.75	2.5±1.8	0.01€	5.3±3.7	0.38	−0.4±1.2	0.53
	25–30	−3.7±1.4		2.5±2.2		−3.9±1.7		−0.3±0.5		0.6±0.4
	>30	−2.4±2.0		1.6±1.8		1.5±1.8		−1.2±2.1		1.8±0.8
Coronary artery	LAD	−1.7±1.3	0.05	2.0±1.1	0.62	−1.1±1.1	0.85	−0.1±0.7	0.32	0.7±0.5	0.16
	LCX	2.4±2.5¶		1.6±3.6		2.6±2.6		3.0±2.4		0.7±1.0
	RCA	−3.6±1.5		1.1±2.0		−1.3±1.9		−1.7±1.9		1.6±0.8
Presence and severity of CAD by CT♯	Normal	−2.1±1.8	0.10	1.1±2.5	0.43	−2.0±1.4	0.14	−0.4±1.7	0.29	1.5±0.7	0.16
	Non- obstructive	−0.3±1.1		1.7±1.3		0.4±1.0		0.6±0.3		−0.1±0.3
	Obstructive	0.7±2.4		2.5±1.7		3.0±3.4		1.9±2.4		1.3±1.1
CAD by Reference Standard♮	Normal	−1.7±2.5	0.26	3.9±2.7	0.55	−1.8±1.6	0.34	0.6±2.6	0.40	1.7±0.8	0.34
	Non- obstructive	−0.1±1.9		2.8±1.4		0.9±0.9		2.0±1.7		0.3±0.3
	Obstructive	−0.1±2.9		0.8±1.9		3.5±5.6		0.7±2.9		1.7±1.8
Plaque Composition♭	Non- calcified	0.2±1.6	0.47	1.4±1.5	0.44	1.3±0.8	0.84	3.2±1.7	0.09	0.6±0.8	0.33
	Calcified	0.2±2.0		2.9±1.5		2.2±4.8		−1.2±1.1		0.5±0.8
Image Reconstruction	AIDR3D	−0.5±1.3	0.70	1.3±1.7	0.63	**		1.6±1.1	0.42	**
	FBP	−1.1±2.0		2.5±1.9		**		−0.5±2.4		**

^*Difference between reads 1 and 2 calculated from the mean TAG among readers.

^¶There was a borderline significance between LCX and RCA (p=0.05).

^€BMI 25-30 was significantly different from BMI<25 and BMI>30.

^♯Non-obstructive defined as most stenotic segment <50% diameter stenosis; obstructive defined as most stenotic segment ≥50% diameter stenosis.

^♮Available in n=14 of 20 patients; obstructive CAD defined as: >70% stenosis at invasive angiography, summed stress score ≥4 and defect identified in the vessel territory at rest-stress nuclear myocardial perfusion imaging, or stress-induced severe hypokinesis or akinesis in ≥3/16 segments at stress echocardiography.

^♭Non-calcified defined as ≤30%, and calcified as >30% (3) of plaque volume with >350HU (19)

^**Repeat TAG measurements were not performed for FBP reconstructions.

Statistical analysis

Analyses were performed in GraphPad Prism 6.0 (GraphPad Inc., San Diego, CA). Results are expressed as mean±standard error of mean or median and interquartile range. Comparisons of means among normally distributed variables were performed with the Student’s t-test, or one way analysis of variances (ANOVA) with Tukey’s test to correct for multiple comparisons. Comparisons of means among non-normally distributed variables were performed with the Wilcoxon matched-pairs signed rank test and Friedman test for paired data, as well as the Kruskal-Wallis test for non-paired data. The Dunn’s test was further used to correct for multiple comparisons. For the method comparison study of automated versus manual assessment, as well as for reproducibility of automated and manual TAG calculations, Bland-Altman analysis and linear regression analysis with Spearman correlation were performed (12). In Bland-Altman plots, the difference between corresponding measurements (y-axis) was plotted against their mean (x-axis). The bias and limits of agreement (mean±1.96×standard deviation [SD]) were calculated. The automated method was considered to agree with the manual measurements when the mean difference and the limits of agreement between the automated and manual analysis were comparable or better than the inter- and intra-observer agreement of the manual TAG measurements. A p-value ≤0.05 was considered significant.

Results

There were no automated software failures. For the timed measurements (n=30 arteries), the analysis time per artery was 68% less for the automated software (4.2±0.3 vs. 13.0±0.7 min/artery, p<0.001). Considering all 60 vessels, the difference between the mean of the automated TAG (−24.2, IQR: [−41.0, −15.8]) and the manual TAG (−25.2, IQR [−38.5, −16.1]) was less than 1 HU/cm and was not significantly different (p=0.10). Bland-Altman analysis (Figure 1C) showed a bias of 1.0HU/cm and limits of agreement ranging from −15.6 to 17.6 HU/cm. Linear regression analysis (Figure 1D) indicated a unit slope and small intercept (y=1.0x+2.0 HU/cm, r=0.92, p<0.001), further supporting the agreement of the two approaches.

Considering all 60 arteries, there was high inter-observer agreement for both automated (linear regression y=1.1x+3.3, r=0.92, p<0.001) and manual (y=1.1x+1.9, r=0.90, p<0.001) TAG; Bland-Altman plots revealed slightly lower bias and narrower limits of agreement for automated versus manual gradient measurements (Figure 2). Intra-observer reproducibility was very high for both automated and manual approaches (Figure 3); automated measurements had narrower limits of agreement and higher correlation coefficients (r=0.93 for both readers, p<0.001) compared to manual (r=0.84 and 0.91 for readers 1 and 2, respectively, both p<0.001) TAG measurements (Figure 3).

Figure 2.

Inter-observer agreement of automated and manual TAG: (A) Bland-Altman plots for automated and (B) manual TAG; (C) Linear regression analysis for automated and (D) manual TAG.

Figure 3.

Reproducibility of automated and manual TAG: (A, C) Bland-Altman plots for reader 1 and 2 using automated, and (E,G) for reader 3 and 4 using manual TAG methods. (B,D) Linear regression plots for reader 1 and 2 using automated, and (F,H) for reader 1 and 2 using manual TAG methods.

There was no difference between automated and manual TAG (Table 2) across BMI categories, presence and severity of CAD, plaque composition, or reconstruction algorithm. Although not significantly different, agreement between automated and manual gradients was reduced in the LCx compared to LAD and RCA. This was reflected in wider limits of agreement for the LCx (−24.7 to 19.9 HU/cm, compared to −9.3 to 12.8 HU/cm for the LAD and −9.1 to 16.4 HU/cm for the RCA). None of the parameters studied influenced either inter-observer agreement or intra-observer reproducibility of automated TAG measurements. For manual TAG, only BMI influenced intra-observer reproducibility (Table 2).

Discussion

This study introduced automated gradient software and provides, to our knowledge, the initial study of gradient variability due to measurement strategy. While we only considered 20 patients that span several indications for CCTA, a strength of this work is the comprehensive assessment over multiple readers. An accurate automated gradient software package can be implemented into clinical practice with a 68% time savings with respect to manual ROI measurements. Moreover, the automated system slightly improves the inter- and intra-observer limits of agreement.

The value of techniques such as TAG stems from the fact that CCTA remains limited largely because it does not readily provide information regarding functional significance of anatomical lesions. Demonstration of the hemodynamic significance of CAD, for which invasive FFR is the accepted gold-standard (13), has proven beneficial for guiding revascularization (2, 14). As a result, techniques are being tested to enhance CCTA with the ability to confer functional information, thereby providing a stand-alone single-modality test for CAD assessment and risk stratification. Among those are CT perfusion (15) and “FFR-CT” (16, 17) which rely on either the acquisition of a second CT scan at induced hyperemia, or the simulation of hyperemia by computational means, respectively.

Coronary opacification gradients are another approach toward extracting hemodynamic information from CCTA. They emerged from the observation that contrast distribution at single heart beat 320-detector row CCTA is not homogeneous along the length of a coronary artery (7), but is instead characterized by a linear drop-off (gradient) in luminal HU along the length of the artery (7). In an in vitro validation study, Lackner et al established that the temporal and contrast resolution of ≥16 detector row CT indeed enables detection of flow-induced variations in luminal HU that suffice to differentiate stenosis grade (5). Choi et al reported the in vivo correlation of TAG to Thrombolysis in Myocardial Infarction (TIMI) grade and additionally found incremental value in diagnostic accuracy for calcified lesions (3). In terms of functional information at stress, Choi et al reported a low sensitivity but high specificity for TAG measured in 64-detector row CCTA that when added to CCTA percent stenosis information significantly increased the area under the receiver operating characteristic curve for the detection of abnormal invasive FFR ≤0.8 (8). Yoon et al similarly reported a low sensitivity but high specificity of TAG from 64-detector row CCTA for the detection of FFR ≤0.8, but did not report accuracy when added to CCTA percent stenosis (9). More recently, Wong et al reported that TAG measured in 320-detector row CCTA independently predicted FFR ≤0.8 and increased both the sensitivity and specificity of CCTA percent stenosis information (6).

While gradients are conceptually simple, the manual placement of 20–100 lumen ROIs in each coronary artery is impractical for CCTA workflow. Furthermore, TAG inter- and intra-observer variability have been only sporadically reported, and not analyzed with respect to the physiological and technical variations encountered at CCTA. In addition to the time savings, the automated software resulted in lower measurement variation, both between observers and for the same observer (25% and 36% reduction in limits of agreement, respectively). This finding is likely due to the fact that automated software can perform HU measurements at much more closely spaced intervals than reasonable for manual measurement (e.g. every 1 compared to 5 mm). Using more closely spaced measurements reduces the effect of plaque inclusion in some ROIs. Automated TAG measurements may thus have the potential to enable increased statistical power with a reduced sample size for the detection of significant differences with respect to a reference standard.

Furthermore, it is unknown what factors may affect TAG measurements. This study provides evidence that observer agreement and reproducibility of TAG remains largely unaffected within the user bias present for many common facts. Importantly, presence of obstructive CAD and plaque composition did not alter accuracy or reproducibility of automated TAG, suggesting the ability of the software to effectively perform in all cases. We also found that TAG measurements are both feasible and accurate for images reconstructed with iterative methods. A borderline significant difference in accuracy was observed in this study for the LCx compared to the RCA, with the widest limits of agreement found in the LCx. For future trials, it may be prudent to consider separate comparison of coronary territories against reference standards such as FFR, and/or defining different cutoff values for normal versus abnormal TAG per territory. We also found gradient discrepancies (up to approximately 6 HU/ cm) among the BMI categories. While this may be related to image noise, further study is required to assess if future gradient measurements will require stratification with respect to patient BMI.

Limitations

For this study, the automated TAG reference standard was manual measurements performed as previously described in the same cohort. As this study was not designed to assess imaging findings of CAD with a reference standard, it is in theory possible that automated gradients have different accuracy (when compared to manual gradient measurements) and reproducibility profiles for patients with versus without CAD. Moreover, if binary cutoffs for normal versus abnormal gradients are implemented into clinical practice, these should be validated against a clinical reference standard for any gradient method under consideration. However, the sub-analyses of normal versus non-obstructive versus obstructive CAD here provides pilot evidence that the automated measurements are likely to be accurate (when compared to clinical standards). Larger studies using a clinical reference standard such as FFR (6) or combined anatomy and function (15) are needed, but can in theory be simpler to perform with the new automated TAG tool. Another limitation is that our validation inherently applies to 320-detector row single heartbeat CCTA, in which the contrast opacification gradient is directly related to the pace of contrast transport through the coronaries. This and many other factors that can affect gradient reproducibility still remain to be assessed, such as spiral vs. single heart beat CCTA, amount and rate of contrast injection, and overall contrast bolus geometry, which in turn is also affected by systemic cardiovascular function (18). Investigating those aspects will require larger studies but that are again rendered more readily possible given access to a quick, automated TAG approach.

Conclusion

Coronary gradient measurements using automated software are feasible, accurate and reproducible over several clinical and imaging-based factors. Automated gradients could allow for more rapid assessment of the functional significance of anatomical lesions without requiring more contrast, pharmacologic stress agents or sophisticated computational analyses. Future studies assessing coronary gradients as a metric to determine the hemodynamic significance of anatomical lesions can standardize gradient measurements via such automated software so as to maximize the possibility to detect correlations to study endpoints.