Reproducibility of quantitative plaque measurement in advanced coronary artery disease

Abstract

Background:

The ability to characterize and to quantify the extent of coronary artery disease has the potential to improve the prognostic capability of coronary computed tomography angiography. Although reproducible techniques have been described in those with mild coronary disease, this has yet to be assessed in patients with advanced disease.

Methods:

Twenty patients with known multivessel disease underwent repeated computed tomography coronary angiography, 2 weeks apart. Coronary artery segments were analysed using semi-automated software by two trained observers to determine intraobserver, interobserver and interscan reproducibility.

Results:

Overall, 149 coronary arterial segments were analysed. There was excellent intraobserver and interobserver agreement for all plaque volume measurements (Lin’s coefficient 0.95 to 1.0). There were no substantial interscan differences (P>0.05 for all) for total (2063±1246 mm³, mean of differences −35.6 mm³), non-calcified (1795±910 mm³, mean of differences −4.3 mm³), calcified (298±425 mm³, mean of differences −31.3 mm³) and low-attenuation (13±13 mm³, mean of differences −2.6 mm³) plaque volumes. Interscan agreement was highest for total and noncalcified plaque volumes. Calcified and low-attenuation plaque (−236.6 to 174 mm³ and −15.8 to 10.5 mm³ respectively) had relatively wider 95% limits of agreement reflecting the lower absolute plaque volumes.

Conclusion:

In the presence of advanced coronary disease, semi-automated plaque quantification provides excellent reproducibility, particularly for total and non-calcified plaque volumes. This approach has major potential to assess change in disease over time and optimize risk stratification in patients with established coronary artery disease.

TOC Summary

Original research: We demonstrate the excellent reproducibility of semi-automated plaque quantification on computed tomography coronary angiography in a cohort of patients with advanced coronary artery disease. On a per patient level, intraobserver, interobserver and interscan correlations were excellent for total plaque volume and non-calcified plaque volume. Plaque quantification represents a robust approach for the assessment of disease progression and facilitates the optimisation of risk stratification in patients with established coronary artery disease.

1. INTRODUCTION

Coronary computed tomography angiography (CCTA) is often the imaging modality of choice for those suspected of having coronary artery disease due to its diagnostic accuracy, prognostic value and ability to guide evidence-based treatments.^1,2 As technology has evolved, our ability to quantify the extent of coronary artery plaque has improved. Semi-automated software now facilitates the rapid assessment of coronary plaque subtype and burden which is comparable to that achieved with intravascular ultrasound.³ Recent studies have reported that quantitative evaluation of non-calcified plaque burden, particularly low-attenuation plaque, holds high prognostic value in identifying patients at risk of myocardial infarction.⁴ Serial imaging can be performed to monitor the extent and distribution of coronary atheroma such that there is a growing interest in the use of coronary plaque composition as a surrogate endpoint in randomised clinical trials.⁵ However, if this metric is to gain more widespread adoption, it is important to validate the precision of measurement on repeated testing.

Robust reproducibility for the quantitative analysis of plaque burden is essential for clinical application. Previous studies have reported excellent reproducibility with quantitative coronary plaque analysis in patients with a low burden of atheroma.^6,7 However, as coronary artery disease becomes more advanced, extensive vascular calcification can cause blooming artefact and photon starvation which hinders assessment of the coronary lumen.^8,9 Moreover, the scan-rescan reproducibility of low-attenuation non-calcified plaque, has not been established. As the residual risk of cardiovascular events is highest in patients with higher burdens of atherosclerosis, we sought to assess whether non-calcified plaque could be precisely quantified in patients with advanced coronary artery disease.

2. METHODS

2.1. Study Population

The study population was recruited from a randomized controlled trial using CCTA in patients with advanced coronary artery disease [NCT02110303], the results of which have been described previously.¹⁰ In brief, patients were eligible if they were over the age of 40 years and had angiographically proven multivessel coronary artery disease defined as at least 2 major epicardial vessels with either ≥50% luminal stenosis or previous revascularisation (percutaneous coronary intervention or coronary artery bypass graft surgery).

2.2. CCTA Image Acquisition

The CCTA image acquisition has been described previously.¹⁰ Briefly, patients underwent baseline CCTA (64-multidetector Biograph mCT, Siemens Medical Systems, Erlangen, Germany) after receiving 50 to 100 mg of oral metoprolol to achieve a heart rate of <65/min and 400 μg of sublingual glyceryl trinitrate. A contrast-enhanced CCTA was performed using prospective electrocardiogram gating, triggering in mid-diastole (60–75% R-R interval) during an expiratory breath-hold (Prospective CareKV, tube voltage of 100 kVp (body-mass index <25 kg/m²) or 120 kVp (body-mass index ≥25 kg/m²)). Scans were reconstructed using 180-degree rotation, 512 × 512 matrix and B26f reconstruction kernel with 0.75-mm slice thickness at 0.5-mm increments. Repeated scans were conducted using the same acquisition protocol within 2 weeks of the baseline scan.

2.3. CCTA Image Evaluation

CCTA data sets were anonymised and exported in Digital Imaging and Communications in Medicine (DICOM) format for measurement of coronary plaque. Plaque measurements were performed using semi-automated software (AutoPlaque, Version 2.5, Cedars-Sinai Medical Center, Los Angeles, USA, Figure 1) by two trained observers. AutoPlaque has previously been validated against intravascular ultrasound.³

Figure 1:

[A] Analysis of plaque burden in the left main stem of a 76-year-old male. [B] Vessel wall delineated in red dotted line. [C] Once vessel wall confirmed (green dotted line) and calcified plaque identified (in yellow), vessel lumen delineated in blue [D] Result of analysis with non-calcified plaque volume of 231.5 mm³ (highlighted in red) and calcified plaque volume of 12.0 mm³ (highlighted in yellow)

Coronary artery centerlines were extracted using a semi-automated method. A region of interest was placed in the proximal aorta to define blood pool attenuation. Where possible, the entire length of each major artery was extracted as well as major tributary branches with visible disease. Coronary artery segments were manually defined by each observer independently, using side-branches to mark progression from proximal to distal segments consistently across both scans according to the Society of Cardiovascular Computed Tomography (SCCT) guidelines.¹¹ The length of the distal segment was dependent on vessel diameter with a 2-mm minimum threshold for inclusion.¹² Stented segments and graft insertion points were excluded from the final analysis.

Image quality was deemed excellent if every coronary artery was clearly visualised, good if one or more artery had artefact that made analysis suboptimal (requiring multiple or manually generated centerlines), limited if one or more artery could not be analysed, and unanalysable if none of the arteries could be analysed. If a segment or an artery was deemed too poor to analyse in the baseline scan, it was excluded from analysis in the repeat scan, and vice versa. Two blinded observers independently analysed all scans. One observer analysed the baseline scans twice, 3 months apart, in random order to minimise recall bias.

The vessel lumen, wall and plaque constituents were automatically detected with manual adjustments performed where required. Scan-specific Hounsfield unit (HU) thresholds were applied to define non-calcified and calcified plaque constituents as described previously.^13,14 Low-attenuation plaque was defined by a fixed attenuation of <30 HU as this demonstrates the best correlation with intravascular ultrasound.¹⁵ For each patient, plaque volume (mm³) and burden (normalized to the vessel volume, as a percentage) were calculated for total, non-calcified, calcified and low-attenuation plaque subtypes, as well as diameter stenosis and the maximal remodelling index.

2.4. Statistical Analysis

Data are expressed as mean ± standard deviation or mean (95% confidence interval) for continuous variables or median [interquartile range] where not normally distributed. Categorical variables are presented as number (percentage). Data were analysed by paired two-sided t-tests, linear regression analysis and Lin’s concordance correlation coefficients. Repeatability coefficient was defined as 1.96 x the standard deviation of the differences. Coefficient of variation was defined as the average of means divided by the standard deviation of mean difference. Reproducibility was determined using Bland-Altman analysis and bias (mean difference) is presented alongside 95% limits of agreement (95% confidence intervals). Statistical analysis was performed using GraphPad Prism (Version 8.0, GraphPad Software, San Diego, California, USA). Two-sided P-value <0.05 was considered to be statistically significant.

3. RESULTS

Study participants were predominantly male (n=17; 85%) with a mean age of 69±7 years (Table 1). All but one participant had previously undergone either percutaneous coronary intervention or coronary artery bypass graft surgery. The mean time between CCTA scans was 12±4 days and heart rate at baseline and during interval scanning were comparable (57±8 and 57±10/min respectively, Table 2). The median calcium score was 371 [154–1183] Agatston units. Two participants were excluded from the final analysis due to inadequate image quality of one of the paired scans. Branches without visible disease were not analysed, as such, from a total of 177 potential paired segments, 149 (84%) were analysed, the remaining 28 pairs were excluded either due to poor image quality or stent placement.

Table 1:

Participant Characteristics

	Number of patients (n=20)
Mean age (years)	69 ± 7
Male	17 (85%)
Hypertension	14 (70%)
Hypercholesterolaemia	20 (100%)
Diabetes	2 (10%)
Peripheral vascular disease	1 (5%)
Family history of CHD	12 (60%)
Previous acute coronary syndrome	13 (65%)
Previous percutaneous coronary stenting	13 (65%)
Coronary artery bypass graft	9 (45%)

Mean ± standard deviation or number (%)

Table 2:

Scan characteristics

	Number of patients (n=20)
Time between scans (days)	12 ± 4
Heart rate at baseline scan	57 ± 8
Heart rate at interval scan	57 ± 10
Difference of heart rate	4 ± 3
Median coronary calcium score (Agatston)	371 (154–1183)
Image quality – Excellent	10 (50%)
Image quality – Good	5 (25%)
Image quality – Limited	3 (15%)
Image quality – Unanalysable	2 (10%)
Total patients analysed	18
Total segments analysed	149

Mean ± standard deviation, number (%), or median (interquartile range)

3.1. Intraobserver Repeatability

There were no differences between mean volumes for total, non-calcified and calcified plaque on repeated analysis by the same observer, (p = 0.29, 0.28 and 0.68 respectively, Table 3). The mean volume of low-attenuation plaque was 13±13 mm³ on the first analysis and 12±13 mm³ on the second (mean difference −1.6 mm³, p=0.02). Correlation was excellent for all plaque volume measurements with Lin’s concordance correlation coefficient ranging from 0.97 to 1.0. The coefficients of variation were 2.6%, 3.3%, 4.1% and 21.4% for total, non-calcified, calcified and low-attenuation plaque volumes. Per patient repeatability coefficient ranged between 5.2 and 114.1 mm³. Bland Altman plots highlighting limits of agreement (Figure 2) demonstrate the excellent intraobserver agreement throughout plaque subtypes.

Table 3:

Intraobserver, Interobserver and Scan-Rescan variability of per patient plaque volume measurements.

Intraobserver Variability	Observation 1	Observation 2	LCC (95% CI)	Mean difference	P value	Bland-Altman 95% LoA (95% CI)	CoR
Total Plaque (mm3)	2063 ± 1246 [1443–2683]	2077 ± 1260 [1450–2704]	1.0 (1.0 – 1.0)	14.1	0.29	−93 (−155, −58) to +121 (87, 183)	107.0
Non-calcified Plaque (mm3)	1765 ± 910 [1312–2217]	1780 ± 921 [1322–2238]	1.0 (1.0 – 1.0)	15.4	0.28	−99 (−165, −62) to +129 (93, 196)	114.1
Calcified Plaque (mm3)	298 ± 425 [87–510]	297 ± 425 [86–508]	1.0 (1.0 – 1.0)	−1.2	0.68	−25 (−39, −17) to +23 (15, 36)	23.7
Low-attenuation Plaque (mm3)	13 ± 13 [7–20]	12 ± 13 [5–18]	0.97 (0.93 – 0.99)	−1.6	0.02	−7 (−9, −5) to +4 (1, 6)	5.2
Interobserver Variability	Investigator 1	Investigator 2	LCC (95% CI)	Mean difference	P value	Bland-Altman 95% LoA (95% CI)	CoR
Total Plaque (mm3)	2063 ± 1246 [1443–2683]	1985 ± 1161 [1443–2683]	0.97 (0.93 – 0.99)	−74.2	0.24	−615 (−927, −442) to +459 (286, 770)	537.0
Non-calcified Plaque (mm3)	1765 ± 910 [1312–2217]	1688 ± 803 [1289–2087]	0.95 (0.88 – 0.98)	−76.5	0.24	−594 (−894, −428) to +441 (275, 742)	517.6
Calcified Plaque (mm3)	298 ± 425 [87–510]	297 ± 432 [82–512]	1.0 (0.99 – 1.0)	−1.7	0.84	−69 (−97, −41) to 66 (38, 94)	67.5
Low-attenuation Plaque (mm3)	13 ± 13 [7–20]	14 ± 12 [8–20]	0.90 (0.76 – 0.96)	1.2	0.39	−10 (−14, −5) to +12 (8, 17)	11.0
Scan-Rescan Variability	Baseline Scan	Repeat Scan	LCC (95% CI)	Mean difference	P value	Bland-Altman 95% LoA (95% CI)	CoR
Total Plaque (mm3)	2063 ± 1246 [1443–2683]	2027 ± 1223 [1419–2635]	0.98 (0.94 – 0.99)	−35.6	0.56	−531 (−735, −328) to 460 (257, 664)	495.7
Non-calcified Plaque (mm3)	1765 ± 910 [1312–2217]	1760 ± 928 [1299–2221]	0.97 (0.93 – 0.99)	−4.3	0.93	−418 (−587, −248) to +409 (239, 579)	413.4
Calcified Plaque (mm3)	298 ± 425 [87–510]	267 ±377 [79–455]	0.96 (0.91 – 0.98)	−31.3	0.22	−237 (−321, −152) to +174 (90, 258)	205.2
Low-attenuation Plaque (mm3)	13 ± 13 [7–20]	10 ±13 [4–17]	0.85 (0.64 – 0.94)	−2.6	0.11	−16 (−21, −10) to +11 (5, 16)	13.1

Mean ± standard deviation [95% confidence interval], LCCC – Lin’s concordance correlation coefficient (95% confidence interval), LoA – Limits of agreement (95% confidence interval), CoR - coefficient of repeatability.

Figure 2: Intraobserver variability:

Panel [A–D] Bland-Altman plots demonstrating 95% limits of agreement.

3.2. Interobserver Reproducibility

There were no differences between mean volumes for total, non-calcified, calcified and low-attenuation plaque volumes when comparing analysis done by two independent investigators (p = 0.24, 0.24, 0.84 and 0.39 respectively, Table 3). There was excellent correlation for total, non-calcified and calcified plaque volumes measurements, Lin’s coefficient 0.97, 0.95 and 1.0 for total, non-calcified and calcified plaque volumes respectively. Correlation was modest with low-attenuation plaque volumes, Lin’s coefficient 0.90. The coefficients of variation were 13.5%, 15.3%,11.6% and 41.0% for total, non-calcified, calcified and low-attenuation plaque volumes. Per patient repeatability coefficients ranged from 11 to 537 mm³. Bland Altman plots (Figure 3) highlight excellent agreement particularly for total and non-calcified plaque volumes. Agreement was lower for low-attenuation plaque volume with 95% limits of agreement ranging from −9.9 to +12.2 mm³.

Figure 3: Interobserver variability:

Panel [A–D] Bland-Altman plots demonstrating 95% limits of agreement.

3.3. Interscan Reproducibility and Coronary Characteristics

The mean total plaque volume was similar on the baseline and repeated scan (2063±1246 and 2027±1223 mm³ respectively, p = 0.56). Similarly, no difference was noted between mean plaque volume for non-calcified, calcified and low-attenuation plaque volume (Table 3). Correlations for plaque volumes were excellent for total, non-calcified and calcified plaque volumes (Lin’s coefficient 0.98, 0.97 and 0.96 respectively) but lower for low-attenuation plaque (Lin’s coefficient 0.85). The 95% limits of agreement were narrow for total plaque volume and non-calcified plaque volume but wider for calcified plaque (−236.6 to +174 mm³) and for low-attenuation plaque (−15.8 to +10.5 mm³) (Figure 4). The coefficients of variation were 12.4%, 12.0%, 37.0% and 56.8% for total, non-calcified, calcified and low-attenuation plaque volumes respectively. Per patient repeatability coefficients ranged from 13 to 496 mm³.

Figure 4: Interscan reproducibility:

Panel [A–D] Bland-Altman plots demonstrating 95% limits of agreement.

4. DISCUSSION

In this scan-rescan reproducibility study of patients with advanced coronary artery disease, we have demonstrated that coronary plaque volume can be precisely quantified using semi-automated analysis software. Non-calcified plaque has excellent inter- and intraobserver agreement and scan-rescan reproducibility. Low-attenuation plaque and calcified plaque performed less well on repeated testing with narrow absolute but wide relative 95% limits of agreement. Thus, semi-automated plaque volume quantification is a robust and reproducible method that could be used as valuable and precise measure of disease burden and disease progression in patients with advanced coronary disease.

The current standard for the assessment of coronary atherosclerosis using CCTA requires visual assessment of luminal stenosis severity alongside plaque characterization (calcified, non-calcified or mixed).¹¹ Whilst these factors are undoubtedly important, recent studies have highlighted the incremental prognostic value of quantifying subtypes of atherosclerotic plaque.^12,16–18 Previous iterations of plaque quantification software were time consuming and have limited their widescale applicability.¹⁹ In recent years, the use of a more rapid semi-automated analysis of plaque has expanded and provides a reproducible analytical method, especially when used in patients with a relatively low disease burden.^6,7 However, as disease burden increases, the prominence of coronary calcification rises and the overall image quality is often suboptimal. Our study established that despite these factors, semi-automated plaque analysis remained reliable and reproducible, particularly for measuring non-calcified plaque.

In our study cohort, all patients had multivessel disease and 95% had previous coronary revascularisation. Within a two-week window, scans were performed prospectively at the same site using identical reconstruction protocols to ensure consistency. The quality of acquired images was variable, with only 50% being deemed acceptable despite excellent heart rate control at both baseline and interval scans.²⁰ This is perhaps unsurprising considering the high coronary calcium score, extensive use of coronary revascularisation, and the advanced nature of the coronary artery disease. Reflecting this advanced disease, plaque volumes were at least 5 times larger than in any other previous reproducibility study performed to date. The highest mean total plaque volume reported by Schuhbaeck and colleagues in their study of observer variability was only 399±247 mm³ compared to our 2063±1246 mm³.⁶

Previous studies have established that the assessment of stenosis severity on CCTA has excellent agreement between observers.²¹ However, the observer variability of visual plaque analysis appears to be poor.²² Quantitative plaque analysis removes many of the limitations of visual assessment, and has been shown to be reproducible in patients with low and intermediate disease severity. Ovrehus et al showed excellent scan-rescan, interobserver and intraobserver agreement for the quantitative assessment of non-calcified and calcified plaque volumes.⁷ Tzolos et al similarly showed excellent intraobserver and interobserver repeatability for the assessment of non-calcified plaque burden.²³ However, these studies involved patients with less severe coronary artery disease and lower total plaque volumes. Our study adds to this literature by establishing that the observer and scan-rescan agreement for total and non-calcified plaque volume persists in patients with advanced coronary artery disease.

The narrow absolute and wide relative limits of agreements seen with low-attenuation plaque reflect the smaller volumes measured. As a consequence, small absolute changes in volumes have a large impact on relative and proportionate changes, translating into relatively large coefficients of variability. Low-attenuation plaque accounted for less than 1% of total plaque volume. In patients who have high use of statin and preventative therapies, this is perhaps not surprising as the phenotypic transformation from low-attenuation to calcified plaque may have already occurred. Moreover, with 65% of patients having had previous percutaneous coronary intervention, much of the ‘at-risk’ plaque may have already been treated and excluded from our analysis. Our coefficient of variation of 21 to 41% for low-attenuation plaque is lower than previous reports of 30 to 57% in patients with a lower burden of disease.²⁴ To our knowledge scan-rescan variation for low-attenuation plaque has not previously been reported and as anticipated, was slightly higher than intraobserver and interobserver variability. The presence of low-attenuation plaque can identify patients at high risk of future cardiovascular events, so whilst its measurement is more variable, this does not detract from its potential clinical utility. However, in this population of people with advanced and well treated disease, measuring progression of low-attenuation plaque may be challenging as small volumes do not allow for precise reproducible quantification. Trialists have begun to use low-attenuation volume as an endpoint to monitor the effect of novel therapies on coronary atherosclerosis.⁵ We demonstrate here, that in patients with advanced coronary disease, non-calcified plaque volume is potentially a more reliable metric to use when monitoring disease progression.

Our study has some limitations. The number of patients was relatively small although repeated scanning and radiation exposure does present challenges to conducting such a study in larger numbers of patients. The population was predominantly male, but the results of the plaque reproducibility would be expected to be similar irrespective of gender. We did not compare plaque volumes with a reference standard, such as intravascular ultrasound, although this has been previously reported by others.^3,25

In conclusion, we have demonstrated the excellent intraobserver, interobserver and scan-rescan reproducibility of semi-automated plaque volume quantification in patients with advanced coronary artery disease. This validates its use as a novel approach to quantify change in coronary artery disease over time and optimise risk stratification in patients with coronary artery disease.

ABBREVIATIONS

CCTA

Coronary computed tomography angiography

DICOM

Digital Imaging and Communications in Medicine

HU

Hounsfield unit

SCCT

Society of Cardiovascular Computed Tomography