Observer repeatability and interscan reproducibility of 18F-sodium fluoride coronary microcalcification activity

Abstract

Background

We aimed to establish the observer repeatability and interscan reproducibility of coronary ¹⁸F-sodium-fluoride positron emission tomography (PET) uptake using, a novel semi-automated approach, coronary microcalcification activity (CMA).

Methods

Patients with multivessel coronary artery disease underwent repeated hybrid PET and computed tomography angiography (CTA) imaging (PET/CTA). CMA was defined as the integrated standardized uptake values (SUV) in the entire coronary tree exceeding 2 standard deviations above the background SUV. Coefficients of repeatability between the same observer (intraobserver repeatability), between 2 observers (interobserver repeatability) and coefficient of reproducibility between 2 scans (interscan reproducibility), were determined at vessel and patient level.

Results

In 19 patients, CMA was assessed twice in 43 coronary vessels on two PET/CT scans performed 12±5 days apart. There was excellent intraclass correlation for intraobserver and interobserver repeatability as well as interscan reproducibility (all ≥0.991). There was 100% intraobserver, interobserver and interscan agreement for the presence (CMA>0) or absence (CMA=0) of coronary¹⁸F-NaF uptake. Mean CMA was 3.12±0.62 with coefficients of repeatability of ≤10% for all measures: intraobserver 0.24 and 0.22, interobserver 0.30 and 0.29 and interscan 0.33 and 0.32 at a per-vessel and per-patient level respectively.

Conclusions

CMA is a repeatable and reproducible global measure of coronary atherosclerotic activity.

Introduction

Hybrid positron emission tomography and computed tomography (PET/CT) imaging with ¹⁸F-sodium fluoride (¹⁸F-NaF) can be used as a marker of developing microcalcification across multiple different vascular and valvular disease states (1–6). It has been used to assess atherosclerotic disease activity in the coronary arteries, with the potential to identify high-risk plaques (1, 7–10). To date, most studies have reported coronary ¹⁸F-NaF PET uptake using maximum target to background ratio (TBR_MAX) (3, 7, 9, 11, 12). While TBR_MAX has been used as the primary measure for individual lesions (11, 13), its measurement can vary depending on plaque activity, the anatomical location of the background activity measured, and the partial volume effects of such small structures (14, 15).

We have developed a novel semi-automated approach to measure ¹⁸F-NaF uptake throughout the entire coronary vasculature. This method uses centerlines defined by coronary CT angiography to build 3-dimensional tubular volumes of interest around each of the main epicardial coronary arteries (8) and thereby derive a single summary measure of total coronary microcalcification activity (CMA). This method allows evaluation of coronary ¹⁸F-NaF activity on a per-vessel and per-patient basis, providing a global assessment of disease activity in the coronary arteries that is akin to the Agatston coronary artery calcium score for CT-defined coronary macrocalcification (8). CMA appears to correlate more closely with established CT-derived markers of plaque vulnerability than TBR_MAX (9). However, the observer repeatability and interscan reproducibility of CMA has yet to be established. In this study, we aimed to evaluate the intra- and interobserver repeatability and interscan reproducibility of CMA.

Materials and Methods

Study Population

Twenty patients underwent two ¹⁸F-NaF PET/CT examinations of the coronary arteries within three weeks of each other as a part of the Dual Antiplatelet Therapy to Inhibit Coronary Atherosclerosis and Myocardial Injury in Patients with Necrotic High-risk Coronary Plaque Disease (DIAMOND) Study (NCT02110303) (16). Inclusion in the study required angiographically confirmed multivessel coronary artery disease, defined by either previous revascularization or stenosis >50% in at least two major epicardial coronary arteries. Exclusion criteria were an acute coronary syndrome within 12 months prior to the examination, renal dysfunction (estimated glomerular filtration rate ≤30 mL/min/1.73 m²) and contraindication to iodinated contrast media. This study was approved by the Scottish research ethics committee (REC reference: 14/SS/0089) and written informed consent was obtained from all participants.

Image Acquisition Protocol (PET/CT acquisition)

All patients underwent two ¹⁸F-NaF PET/CT scans within three weeks.

PET acquisition

Patients underwent a 30-min list-mode PET-emission acquisitions approximately 1 h after injection of 250 MBq of ¹⁸F-NaF. All patients were scanned with arms positioned above the head in a 128-slice Biograph mCT system (Siemens Healthineers, Knoxville, TN, USA). A low-dose CT for attenuation correction was acquired immediately before the PET acquisition (120 kV, 50 mAs, 3-mm slice thickness). All patients were imaged with 3-lead electrocardiogram cardiac gating.

CCTA acquisition

For anatomical localization of PET uptake, coronary computed tomography angiography (CCTA) was performed immediately after the PET acquisition. The CCTA was performed using prospective gating, 330 ms rotation time, body-mass index (BMI) dependent voltage (<25 kg/m², 100 kV; ≥25 kg/m², 120 kV), and tube-current time product of 160–245 mAs. Patients were administered oral or intravenous beta-blocker therapy to achieve a target heartrate of <60 beats/min. A BMI-dependent bolus-injection of contrast media (400 mg/mL) was administered to the patients with a flow of 5–6 mL/s after determining the appropriate trigger delay defined by a test bolus of 20 mL of contrast material.

Image Analysis Protocol

PET reconstruction

PET images were reconstructed into 4 cardiac phases using a vendor provided software (JS-Recon12, Siemens, Knoxville, TN, USA). All PET image reconstructions were performed with corrections for time-of-flight and point-spread function. Using 4 cardiac gates, we reconstructed the data on a 256 × 256 matrix (109 slices, slice thickness 2.027 mm) using 2 iterations, 21 subsets and 5-mm Gaussian filter.

Cardiac Motion correction

Cardiac motion corrected images were obtained from the gated PET reconstructions through PET-PET image co-registration using a diffeomorphic registration and dedicated software (FusionQuant version 1.19.2.7, Cedars-Sinai Medical Center) (17).

Image registration

Prior to image analysis, the PET reconstructions were registered to the CCTA images, using a rigid translation of the PET images. The PET to CCTA registration was ensured using five key points of reference: sternum, vertebrae, blood pool in the left and right ventricle (based upon high ¹⁸F-NaF activity in the blood pool in comparison to the surrounding myocardium), and the great vessels (18).

Blood clearance correction

To minimize the impact of variations in background blood pool activity introduced by the injection-to-scan delays (13, 19), we standardized the background blood pool activity to an injection-to-scan delay of 60 minutes using a previously described correction factor (13) (Equation 1):

$$\mathit{\text{SUVBackground}}\mathit{\text{corrected}}=\mathit{\text{SUVBackground}}*e^{\left(-0.004*\left(60-t\right)\right)}$$ (Equation 1)

where t represents the injection-to-scan delay in minutes.

CMA quantification

Based upon PET image analysis techniques widely used in oncology and cardiac sarcoidosis (20–22) as well as the Agatston method for quantifying coronary CT calcium scores, we developed a novel measure to assess whole vessel activity in the coronary tree (8). To obtain the CMA values, two distinct steps were performed. First, we selected the proximal and distal end of the vessel (>2 mm) and applied a vessel tracking algorithm to extract whole-vessel tubular 3D volumes of interest from CCTA using dedicated semi-automated Autoplaque software (version 2, Cedars-Sinai Medical Center, Los Angeles, CA)(23) (Figure 1). These encompass all the main epicardial coronary vessels and their immediate surroundings (4-mm radius) facilitating per-vessel and per-patient uptake quantification. In a tubular VOI, along the extracted centerlines, with 4-mm radius, we measured the CMA on the PET/CCTA co-registered images. For this study, we evaluated ¹⁸F-NaF activity along the entire course of coronary arteries regardless of the presence of coronary stents, and we included the left main in the left anterior descending artery VOI. To avoid overspill of aortic root activity (spill-over effects), we excluded coronary activity in the orifice of the left main stem of being incorporated in the analysis. CMA was defined as the average SUV within the activity volume above threshold of background SUV mean +2 standard deviations. The background activity was measured in the right atrium.

Figure 1.

Three-dimensional rendering of coronary computed tomography (CT) angiography with superimposed tubular whole vessel volumes of interest (light green) employed for evaluation of ¹⁸F-sodium fluoride (¹⁸F-NaF) positron emission tomography (PET) uptake (blue and red). Despite the relatively low maximum tissue-to-background ratio (TBR_MAX) due to multiple foci of increased ¹⁸F-NaF activity, the coronary microcalcification activity (CMA) in the right coronary artery (RCA) is higher than in the left anterior descending (LAD) coronary artery which presented with a higher TBR_MAX (LCx= Left circumflex artery).

TBR_MAX quantification

CTA studies were assessed visually for percent stenosis according to the Society of Cardiovascular Computed Tomography guidelines (24). For a signal to be co-localized to a coronary artery, an atherosclerotic plaque had to be present on CT angiography, and the increased pattern of radiotracer had to arise from the coronary artery and follow its course over >5 mm in three dimensions on orthogonal views (25). On the co-registered PET and CTA images, ¹⁸F-NaF PET uptake was measured in all coronary segments with a vessel diameter ≥2 mm and >25% stenosis as defined by CTA. The ¹⁸F-NaF uptake in these lesions was evaluated in a 3D spherical volume of interest (VOI) (radius 5 mm). In all plaques meeting these criteria, the maximum standardized uptake values (SUV_MAX) were measured within manually drawn regions of interest. TBR_MAX values were calculated by dividing the coronary SUV_MAX by the blood pool activity measured in the right atrium (cylindrical volume of interest radius 10 mm and thickness 5 mm) at the level of the right coronary artery ostium.

Diagnostic Evaluation of CMA and TBR_MAX

Using CMA, the individual coronary arteries were marked as ¹⁸F-NaF positive if CMA>0 and negative if CMA =0 (8). We also assessed the burden of activity on a per-vessel and per-patient level. To allow a per-patient analysis, we added the CMA activity of all major epicardial vessels (CMA_total). For TBR, PET uptake was quantified based on the CCTA lesion position, with lesions categorized as ¹⁸F-NaF-positive (TBR_MAX ≥1.25) or ¹⁸F-NaF-negative (TBR_MAX <1.25) (3, 7, 26). In order to compare TBR_MAX with CMA, we plotted the percentage differences in Bland-Altman plots.

Observer Repeatability and Interscan Reproducibility

Two anonymized scans for each of the 20 patients were presented to 2 experienced observers in random order. First, repeat assessments (observations 1 & 2) were performed by observer 1 at least 12 weeks apart in random order to prevent recall bias (intraobserver repeatability). Secondly, a second observer (observer 2) performed analysis for both scans and we compared them with measurements performed by the first observer (interobserver repeatability). Lastly, we compared the measurements between baseline scan (scan1) and repeat scan 2–3 weeks later (scan 2) in order to compute the interscan reproducibility.

Statistical analysis

Data were tested for normality using the Shapiro–Wilk test. Statistical analysis was performed using MedCalc Statistical Software version 16.4.3 (MedCalc Software, Ostend, Belgium). Continuous, normally distributed variables were presented as mean ± standard deviation, whereas non-normally distributed continuous data were presented as median [range]. Assessment of CMA and TBR_MAX, observer repeatability and interscan reproducibility were obtained using descriptive statistics with intraclass correlation coefficient (ICC) as well as Bland–Altman plots with mean bias and limits of agreement (LOA).

Results

Twenty patients were recruited (Table 1), although one patient was excluded from the study due to incomplete list mode PET data. From the 57 coronary arteries included in the study, a total of 49 vessels fitted the size criterion (>2 mm in diameter). We excluded 6 vessels from the analysis: all vessels were the left circumflex coronary artery and were excluded due to spillover of ¹⁸F-NaF activity from neighboring mitral valve calcification (3 from scan 1 and 3 from scan 2), leaving a total of 43 vessels for the final analysis. A total of 47 lesions were identified on CCTA in these 43 vessels. The mean uptake values observed for the two readers and the repeated scans were 3.12± 0.62 for CMA and 1.62±0.49 for TBR_MAX.

Table 1.

Baseline characteristics

Age (years)	70 ± 8
Gender (males)	16 (84)
Body-mass index (kg/m2)	27.6 ± 4.0
Cardiovascular risk factors
Diabetes mellitus (type II)	2 (11)
Current smoker	2 (11)
Hypertension	13 (68)
Hyperlipidemia	19 (100)
Agatston Calcium Score (AU)	1075 [0–1890]

Continuous variables reported as mean ± SD or median and interquartile range [IQR]; categorical variables reported as n (%),

Presence or absence of ¹⁸F-NaF activity

There was 100% intraobserver, interobserver and interscan agreement for the presence of ¹⁸F-NaF activity (CMA>0) or absence of ¹⁸F-NaF activity (CMA=0) (Table 2). In addition, there were excellent intraclass correlations between the same observer, different observers and between scans (all intraclass correlation coefficient (ICC) >0.99, Table 3).

Table 2.

Intraobserver (A), interobserver (B) and interscan (C) variability for the presence or the absence of ¹⁸F-sodium fluoride uptake (coronary microcalcification activity) at a vessel level (n=43).

		Observation* 2			Observation* 2
A Observation 1		CMA>0	CMA=0		TBRMAX≥1.25	TBRMAX<1.25
	CMA>0	34	0	TBRMAX≥1.25	35	2
	CMA=0	0	9	TBRMAX<1.25	1	12
B Observer 1		Observer 2			Observer 2
	CMA>0	34	0	TBRMAX≥1.25	35	1
	CMA=0	0	9	TBRMAX<1.25	2	12
C Scan 1		Scan 2			Scan 2
	CMA>0	34	0	TBRMAX≥1.25	34	2
	CMA=0	0	9	TBRMAX<1.25	2	13

CMA: Coronary Microcalcification Activity, TRB_MAX: maximum target to background ratio

^*Observations 1 and 2 were performed by Observer 1 at least 12 weeks apart in random order to prevent recall bias.

Table 3.

Intraclass correlation coefficient (ICC) for intraobserver and interobserver repeatability and interscan reproducibility for coronary microcalcification activity.

	Per-vessel (95% confidence interval)	Per-patient (95% confidence interval)
Intraobserver	0.997 (0.995–0.998)	0.999 (0.998–0.999)
Interobserver	0.994 (0.991 – 0.997)	0.995 (0.992 – 0.997)
Interscan	0.991 (0.986 – 0.994)	0.993 (0.992 – 0.996)

Intraobserver analysis

There was excellent intraobserver repeatability for CMA measurement on a per-vessel level, with a coefficient of repeatability of 0.24, mean bias of −0.02 (p=0.39) and narrow limits of agreement (95% LOA −0.25 to 0.22; Figure 2A). Similarly, at a patient level, CMA_total repeatability coefficient was 0.22 with a mean bias of −0.03 (p=0.12) and narrow limits of agreement (95% LOA −0.25 to 0.18; Figure 2B).

Figure 2.

Intraobserver repeatability of coronary microcalcification activity at vessel (A) and patient (B) level. [CMA: coronary microcalcification activity, SD: standard deviation, CR: Coefficient of repeatability]

Interobserver analysis

Comparable to the intraobserver analysis, there was also excellent interobserver repeatability for CMA measurement on a per-vessel level, with a coefficient of repeatability of 0.30, mean bias of −0.01 (p=0.79) and narrow limits of agreement (95% LOA −0.31 to 0.29; Figure 3A). At a patient level, CMA_total repeatability coefficient was 0.29 with a mean bias of −0.04 (p=0.17) and narrow limits of agreement (95% LOA −0.33 to 0.25; Figure 3B).

Figure 3.

Interobserver repeatability of coronary microcalcification activity at vessel (A) and patient (B) level. [CMA: coronary microcalcification activity, SD: standard deviation, CR: Coefficient of repeatability]

Interscan analysis

Similar to the interobserver analysis, interscan analysis of CMA showed very good reproducibility. At a vessel level, there was excellent interscan reproducibility with a coefficient of reproducibility of 0.33, mean bias of 0.02 (p=0.89) and narrow limits of agreement (95% LOA −0.31 to 0.35; Figure 4A). Likewise, at a patient level, CMA_total reproducibility coefficient was 0.32 with a mean bias of −0.02 (p=0.30) and narrow limits of agreement (95% LOA −0.34 to 0.28; Figure 4B).

Figure 4.

Interscan repeatability of coronary microcalcification activity at vessel (A) and patient (B) level. [CMA: coronary microcalcification activity, SD: standard deviation, CR: Coefficient of reproducibility]

CMA and TRB_MAX

The activity of separate lesions (n=47) was assessed using maximum target-to-background ratio (TBR_MAX. Compared to TBR_MAX, CMA showed superior limits of agreement and smaller coefficients of repeatability for intraobserver and interobserver analysis, as well as smaller coefficient of reproducibility for interscan analyses. Between the same observer, CMA showed a mean bias and limits of agreement of 0.2% (95% LOA −10.6 to 11.1%) and TBR_MAX showed a bias of −2.4% (95% LOA −20.7 to 15.9%) with coefficients of repeatability of 10.8% and 18.3% respectively (Supplement Figure 1). Similarly, between different observers, CMA was more reproducible than TBR_MAX with a mean bias and limits of agreement of −0.1% (95% LOA −14.1 to 4.0) and −3.75% (95% LOA −25.2% to 17.8%) respectively and coefficients of repeatability of 14.0% and 19.5% (Supplement Figure 2). Finally, between scans, CMA showed a mean bias and limits of agreement of 0.1% (95% LOA −17.4 to 17.2%) and TBR_MAX showed a bias of 0.3% (95% LOA −24.0% to 24.6%) with coefficients of reproducibility of 17.2% and 24.3% respectively (Supplement Figure 3).

Discussion

Coronary ¹⁸F-NaF PET is a non-invasive tool for imaging vulnerable atherosclerotic plaques (7, 8). We evaluated the repeatability and reproducibility of a novel methodology (CMA) for assessing whole vessel coronary ¹⁸F-NaF uptake on PET/CT as a single measure of patient risk (8). We showed that CMA has excellent intra- and interobserver repeatability and interscan reproducibility and narrow limits of agreements within and between scans. This suggests that CMA is sufficiently robust to be used as single measure of coronary atherosclerotic activity and has the potential to provide a summary score of coronary risk similar to that described by the Agatston coronary artery calcium score (27).

It is now widely accepted that the anatomic assessment of the whole-coronary disease burden (vulnerable patient) is of greater importance than the identification of a single vulnerable plaque (28–30). We have shown that both TBR_MAX and CMA assessments were repeatable and reproducible, albeit CMA had the best repeatability and reproducibility. This has important implications for serial scanning and monitoring disease progression with coronary PET. Furthermore, there was no diagnostic discordance between observers or scans for CMA, something that is not always true for TBR_MAX. For the presence of ¹⁸F-NaF activity (CMA>0) or absence of activity (CMA=0), there was 100% agreement in the categorization between observers or scans with no crossover of cases observed (Table 2). On the other hand, there were 3 discordant values for TBR_MAX between observers and 4 discordant values between scans. This is of high importance in populations without known coronary artery disease if ¹⁸F-NaF is to be used as a screening tool where little or no activity is expected. In such population the fact that CMA=0 is reproducible between observers and scans is of extreme importance as it could reassure the physician.

This is the first study investigating the repeatability and interscan reproducibility of both CMA and TBR_MAX. In a previous study (11), we showed that TBR_MAX displayed good observer repeatability and interscan reproducibility when employing mid-diastolic PET images. This can be further improved using cardiac motion-corrected reconstructions (13). However, we have here reported narrower limits of agreement for observer repeatability and interscan reproducibility, both at a vessel as well as at a patient level.

Another strength of CMA is that it integrates the total activity of all 3 vessels in a score that resembles the well-established calcium score; i.e. a patient with CMA= 0 has no active microcalcification similar to a patient with zero calcium score. Unlike TBR, CMA measures patient-level total ¹⁸F-NaF activity burden assessment and hence does not rely on a single hot pixel-value leading to better reproducibility. Furthermore, by identifying the coronary artery borders from CTA and limiting the assessment of ¹⁸F-NaF uptake within these borders, the CMA approach has the strength to reduce the subjectivity and the time required to perform image analysis.

Despite the broad application of the individual lesion assessments using TBR_MAX (3, 7, 9, 11–13, 25, 30), this technique can only provide an estimation of activity at an individual plaque level, and as a result, the overall coronary disease burden of active macrocalcification cannot be appreciated. Indeed, this approach bears the risk of underestimating the ¹⁸F-NaF activity in patients with multiple plaques with increased tracer activity across the coronary tree (Figure 5). Using CMA, and computing the total activity burden, we were able to translate ¹⁸F-NaF PET tracer uptake into the total coronary macrocalcification activity mirroring an approach successfully applied in the field of oncology and cardiac sarcoidosis (metabolic activity volume) (20–22). Thus, CMA provides a measure of the overall burden of disease activity for the patient.

Figure 5.

Paired coronary tomography images (CT) and fused positron emission tomography (PET) /CT images of a representative patient with multiple lesions across the coronary tree assessed by two observers and with repeated scans. Panel (A) shows the right coronary artery (RCA) with multiple plaques across the length of the artery and respective ¹⁸F-NaF PET uptake. Tubular whole vessel volumes of interest (light green) employed for evaluation of ¹⁸F-NaF (PET) uptake (bright yellow to red), B) Left main stem and left anterior descending artery (LMS/LAD) with multiple plaques and ¹⁸F-NaF PET uptake and C) left circumflex (LCx) with a calcified proximal plaque and uptake.

[CMA: coronary microcalcification activity, TRB_MAX: maximum target to background ratio]

Finally, despite the excellent prognostic information that coronary calcium scoring provides in asymptomatic individuals (31, 32) and those presenting with chest pain (33), its prognostic capability has been suboptimal in studies involving patients with established advanced coronary artery disease (34, 35). While coronary calcium visualizes advanced and potentially healed disease (macrocalcification), it lacks the ability to visualize active disease (microcalcification) or predict future calcification progression. On the other hand, ¹⁸F-NaF uptake is associated with culprit coronary plaques in patients with myocardial infarction (7), adverse plaque features in patients with apparently stable disease (9) and predicts the future progression of coronary calcium scores, confirming its status as a marker of disease activity (36). In addition, in a small study involving 32 patients, Kitagawa et al (12) showed that higher ¹⁸F‐NaF uptake in established coronary atherosclerotic lesions on PET has a predictive value for future cardiac events (acute coronary syndrome or late coronary revascularization) that was superior to the predictive value of the findings on CCTA, including coronary artery stenosis, high risk plaques and calcification. Moreover our group has recently shown, in a large multicohort study involving 293 patients with established multivessel coronary artery disease, that CMA outperforms coronary calcium score in prediction of myocardial infarction and major adverse cardiovascular events (MACE) (37).

Limitations

This study has several additional limitations; first is the number of patients included in the study (19). Despite the low number of participants, we were able to identify 47 separate lesions and we have evaluated each patient both at a per-vessel level and per-patient level. Another limitation was that we used only cardiac motion corrected images and did not apply gross patient motion and respiratory motion corrections (triple motion corrections)(13). The third limitation of this study was that all lesions were manually delineated by only two expert readers which may affect its broader generalizability. Finally, this study was obtained in a single center using a PET/CT system from one vendor only. A bigger multicenter study including systems from multiple vendors would be required to confirm our findings.

New Knowledge Gained

Using CMA provides an accurate and highly reproducible metric of ^18F-NaF PET/CT uptake. This finding is important to ensure the optimal quantitative accuracy in studies utilizing ¹⁸F-NaF, such as PREFFIR [NCT02278211](11), a study involving 700 patients with multi-vessel coronary artery disease. Based on this finding and given the comparable reproducibility of TBR_MAX, both approaches can be implemented in future clinical trials.

Conclusion

In conclusion, we have shown CMA provides a novel metric of the per-vessel and per-patient coronary ¹⁸F-NaF PET activity that has excellent intraobserver, and interobserver repeatability and interscan reproducibility. We suggest that CMA could be used as a global patient level measure of ¹⁸F-NaF uptake with potential application to clinical trials and clinical practice.

Abbreviations

CMA

coronary microcalcification activity

¹⁸F-NaF

¹⁸F-sodium fluoride

PET

positron emission tomography

CTA

computed tomography angiography

MC

cardiac motion corrected

BC

background blood pool clearance correction

TBR

target to background ratio

SUV

standardized uptake value

TBR_MAX

maximum target to background ratio

SUV_MAX

maximum standardized uptake value

VOI

volume of interest

BMI

body-mass index