Carotid Artery Wall Thickness Measured Using CT: Inter- and Intraobserver Agreement Analysis

SUMMARY:

The purpose of this work was to compare inter- and intraobserver agreement in the analysis of CAWT by using MDCTA. The CAWT in 35 patients was quantified by 4 observers. Bland-Altman statistics were used to measure the agreement between observers. The results of our study demonstrated that the CAWT measured by using MDCTA shows a good reproducibility between observers by considering inter- and intraobserver agreement.

Measurement of the IMT is an established marker for early changes of atherosclerosis,¹ and it was demonstrated that IMT is a strong predictor for cerebrovascular and coronary complications.^2–4 One of the major limitations of IMT is the poor interobserver and intraobserver reproducibility, which can be determined by several parameters such as the type of sonographic scanner and the sonographer's experience.^5–8

In past years, MDCTA was found to be an excellent technique for the analysis of carotid arteries^9–11 with good results in carotid artery stenosis degree quantification,^12,13 plaque composition analysis,^14,15 and identification of complications of plaque such as ulcers.^16–18 In 2008, MDCTA was proposed as a technique to study the CAWT,¹⁹ and an excellent agreement with IMT²⁰ was demonstrated; moreover, a significant association between CAWT and classic cardiovascular risk factors was described.²¹

However, until now, no reproducibility study has been proposed; the purpose of this article was to compare inter- and intraobserver agreement among 4 readers in the analysis of CAWT by using MDCTA.

Technique

Patient Population

In this retrospective study, 35 consecutive symptomatic patients (24 men, 11 women; mean age, 66 years; range, 51–83 years) examined with MDCTA from January 2010 to April 2010 were included. We obtained approval of the institutional review board. This retrospective review evaluated existing clinical data and records. No additional procedures were performed. The review was conducted in accordance with the guidelines of the research committee of our institution.

MDCTA Technique

All patients underwent MDCTA of the supra-aortic vessels by using a 16−detector row CT system (Philips Healthcare, Best, the Netherlands) by using a technique previously described.^19–21 In our protocol for the analysis of carotid arteries, a basal scan was obtained and was followed by the angiographic phase in which 80 mL of contrast medium (iomeprol, Imeron 370; Bracco, Milan, Italy) was injected into a cubital vein, by using a power injector at a flow rate of 5 mL/s and an 18-ga intravenous catheter. A bolus-tracking technique was used to calculate the correct timing of the scan. Dynamic monitoring scanning began 6 seconds after the beginning of the intravenous injection of contrast material, and the region of interest was placed in the aortic arc. The trigger threshold inside the region of interest was set at +90 HU above the baseline. The delay between the acquisitions of each monitoring scan was 1 second. When the threshold was reached, the patient was instructed not to breathe; and after an interval of 4 seconds, the scan was started in the caudocranial direction. CT technical parameters included the following: matrix, 512 × 512; FOV, 14–19 cm; 180–220 mAs; 120–140 kV; section thickness, 1 mm; and gap, 0.5 mm. An intermediate reconstruction filter algorithm (C-filter) was used. The spatial resolution was 0.39 mm. Angiographic acquisition included the carotid siphon. None of the patients included in the study had a medical history of cardiac output failure or any contraindications to iodinated contrast media.

CAWT Evaluation with MDCTA

For the MDCTA examination, both the right and left carotid arteries were measured. Magnification was freely modifiable, and the window level was preset according to Saba et al.^19–21 Three measurements for each carotid artery were performed at the 6, 9, and 12 o'clock positions in the distal common carotid artery, where no evidence of plaque was detected (Fig 1). We measured CAWT between the leading edge of the opacified lumen vessel and the external visible limit of the artery wall, where it was surrounded by adjacent adipose tissue. The individual subject's mean CAWT values were then obtained by averaging the values obtained for each carotid artery. Four different observers (with 11 years, 10 years, 7 years, and 3 years of experience in CT angiography of carotid arteries) analyzed the datasets and measured the CAWT by using dedicated software. Each observer analyzed the dataset twice; the second review was 6 months after the first. Analysis was performed with the observers blinded to each other. The second measurement of each observer was used to assess intraobserver reproducibility.

Fig 1.

MDCTA axial image of a 63-year-old woman. The white arrow indicates the CAWT of the left common carotid artery. Panel A is a 100% zoom of the CAWT.

Statistical Analysis

The Kolmogorov-Smirnov Z-test for the distribution the normality of each continuous variable group was calculated. Continuous data were described as the mean value ± SD, and they were compared by using a Student t test for paired samples. Inter- and intraobserver agreement was evaluated by using the Bland-Altman analysis.²² A folded empiric cumulative distribution plot (mountain plot) was also calculated. A P value < .05 was considered significant. R software (www.r-project.org) was used for statistical analyses.

Results

General Analysis

In Table 1, the summary statistics of the 8 groups of CAWT measurements are given. In all groups, the Kolmogorov-Smirnov Z-test demonstrated the normality of the distribution. The minimum CAWT value ranged from 0.45 to 0.72 mm, whereas the maximum CAWT value ranged from 1.69 to 2.09 mm. By applying the Student t test for paired groups only in 1 case, a statistically significant difference between groups was detected (Tables 2 and 3).

Table 1:

Summary statistics table

	Mean	95% CI	Variance	SD	RSD	SEM	Median	95% CI	Minimum	Maximum	2.5–97.5 P	5–95 P	Normal Distr.
OBS 1 first	1.23	1.158–1.292	0.0853	0.292	0.238	0.034	1.28	1.117–1.360	0.67	1.83	.720–1.712	.732–1.665	0.251
OBS 1 second	1.28	1.212–1.342	0.0793	0.282	0.221	0.033	1.32	1.227–1.423	0.72	2.09	.739–1.811	0.828–1.708	0.883
OBS 2 first	1.21	1.145–1.273	0.0783	0.28	0.232	0.032	1.23	1.132–1.320	0.61	1.87	.650–1.768	.726–1.679	0.692
OBS 2 second	1.19	1.126–1.243	0.0647	0.254	0.215	0.029	1.22	1.113–1.320	0.65	1.8	.684–1.689	.757–1.623	0.636
OBS 3 first	1.21	1.134–1.279	0.0989	0.315	0.261	0.036	1.21	1.107–1.340	0.45	1.89	.632–1.810	.720–1.670	0.478
OBS 3 second	1.2	1.130–1.261	0.0809	0.285	0.238	0.033	1.21	1.090–1.320	0.54	1.87	.629–1.761	.690–1.575	0.632
OBS 4 first	1.24	1.182–1.305	0.0709	0.266	0.214	0.031	1.32	1.194–1.413	0.66	1.69	.760–1.639	.770–1.602	0.092
OBS 4 second	1.25	1.179–1.311	0.0825	0.287	0.231	0.033	1.26	1.147–1.323	0.65	1.83	.708–1.769	.760–1.717	0.36

Note:—OBS indicates observer; RSD, relative standard deviation; SEM = standard error of the mean; CI, confidence interval; Distr., distribution.

Table 2:

T test analysis

Second Measurement P Value	First Measurement P Value
	OBS 1	OBS 2	OBS 3	OBS 4
OBS 1	.126	.319	.077	.145
OBS 2	.061	.289	.321	.094
OBS 3	.171	.571	.407	.027a
OBS 4	.366	.059	.073	.837

Note:—OBS indicates observer.

^aStatistically significant value.

Table 3:

T test analysis

First Measurement		Second Measurement
Observer	P Value	Observer	P Value
OBS 1-OBS 2	.319	OBS 1-OBS 2	.214
OBS 1-OBS 3	.389	OBS 1-OBS 3	.158
OBS 1-OBS 4	.301	OBS 1-OBS 4	.099
OBS 2-OBS 3	.889	OBS 2-OBS 3	.567
OBS 2-OBS 4	.076	OBS 2-OBS 4	.121
OBS 3-OBS 4	.105	OBS 3-OBS 4	.108

Note:—OBS indicates observer.

Bland-Altman Analysis

We evaluated the measurement reproducibility by using the Bland-Altman analysis, and 36 plots were generated. The intraobserver agreement analysis is given in Fig 2, whereas the interobserver agreement is given in Figs 3 and 4. In the intraobserver analysis, the Bland-Altman analysis demonstrated good results with a measurement error variable from 0.05 mm to values close to 0 mm, whereas the 95% limits of agreement were from 0.22 to 0.46 mm. By analyzing the interobserver agreement, we detected a measurement error variable from 0.09 mm to values close to 0 mm, whereas the 95% limits of agreement were from 0.24 to 0.44 mm.

Fig 2.

Bland-Altman plot for intraobserver agreement.

Fig 3.

Bland-Altman plot for interobserver agreement in the first measurement.

Fig 4.

Bland-Altman plot for interobserver agreement in the second measurement.

Mountain Plot Analysis

A folded empiric cumulative distribution plot for interobserver analysis is given in Fig 5, and the plots demonstrated that the percentiles are distributed according to a normal distribution.

Fig 5.

Mountain plot analysis for the intraobserver agreement.

Discussion

The purpose of this article was to compare inter- and intraobserver agreement in the analysis of CAWT by using MDCTA. In past years, MDCTA was proposed as a technique to study CAWT,¹⁹ and an excellent agreement with IMT²⁰ and a significant association between CAWT and classic cardiovascular risk factors were described.²¹

In this study the minimum CAWT value ranged from 0.45 to 0.72 mm, whereas the maximum CAWT value ranged from 1.69 to 2.09 mm with a mean value of 1.26 mm. These values are higher compared with those in a previous article¹⁹; this difference can be ascribed to the fact that our patient cohort was composed by symptomatic patients (and patients with cerebral symptoms have a thicker CAWT compared with the asymptomatic ones^19–21), whereas in the study of Saba et al,¹⁹ only 45.6% (99/217) of the analyzed patients were symptomatic.

We compared the measurement performed by the 4 readers by applying the Student t test for paired groups to check whether there was a statistically significant difference. Therefore, we tested 28 combinations (Tables 2 and 3), and only in 1 case (3.57%) was a statistically significant difference observed (between the first measure of observer 4 and the second measurement of observer 3); these data suggest that the group analyses are homogeneous and similar and that the measures obtained in each population we analyzed are equivalent.

To evaluate the reproducibility of CAWT, we used the graphic method of Bland-Altman statistics to compare the 2 measurement techniques. In this graphic method, the differences between the 2 techniques are plotted against the averages of the 2 techniques. In the intraobserver analysis (Fig 2), the Bland-Altman analysis demonstrated good results with a measurement error variable from 0.05 mm to values close to 0 mm, whereas the 95% limits of agreement had a variability from 0.22 to 0.46 mm. By analyzing the interobserver agreement (Figs 3 and 4), we detected measurement error variability from 0.09 mm to values close to 0 mm, whereas the 95% limits of agreement had a variability from 0.24 to 0.44 mm. These results indicate that in some cases, there was a big difference between measurements (in the same observer and between different observers).

We suggest 3 potential causes for this fact: halo effect, edge blur effect, and spatial resolution. In the analysis of the carotid vessel, 2 of the most recurrent artifacts connected with endoluminal contrast injection are the so called “halo” and the “edge blur.”^23,24 “Edge blur” refers to the transition or sharpness of the outer luminal margin as a percentage of the luminal diameter. “Halo artifacts” refer to periluminal increased attenuation (partially saturated pixels).²³ Actually the tendency is to use speed flows (≥3 mL/s) to obtain a major intraluminal opacification and, therefore, a better postprocessing visualization. Moreover, a high intraluminal Hounsfield unit value allows a clear evaluation of luminal shape by producing a high-contrast interface between the contrast medium and vascular wall. In 1997, Claves et al²³ reported, using a phantom, intraluminal values that ranged between 150 and 200 HU as optimal values for a correct evaluation of stenosis degree. Nevertheless high Hounsfield unit values lower the edge blur artifacts, whereas halo artifacts do not appear to be connected with intraluminal values. The degree of halo artifacts does not increase with higher attenuation values.

The third parameter that may explain the high limits of agreement in the Bland-Altman analysis is the axial spatial resolution that, in our study, was 0.39 mm. So the 95% limits of agreement in this analysis are near-equivalent to 2 pixels. Probably in the future, automated software will offer analysis of the CAWT, as ours does currently for the IMT.^25,26

In our opinion, at this moment, it is unethical and not justified to perform MDCTA only for the evaluation of the CAWT. MDCTA of the carotid arteries should be performed when there are indications (sonograms that demonstrated a suspect important stenosis, symptomatic patients, suspected presence of carotid artery vulnerable plaque). However, when MDCTA is performed, we think that the radiologists should also analyze the CAWT. It is a parameter that is easy to analyze and may be important for assessing the cardiovascular risk.

We also analyzed the folded empiric cumulative distribution plot (mountain plot) for interobserver analysis (Fig 5). A mountain plot is created by computing a percentile for each ranked difference between a new method and a reference method. To get a folded plot, one must perform the following transformation for all percentiles above 50: percentile = 100 − percentile. These percentiles are then plotted against the differences between the 2 methods. The mountain plot is considered a useful complementary plot to the Bland-Altman plot. In particular, the mountain plot offers the following advantages: 1) It is easier to find the central 95% of the data, even when the data are not normally distributed, and 2) different distributions can be compared more easily. We used this form of illustration to emphasize the median and dispersion of the distribution of the data. We observed that the percentiles are distributed according to a normal distribution.

This study has some limitations. First, it is a retrospective analysis. Further to this point, we used the same hardware, techniques, operators, and data standardization; so the variability in the retrospective analysis should be reduced. Second, we did not compare the CAWT with a criterion standard such as a histologic specimen; however, the focus of this study was the reproducibility analysis and not the sensitivity.

Conclusions

The results of our study demonstrate that the CAWT measured by using MDCTA shows a good reproducibility between observers by considering inter- and intraobserver agreement. Therefore, the quantification of CAWT by using CT can be considered a reproducible value.

ABBREVIATIONS:

CAWT

carotid artery wall thickness

MDCTA

multidetector row CT angiography

IMT

intima-media thickness