SUMMARY
In this study, we compared the measurement of carotid stenosis by computed tomography angiography (CTA) based on the narrowest diameter versus cross sectional area (CSA) with the measurement by color Doppler ultrasonography (CDUS) as a reference standard, and analyzed how the application of different statistical methods affected the result. On 113 carotid arteries with ≥50% stenosis, we quantified the level of correlation among the three measurements, sensitivity, specificity, and differences in the estimated stenosis level. Correlation between both CTA measurements was good with Pearson’s ρ between 0.87 and 0.91 (p<0.001). Correlation between CDUS and CTA measurements was only modest with Pearson’s ρ between 0.2 (p=0.075) and 0.4 (p=0,007) for CDUS CTA (CSA), and between 0.23 (p=0.062) and 0.39 (p=0.008) for CDUS CTA (diameter). Differences in stenosis between CTA (CSA) and CDUS were centered around 0%, and between CTA (diameter) and CDUS around 20%. Sensitivity and specificity for CTA (CSA) method were 81% and 77%, and for CTA (diameter) 23% and 100%, respectively. A good correlation between CSA and diameter measurement just means that these are two related features of stenosis, it does not mean good agreement. CTA (CSA) method better detected surgical stenoses, whereas CTA (diameter) systematically underestimated stenosis level. The study of differences between the measurements indicated agreement better than the calculation of correlation coefficients.
Key words: Carotid stenosis, CT angiography, Color Doppler ultrasonography, Statistical data analysis
Introduction
According to large clinical studies, North American Symptomatic Carotid Endarterectomy Trial (NASCET) and European Symptomatic Carotid Surgery Trial (ESCT), clinical decision regarding the indication for carotid endarterectomy is primarily based on the degree of carotid stenosis (1, 2). Therefore, accurate grading of stenosis is necessary for correct selection of patients for medical or surgical treatment. The gold standard in the evaluation of atherosclerotic carotid disease has been intra-arterial digital subtraction angiography (DSA). However, because of the risk to the patient, in clinical practice many physicians tend to use less invasive methods such as color Doppler ultrasonography (CDUS), computed tomography angiography (CTA), and magnetic resonance angiography (MRA) (3-7). CDUS is more hemodynamically oriented and uses velocity criteria for stenosis grading, unlike other methods that are more morphologically oriented (8-10). CTA and MRA have the ability to generate multiplanar views of the vessel and allow for visualization of the vessel wall and surrounding tissue, when they are compared to DSA, which provides luminograms (11-15). NASCET and ESCT methodology were based on intra-arterial angiography and used the narrowest diameter for stenosis grading. CTA enables analysis of stenosis in axial projection and allows for the measurement of the cross-sectional area (CSA) as a basis for calculating stenosis instead of the traditionally used narrowest diameter (16). All new CT scanners have vessel analysis software for automatic stenosis calculation, which is less operator dependent and better reproducible than the manual method (17, 18). There are many studies that compare diagnostic methods in stenosis evaluation, with different statistical approaches used to determine agreement between them. In this study, we compared CTA diameter based and CTA area based measurements with CDUS as a reference standard and analyzed how the application of different statistical methods affected the result.
Patients and Methods
We retrospectively collected examinations of extracranial carotid arteries performed in our hospital using both CDUS and CTA methods from February 2014 until July 2015. All patients underwent CTA examinations as part of the institutional routine diagnostic workup and gave written informed consent. The study was approved by the institutional medical Ethics Committee as consistent with the Declaration of Helsinki. The patients who underwent examinations had neurological symptomatology or were asymptomatic but at risk of atherosclerosis. Those with previous endarterectomy or stenting were not included in the analysis. During these 17 consecutive months, a total of 91 patients were examined, 47 males and 44 females, mean age 71 (range 47-89) years. The mean interval between the CDUS and CTA examinations was 56 days (not longer than 6 months). Out of 182 imaged carotid arteries, 113 had stenosis ≥50% as diagnosed by ultrasonography, and only these were taken for analysis in this study. The stenoses were classified into moderate (50%-69%) and severe (70%-99%), and were evaluated separately. There were 44 vessels in the group of moderate and 69 vessels in the group of severe stenosis. Ultrasonographic reading served as a reference standard for CTA.
Color Doppler ultrasonography examinations were performed using ProSound Alpha 7 ultrasound system (Aloka, Mitaka, Tokyo, Japan) with a 7.5 MHz linear transducer, by five radiologists with a minimum of four-year experience in ultrasonographic diagnostics. Grading of stenosis was done according to the good-quality criteria recommended by the Society of Radiologists in Ultrasound Consensus Conference in 2003 (9). Each carotid artery was examined morphologically using B-mode and color imaging, and hemodynamically using pulse-wave Doppler measurements. To define stenosis severity, the peak systolic velocity (PSV) was primarily used (PSV of 125 cm/s to indicate 50% stenosis and PSV of 230 cm/s to indicate 70% stenosis). End-diastolic velocity and the internal carotid artery/common carotid artery ratio were used as additional parameters. Grading of stenosis was established by combining both the morphological and hemodynamic approach, and the degree of stenosis was expressed as percentage.
Computed tomography angiography examinations were performed using a Mx8000 Quad 4-detector-row CT system (Philips Medical System, Cleveland, Ohio, USA) and the images were interpreted by four experienced radiologists. We used a collimation of 1 mm and reconstructed the overlapping sections of 1.2 mm slice thickness at a reconstruction interval of 0.6 mm. The scan started at the aortic arch and ended at the circle of Willis level. A total of 100 mL of the contrast medium (Xenetix 350, Ile-de-France, Paris, France) was injected at a flow rate of 4 mL/s. The scan was initiated by automated bolus triggering (Bolus Pro Ultra, Philips, Cleveland, Ohio, USA) with the region of interest placed in the aortic arch and Hounsfield unit threshold set at 100. The resulting images were sent to a dedicated CT workstation for further evaluation. For quantification of stenosis we used NASCET methodology, comparing the most narrowed part of the vessel to the first normal distal arterial segment (1). A radiologist defined the narrowest segment and the referential segment on the curved planar reformation images and measurements were performed on the cross-sectional slices perpendicular to the long axis of the vessel. Calculation of stenosis was performed using the Advanced Vessel Analysis (AVA) automated program with commercially available post processing software (Extended Brilliance Workspace, version R.1.0.91, Philips Medical System, Cleveland, Ohio, USA), which evaluated the degree of stenosis based on the narrowest diameter and CSA expressing it as a percentage (Fig. 1).
Fig. 1.
Quantification of an eccentric right carotid artery stenosis on CTA using AVA is shown. The software automatically outlined the residual and normal vessel lumen and calculated the degree of stenosis in terms of diameter reduction and area reduction. The area of residual lumen was 8.4 mm2, and the area of normal lumen was 25.9 mm2, resulting in 68% stenosis. The residual lumen diameter was 3.3 mm, and normal lumen diameter was 5.7 mm, resulting in 43% stenosis. CTA = computed tomography angiography; AVA = Advanced Vessel Analysis
Statistics
We visually presented empirical distribution of data from CDUS, CTA (CSA) and CTA (diameter) measurements. Further we presented relationships among the three measurements as a scatter plot for each pair (CDUS CTA (CSA), CDUS CTA (diameter) and CTA (CSA) CTA (diameter), together with the estimated regression line and r2 coefficient of determination from the regression. To quantify the level of correlation among the three measurements, we calculated Pearson correlation coefficient (linear correlation) and Kendall tau rank correlation (rank based, non-parametric), with the p-value from the test of no correlation between the methods. We further calculated differences in estimated stenosis level from all 3 possible pairs of methods for each vessel. Using T or Wilcoxon test, we tested whether any distribution of differences had a central tendency at 0. To test the predictive power for detecting severe stenoses (70% or greater) requiring surgical intervention, we calculated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for both CTA methods. All statistical analyses were performed using R: A Language and Environment for Statistical Computing.
Results
Figure 2 shows empirical distributions of stenosis measured by the three methods (CDUS, CTA (CSA) and CTA (diameter) for all patients. We can see that CDUS and CTA (CSA) measurements have more similar distribution when compared to the CDUS CTA (diameter) pair. There is a tendency of left shift of stenosis level by CTA (diameter) relative to CDUS measurements with smaller central tendency and larger spread. The mean stenosis measured by CDUS method was 67.9%, with median 70%. In the measurements by CTA (CSA) method, the mean stenosis was very similar, 68.4%, and median was also 70%. The CTA (diameter) measurements yielded the mean stenosis of 47.2% and median of 45%, which is seen as a left shift in Figure 2; in clinical terms, it means underestimation of the stenosis level in comparison with ultrasonography.
Fig. 2.
Empirical distributions of stenosis measurements in the whole sample. CDUS = color Doppler ultrasonography; CTA = computed tomography angiography; CSA = cross sectional area
Figures 3, 4 and 5 show correlation scatter plots of the pairs of stenosis measurements.
Fig. 3.
Scatter plot of stenosis measurements by CDUS and CTA (CSA); red points denote patients with stenosis <70%, and blue triangles denote patients with stenosis >70%. CDUS = color Doppler ultrasonography; CTA = computed tomography angiography; CSA = cross sectional area
Fig. 4.
Scatter plot of stenosis measurements by CDUS and CTA (diameter); red points denote patients with stenosis <70%, and blue triangles denote patients with stenosis >70%. CDUS = color Doppler ultrasonography; CTA = computed tomography angiography
Fig. 5.
Scatter plot of stenosis measurements by CTA (CSA) and CTA (diameter); red points denote patients with stenosis <70%, and blue triangles denote patients with stenosis >70%. CTA = computed tomography angiography; CSA = cross sectional area
We fitted the regression line in each group of stenosis (moderate and severe) separately. The level of determination r2 shown in the figures is known to be numerically equivalent to the square of Pearson correlation coefficient. We can see that the coefficients of determination between stenosis measurements, where one of them is coming from CDUS, are very low (0.159 for moderate group and 0.0466 for severe group between CDUS CTA (CSA), and 0.154 for moderate group and 0.051 for severe group between CDUS CTA (diameter)). Opposite to that, the coefficients of determination between two CTA methods are much higher (0.836 for moderate and 0.752 for severe group of stenosis).
Tables 1 and 2 show the values of estimated Pearson’s ρ and Kendall’s tau correlation coefficients. Linear and monotone rank based correlations were higher between both CTA measurements. Pearson ρ was 0.87 and 0.91 (p<0.001 both) for severe stenosis and moderate stenosis, respectively. Kendal’s tau in these two groups was very similar for this pair of measurements (0.86 and 0.87 (p<0.001 both)). A less expressed and very similar degree of correlation was found between CDUS and both CTA measurements. In the nonsurgical group of stenosis, Pearson’s ρ was 0.4 and 0.39 with p-values of 0.007 and 0.008 for CDUS CTA (CSA), and CDUS CTA (diameter), respectively. Kendall’s tau for the same pairs in the same group was 0.36 and 0.36 with p-values of 0.003 and 0.008. In the surgical group of stenosis, we could not reject the hypothesis of Pearson correlation coefficient significantly different from 0 to the level of 0.05. Numerical estimates here were around 0.2 for both CTA measurements relative to CDUS. Kendall’s tau was very similar between the same pairs for the same group but in this case the monotone rank correlation was significant at the level of 0.05.
Table 1. Pearson correlation coefficients for all three pairs of measurements.
| CDUS | CTA (CSA) | |||||||
|---|---|---|---|---|---|---|---|---|
| <70% | 70%-99% | <70% | 70%-99% | |||||
| CTA (CSA) | ||||||||
| <70% | 0.4 | (p=0.007) | ||||||
| 70%-99% | 0.2 | (p=0.075) | ||||||
| CTA (diameter) | ||||||||
| <70% | 0.39 | (p=0.008) | 0.91 | (p<0.001) | ||||
| 70%-99% | 0.23 | (p=0.062) | 0.87 | (p<0.001) | ||||
CDUS = color Doppler ultrasonography; CTA = computed tomography angiography; CSA = cross sectional area; bold = statistically significant
Table 2. Kendall tau rank correlation coefficients for all three pairs of measurements.
| CDUS | CTA (CSA) | |||||||
|---|---|---|---|---|---|---|---|---|
| Less than 70% | 70% - 99% | Less than 70% | 70% - 99% | |||||
| CTA (CSA) | ||||||||
| <70% | 0.36 | (p=0.003) | ||||||
| 70%-99% | 0.23 | (p=0.02) | ||||||
| CTA (diameter) | ||||||||
| <70% | 0.36 | (p=0.008) | 0.85 | (p<0.001) | ||||
| 70%-99% | 0.2 | (p=0.03) | 0.86 | (p<0.001) | ||||
CDUS = color Doppler ultrasonography; CTA = computed tomography angiography; CSA = cross sectional area; bold = statistically significant
Table 3 presents means and medians of differences in stenosis from all 3 possible pairs of methods with the corresponding p-values from the test of the hypothesis that the central tendency of their distribution is 0 (19). The first row in Table 3 shows means and medians together with the tests of differences for all vessels in the sample. The central tendency of the distribution of differences between CDUS and CTA (CSA) measurements is around 0, with the mean of -0.4% and median of 0%. Opposite to that, both pairs which involve CTA (diameter) stenosis measurements are centered towards 20%, with significant p-values (p<0.001).
Table 3. Location and spread parameters of distribution of differences in stenosis measurements (p-values from Student’s T-test and Wilcoxon rank sum test (in parentheses) (the table was published in 2018 in Insights into Imaging (19)).
| All stenoses | Stenosis 50%-69% | Stenosis 70%-99% | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Mean (SD) | Median (IQR) | p-value | Mean (SD) | Median (IQR) | p-value | Mean (SD) | Median (IQR) | p-value | |
| CDUS-CTA (CSA) | -0.4 (11.8) | 0 (-5, 5) | 0.68 (0.33) | -4.5 (10.1) | -2.5 (-12.5, 0) |
0.005 (0.004) | 2.1 (12.0) | 0 (-5, 10) | 0.14 (0.22) |
| CDUS-CTA (diameter) | 20.7 (14.2) | 22 (10, 28) |
<0.001 (<0.001) | 17.3 (9.9) | 20 (10, 24) | <0.001 (<0.001) | 22.9 (16.0) | 25 (20, 32) |
<0.001 (<0.001) |
| CTA (CSA)-CTA (diameter) | 21.2 (7.1) |
24 (20, 25) |
<0.001 (<0.001) | 21.7 (4.5) | 23 (20, 25) | <0.001 (<0.001) | 20.8 (8.4) | 24 (18, 25) |
<0.001 (<0.001) |
CDUS = color Doppler ultrasonography; CTA = computed tomography angiography; CSA = cross sectional area; SD = standard deviation; IQR = interquartile range
The second and third rows in Table 3 present the same analysis within the moderate and severe stenosis group separately with similar results. Each of the two pairs involving CTA (diameter) measurements had the central tendency of the distribution of differences around 20%, with the bias between 17.3% and 22.9% when measured by mean and between 20% and 25% when measured by median. CTA (CSA) method slightly but significantly overestimated true stenosis in moderate stenosis group, with the mean difference of -4.5% and p-value <0.005. In severe stenosis group, the mean difference between CDUS and CTA (CSA) was around 2% with a nonsignificant p-value.
Two standard deviations of differences (SDD) between CDUS and CTA (diameter) measurements in the group of 70%-99% stenosis was 32%, which means large dispersion of data from the mean. The interval of two standard deviations of differences for the CDUS CTA (CSA) pair was 24%, meaning better central tendency of these differences.
Comparison of CTA (CSA) method with CDUS in detecting severe stenosis yielded a sensitivity of 81%, specificity of 77%, PPV of 84%, and NPV of 72%. Sensitivity of CTA (diameter) method was low, only 23%, specificity was perfect (100%), PPV was also 100%, and NPV was only 45%. Low sensitivity and perfect specificity of the CTA (diameter) method were to be expected because this method systematically underestimated the true level of stenosis, so all patients that were really in nonsurgical group were classified as such by this method. The predictive power of the CTA (CSA) method was therefore significantly higher than the one of CTA (diameter) for correctly classifying vessels for surgical intervention.
Discussion
In many centers, evaluation of carotid stenosis is based on two noninvasive methods before proceeding to endarterectomy, and CDUS and CTA are most commonly used. The optimal statistical approach in assessing agreement between two methods of clinical measurement is not always easy to choose. When we calculated the correlation coefficients to evaluate the relationship between two CTA methods and CDUS as a surrogate for true stenosis, we found only modest correlation, very similar for the CDUS CTA (CSA) and CDUS CTA (diameter) pairs. We expected the correlation coefficients to be higher, and were completely surprised by these findings. When we look at Figure 2, which shows the distribution of stenosis, we can see that stenoses measured by CDUS methods are left censored at the value of 50% because only those patients with stenosis ≥50% underwent CTA examination and were included in this study. We consider that the presence of this left censoring for one of the variables (in this case for data from CDUS measurements) can explain attenuation of the correlation and determination coefficients (20). CTA derived measurements showed a much higher level of correlation with one another than any of the other two pairs which involved CDUS, ranging between 0.85 and 0.91. However, a good correlation between CTA (CSA) and CTA (diameter) measurements does not automatically imply that there is good agreement between the two methods and that these two methods are equally accurate in stenosis quantification. The correlation coefficient is a statistical measure that calculates the strength of the relationship between the relative movements of the two variables (21, 22). It is quite logical that any reduction of the diameter is accompanied by reduction of the area, and it is therefore understood that there is a positive correlation between these two methods of measurements. However, the correlation coefficient is not able to tell the absolute values of differences between the two paired measurements.
To define the measurement error of the two CTA methods in comparison with CDUS, we calculated differences in the estimated level of stenosis between the methods using Bland-Altman statistics. Any method of stenosis measurement always implies some degree of error. However, if the variability of differences was only linked to imprecision of each of the two methods, the mean of these differences should be zero (22). In our study, CTA (CSA) method in the whole sample and in the group of surgical stenoses showed no significant mean difference in relation to CDUS measurements, meaning that there was no systematic underestimation or overestimation of stenosis. In the group of moderate stenosis, with small but significant mean difference in negative direction, this method only slightly overestimated the true stenosis level. Opposite to that, CTA (diameter) method showed significant mean difference in the whole sample and in both groups of stenosis, ranging from 17.3% to 22.9%, which means that this method systematically underestimated stenosis level throughout the range of stenosis. Furthermore, this method had large dispersion of differences as measured by standard deviation, which together with systematic underestimation may have practical implications on the decision for surgery, if we are only based on CTA (diameter) measurements. The sensitivity and specificity of CTA (CSA) method for detecting severe stenosis were better balanced than those of CTA (diameter), so it also implies greater acceptability of the area based measurement in clinical use.
Our starting hypothesis was that the measurement of stenosis based on the CSA should be more accurate because the hemodynamic effect of luminal stenosis is due to the degree of area reduction. The area of residual lumen is often asymmetric and has an irregular shape, so it is impossible to precisely estimate area reduction by measuring the diameter. The results of the study confirmed our starting presumption.
The use of CSA as a base for stenosis calculation by CTA method is still being studied, since the large clinical trials (NASCET and ESCT) are based on traditionally used diameter measurement (on conventional angiography only diameter can be measured). The published articles show very different results when comparing CTA (CSA) and CTA (diameter) measurements in stenosis quantification. The researchers most often used correlation coefficients to determine the validity of two CTA methods. Some of them did not find significant differences between diameter and area measurements, and concluded that diameter is an adequate approximation for area (16, 23, 24). Others, testing diameter and area measurements to DSA, found better results when area based measurement was used (25, 26). Using Bland-Altman statistics, Muller et al. found that the area method was superior to the diameter in detecting severe stenosis, whereas diameter based measurement had a tendency to underestimate stenosis level (27).The results of our study are closer to the results of the latter group of authors.
One of the limitations of this study was a relatively small sample size, while a greater number of patients involved would give results of greater significance. Second, the study included an inhomogeneous group of symptomatic and asymptomatic patients. Third, we used CDUS as a reference method, not DSA which is still regarded a gold standard in stenosis measurement. Patients in our study did not have DSA because we do not use this method routinely in the preoperative evaluation of carotid stenosis. We believe that CDUS can be used as a substitute for the true gold standard, since this method has been in use for many years and through that period it was strictly tested according to DSA (8, 28).
Conclusion
When assessing comparability of various methods in grading carotid artery stenosis, the study of differences between one measurement and the other using Bland-Altman statistics is a better indicator of agreement than the calculation of correlation coefficients. CTA method is a good choice when additional imaging is required after ultrasonography in preoperative patient evaluation. According to the results of our study, CTA based on the measurement of the vessel lumen CSA is more reliable in deciding on carotid endarterectomy than the CTA based on the narrowest diameter.
References
- 1.North American Symptomatic Carotid Endarterectomy Trial Collaborators . Beneficial effect of carotid endarterectomy in symptomatic patients with high grade carotid stenosis. N Engl J Med. 1991;325:445–53. 10.1056/NEJM199108153250701 [DOI] [PubMed] [Google Scholar]
- 2.European Carotid Surgery Trialists Collaborative Group . Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet. 1998;351:1379–87. 10.1016/S0140-6736(97)09292-1 [DOI] [PubMed] [Google Scholar]
- 3.Kaufmann TJ, Huston J, Mandrekar JN, Schleck CD, Thielen KR, Kallmes DF. Complications of diagnostic cerebral angiography: evaluation of 19,826 consecutive patients. Radiology. 2007;243(3):812–9. 10.1148/radiol.2433060536 [DOI] [PubMed] [Google Scholar]
- 4.Johnston DC, Goldstein LB. Clinical carotid endarterecto-my decision making: noninvasive vascular imaging versus angiography. Neurology. 2001;56(8):1009–15. 10.1212/WNL.56.8.1009 [DOI] [PubMed] [Google Scholar]
- 5.Long A, Lepoutre A, Corbillon E, Branchereau A. Critical review of non- or minimally invasive methods (duplex ultrasonography, MR- and CT-angiography) for evaluating stenosis of the proximal internal carotid artery. Eur J Vasc Endovasc Surg. 2002;24(1):43–52. 10.1053/ejvs.2002.1666 [DOI] [PubMed] [Google Scholar]
- 6.Romero JM, Ackerman RH, Dault NA, Lev MH. Noninvasive evaluation of carotid artery stenosis: indications, strategies, and accuracy. Neuroimaging Clin N Am. 2005;15(2):351–65. 10.1016/j.nic.2005.06.005 [DOI] [PubMed] [Google Scholar]
- 7.Ooi YC. Management of extracranial carotid artery disease. Cardiol Clin. 2015;33(1):1–35. 10.1016/j.ccl.2014.09.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Filis KA, Arko FR, Johnson BL, Pipinos II, Harris EJ, Olcott C, 4th, et al. Duplex ultrasound criteria for defining the severity of carotid stenosis. Ann Vasc Surg. 2002;16(4):413–21. 10.1007/s10016-001-0175-8 [DOI] [PubMed] [Google Scholar]
- 9.Grant EG, Benson CB, Moneta GL, Alexandrov AV, Baker JD, Bluth EI, et al. Carotid artery stenosis: gray-scale and Doppler US diagnosis – Society of Radiologists in Ultrasound Consensus Conference. Radiology. 2003;229(2):340–6. 10.1148/radiol.2292030516 [DOI] [PubMed] [Google Scholar]
- 10.von Reutern GM, Goertler MW, Bornstein NM, Del Sette M, Evans DH, Hetzel A, et al. Grading carotid stenosis using ultrasonic methods. Stroke. 2012;43(3):916–21. 10.1161/STROKEAHA.111.636084 [DOI] [PubMed] [Google Scholar]
- 11.Cinat M, Lane CT, Pham H, Lee A, Wilson SE, Gordon I. Helical CT angiography in the preoperative evaluation of carotid artery stenosis. J Vasc Surg. 1998;28(2):290–300. 10.1016/S0741-5214(98)70165-X [DOI] [PubMed] [Google Scholar]
- 12.Anderson GB, Ashforth R, Steinke DE, Ferdinandy R, Findlay JM. CT angiography for the detection and characterization of carotid artery bifurcation disease. Stroke. 2000;31(9):2168–74. 10.1161/01.STR.31.9.2168 [DOI] [PubMed] [Google Scholar]
- 13.Saba L, Sanfilippo R, Pirisi R, Pascalis L, Montisci R, Mallarini G. Multidetector-row CT angiography in the study of atherosclerotic carotid arteries. Neuroradiology. 2007;49(8):623–37. 10.1007/s00234-007-0244-y [DOI] [PubMed] [Google Scholar]
- 14.Nederkoorn PJ, Elgersma OE, van der Graaf Y, Eikelboom BC, Kappelle LJ, Mali WP. Carotid artery stenosis: accuracy of contrast-enhanced MR angiography for diagnosis. Radiology. 2003;228(3):677–82. 10.1148/radiol.2283020824 [DOI] [PubMed] [Google Scholar]
- 15.Cosottini M, Pingitore A, Puglioli M, Michelassi MC, Lupi G, Abbruzzese A, et al. Contrast-enhanced three-dimensional magnetic resonance angiography of atherosclerotic internal carotid stenosis as the noninvasive imaging modality in revascularization decision making. Stroke. 2003;34(3):660–4. 10.1161/01.STR.0000057462.02141.6F [DOI] [PubMed] [Google Scholar]
- 16.Carnicelli AP, Stone JJ, Doyle A, Chowdhry AK, Mix D, Ellis J, et al. Cross-sectional area for the calculation of carotid artery stenosis on computed tomographic angiography. J Vasc Surg. 2013;58(3):659–65. 10.1016/j.jvs.2013.02.031 [DOI] [PubMed] [Google Scholar]
- 17.White JH, Bartlett ES, Bharatha A, Aviv RI, Fox AJ, Thompson AL, et al. Reproducibility of semi-automated measurement of carotid stenosis on CTA. Can J Neurol Sci. 2010;37(4):498–503. 10.1017/S0317167100010532 [DOI] [PubMed] [Google Scholar]
- 18.Birmpili P, Porter L, Shaikh U, Torella F. Comparison of measurement and grading of carotid stenosis with computed tomography angiography and Doppler ultrasound. Ann Vasc Surg. 2018;51:217–24. 10.1016/j.avsg.2018.01.102 [DOI] [PubMed] [Google Scholar]
- 19.Samarzija K, Milosevic P, Jurjevic Z, Erdeljac E. Grading of carotid artery stenosis with computed tomography angiography: whether to use the narrowest diameter or the cross-sectional area. Insights Imaging. 2018;9(4):527–34. 10.1007/s13244-018-0622-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lyles RH, Fan D, Chuachoowong R. Correlation coefficient estimation involving a left censored laboratory assay variable. Stat Med. 2001;20(19):2921–33. 10.1002/sim.901 [DOI] [PubMed] [Google Scholar]
- 21.Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10. 10.1016/S0140-6736(86)90837-8 [DOI] [PubMed] [Google Scholar]
- 22.Giavarina D. Understanding Bland Altman analysis. Biochem Med (Zagreb). 2015;25(2):141–51. 10.11613/BM.2015.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Bartlett ES, Symons SP, Fox AJ. Correlation of carotid stenosis diameter and cross-sectional areas with CT angiography. AJNR Am J Neuroradiol. 2006;27(3):638–42. [PMC free article] [PubMed] [Google Scholar]
- 24.van Prehn J, Muhs BE, Pramanik B, Ollenschleger M, Rockman CB, Cayne NS, et al. Multidimensional characterization of carotid artery stenosis using CT imaging: a comparison with ultrasound grading and peak flow measurement. Eur J Vasc Endovasc Surg. 2008;36(3):267–72. 10.1016/j.ejvs.2008.04.016 [DOI] [PubMed] [Google Scholar]
- 25.Zhang Z, Berg M, Ikonen A, Kononen M, Kalviainen R, Manninen H, et al. Carotid stenosis degree in CT angiography: assessment based on luminal area versus luminal diameter measurements. Eur Radiol. 2005;15(11):2359–65. 10.1007/s00330-005-2801-2 [DOI] [PubMed] [Google Scholar]
- 26.Bucek RA, Puchner S, Haumer M, Reiter M, Minar E, Lammer J. CTA quantification of internal carotid artery stenosis: application of luminal area vs. luminal diameter measurements and assessment of inter-observer variability. J Neuroimaging. 2007;17(3):219–26. 10.1111/j.1552-6569.2007.00124.x [DOI] [PubMed] [Google Scholar]
- 27.Müller M, Agten CA, Osterreich M, Hoffmann M. Assessing internal carotid artery stenosis with a semiautomated computed tomography angiography tool and duplex ultrasound. J Vasc Surg. 2015;61(6):1449–56. 10.1016/j.jvs.2015.01.028 [DOI] [PubMed] [Google Scholar]
- 28.Jahromi AS, Cina CS, Liu Y, Clase CM. Sensitivity and specificity of color duplex ultrasound measurement in the estimation of internal carotid artery stenosis: a systematic review and meta-analysis. J Vasc Surg. 2005;41(6):962–72. 10.1016/j.jvs.2005.02.044 [DOI] [PubMed] [Google Scholar]