Duplex ultrasound for diagnosing symptomatic carotid stenosis in the extracranial segments

Abstract

Background

Carotid artery stenosis is an important cause of stroke and transient ischemic attack. Correctly and rapidly identifying patients with symptomatic carotid artery stenosis is essential for adequate treatment with early cerebral revascularization. Doubts about the diagnostic value regarding the accuracy of duplex ultrasound (DUS) and the possibility of using DUS as the single diagnostic test before carotid revascularization are still debated.

Objectives

To estimate the accuracy of DUS in individuals with symptomatic carotid stenosis verified by either digital subtraction angiography (DSA), computed tomography angiography (CTA), or magnetic resonance angiography (MRA).

Search methods

We searched CRDTAS, CENTRAL, MEDLINE (Ovid), Embase (Ovid), ISI Web of Science, HTA, DARE, and LILACS up to 15 February 2021. We handsearched the reference lists of all included studies and other relevant publications and contacted experts in the field to identify additional studies or unpublished data.

Selection criteria

We included studies assessing DUS accuracy against an acceptable reference standard (DSA, MRA, or CTA) in symptomatic patients. We considered the classification of carotid stenosis with DUS defined with validated duplex velocity criteria, and the NASCET criteria for carotid stenosis measures on DSA, MRA, and CTA. We excluded studies that included < 70% of symptomatic patients; the time between the index test and the reference standard was longer than four weeks or not described, or that presented no objective criteria to estimate carotid stenosis.

Data collection and analysis

The review authors independently screened articles, extracted data, and assessed the risk of bias and applicability concerns using the QUADAS‐2 domain list. We extracted data with an effort to complete a 2 × 2 table (true positives, true negatives, false positives, and false negatives) for each of the different categories of carotid stenosis and reference standards. We produced forest plots and summary receiver operating characteristic (ROC) plots to summarize the data. Where meta‐analysis was possible, we used a bivariate meta‐analysis model.

Main results

We identified 25,087 unique studies, of which 22 were deemed eligible for inclusion (4957 carotid arteries). The risk of bias varied considerably across the studies, and studies were generally of moderate to low quality. We narratively described the results without meta‐analysis in seven studies in which the criteria used to determine stenosis were too different from the duplex velocity criteria proposed in our protocol or studies that provided insufficient data to complete a 2 × 2 table for at least in one category of stenosis. Nine studies (2770 carotid arteries) presented DUS versus DSA results for 70% to 99% carotid artery stenosis, and two (685 carotid arteries) presented results from DUS versus CTA in this category. Seven studies presented results for occlusion with DSA as the reference standard and three with CTA as the reference standard. Five studies compared DUS versus DSA for 50% to 99% carotid artery stenosis. Only one study presented results from 50% to 69% carotid artery stenosis.

For DUS versus DSA, for < 50% carotid artery stenosis, the summary sensitivity was 0.63 (95% confidence interval [CI] 0.48 to 0.76) and the summary specificity was 0.99 (95% CI 0.96 to 0.99); for the 50% to 69% range, only one study was included and meta‐analysis not performed; for the 50% to 99% range, the summary sensitivity was 0.97 (95% CI 0.95 to 0.98) and the summary specificity was 0.70 (95% CI 0.67 to 0.73); for the 70% to 99% range, the summary sensitivity was 0.85 (95% CI 0.77 to 0.91) and the summary specificity was 0.98 (95% CI 0.74 to 0.90); for occlusion, the summary sensitivity was 0.91 (95% CI 0.81 to 0.97) and the summary specificity was 0.95 (95% CI 0.76 to 0.99).

For sensitivity analyses, excluding studies in which participants were selected based on the presence of occlusion on DUS had an impact on specificity: 0.98 (95% CI 0.97 to 0.99). For DUS versus CTA, we found two studies in the range of 70% to 99%; the sensitivity varied from 0.57 to 0.94 and the specificity varied from 0.87 to 0.98. For occlusion, the summary sensitivity was 0.95 (95% CI 0.80 to 0.99) and the summary specificity was 0.91 (95% CI 0.09 to 0.99). For DUS versus MRA, there was one study with results for 50% to 99% carotid artery stenosis, with a sensitivity of 0.88 (95% CI 0.70 to 0.98) and specificity of 0.60 (95% CI 0.15 to 0.95); in the 70% to 99% range, two studies were included, with sensitivity that varied from 0.54 to 0.99 and specificity that varied from 0.78 to 0.89. We could perform only a few of the proposed sensitivity analyses because of the small number of studies included.

Authors' conclusions

This review provides evidence that the diagnostic accuracy of DUS is high, especially at discriminating between the presence or absence of significant carotid artery stenosis (< 50% or 50% to 99%). This evidence, plus its less invasive nature, supports the early use of DUS for the detection of carotid artery stenosis. The accuracy for 70% to 99% carotid artery stenosis and occlusion is high. Clinicians should exercise caution when using DUS as the single preoperative diagnostic method, and the limitations should be considered. There was little evidence of the accuracy of DUS when compared with CTA or MRA. The results of this review should be interpreted with caution because they are based on studies of low methodological quality, mainly due to the patient selection method. Methodological problems in participant inclusion criteria from the studies discussed above apparently influenced an overestimated estimate of prevalence values. Most of the studies included failed to precisely describe inclusion criteria and previous testing. Future diagnostic accuracy studies should include direct comparisons of the various modalities of diagnostic tests (mainly DUS, CTA, and MRA) for carotid artery stenosis since DSA is no longer considered to be the best method for diagnosing carotid stenosis and less invasive tests are now used as reference standards in clinical practice. Also, for future studies, the participant inclusion criteria require careful attention.

Plain language summary

How accurate is duplex ultrasound (DUS) imaging for diagnosing carotid artery stenosis in symptomatic patients?

Carotid artery stenosis (CAS) is a narrowing of the lumen (the inside space) of the carotid artery (usually due to cholesterol deposits called plaque). CAS is responsible for 8% of all strokes due to a blocked blood vessel (ischemic strokes) and is associated with a high chance of recurrence. In such circumstances, the treatment is to re‐establish adequate blood flow (by surgery or other approaches to open the artery) to prevent further neurologic episodes. Duplex ultrasound (DUS) can help identify the appropriate patients who will benefit from a more invasive treatment and those who should be with drugs alone.

What is the aim of this review?

To determine how accurate DUS is for diagnosing different grades of CAS in individuals with neurologic symptoms.

What was studied in the review?

DUS is used in clinical practice as the first test to detect carotid artery stenosis, usually with the result confirmed by other more expensive and invasive tests, such as computed tomography angiography (CTA), magnetic resonance angiography (MRA), or digital subtraction angiography (DSA). The advantage of DUS is that it is less expensive and helps to reduce the time required to select patients for treatment. We included studies assessing the accuracy of DUS compared with DSA, MRA, or CTA in patients with recent stroke symptoms. We grouped the results from studies that used approximately the same method and threshold to assess accuracy in the following categories of carotid artery stenosis: < 50%, 50% to 99%, 50% to 69%, 70% to 99%, and occlusion (blockage of the vessel).

What are the main results of this review?

This review included 22 studies (4957 carotid arteries tested). The searches were performed up to 15 February 2021. The results indicate the following: If DUS were to be used in a standardized cohort of 1000 patients:

For DUS versus DSA

< 50% CAS (4 studies, 1495 carotid arteries): Estimated 299 patients would have a DUS result indicating the presence of non‐significant CAS, of whom eight (2.7%) would be incorrectly classified. Of the 701 people with a result indicating that < 50% carotid stenosis is not present, 169 (24.1%) would be incorrectly classified.

50% to 99% CAS (5 studies, 1536 carotid arteries): Estimated 642 patients would have a DUS result indicating the presence of 50% to 99% CAS; of these, 147 (22.8%) would be incorrectly classified. Of the 358 people with a result indicating that 50% to 99% carotid stenosis is not present, 15 (4.2%) would be incorrectly classified.

70% to 99% CAS (9 studies, 2770 carotid arteries): Estimated 390 patients would have a DUS result indicating the presence of 70% to 99% CAS; of these, eight (2%) would be incorrectly classified. Of the 610 people with a result indicating that 70% to 99% carotid stenosis is not present, 68 (11.1%) would be incorrectly classified.

Occlusion (7 studies, 1212 carotid arteries): Estimated 205 patients would have a DUS result indicating carotid artery occlusion; of these, 41 (20%) would be incorrectly classified. Of the 795 people with a result indicating that carotid occlusion is not present, 16 (2%) would be incorrectly classified.

For DUS versus CTA

Occlusion (3 studies, 833 carotid arteries): An estimated 606 patients would have a DUS result indicating carotid artery occlusion; of these, 36 (6%) would be incorrectly classified. 394 people with a result indicating that carotid occlusion is not present, 30 (8%) would be incorrectly classified.

For DUS versus MRA

Meta‐analysis was not performed.

How reliable are the results of the studies in this review?

There were some problems with how the studies were conducted that could impair the correct estimates of the diagnostic accuracy. Many of the studies were of poor or unclear quality.

Who do the results of this review apply to?

The results are relevant for patients with neurologic symptoms who are suspected of having carotid artery stenosis.

What are the implications of this review?

The diagnostic accuracy of DUS is high, especially at discriminating between the presence or absence of significant carotid artery stenosis. This evidence, plus its less invasive nature, supports the early use of DUS for the detection of carotid artery stenosis.

Summary of findings

Summary of findings 1. Summary of findings table: Duplex ultrasound for diagnosing symptomatic carotid stenosis in the extracranial segments.

Review question:	What is the diagnostic accuracy of duplex ultrasound for detecting symptomatic carotid stenosis?
Population	Symptomatic patients (sudden visual loss, hemispheric TIA, and ischemic stroke) with suspected carotid artery stenosis
Target condition	Carotid artery stenosis
Index test	Duplex ultrasound
Reference Standard	DSA in 19 studies (Anzidei 2012; Borisch 2003; Chua 2007; Colquhoun 1992; Cui 2018; D'Onofrio 2006; Bray 1995; Eliasziw 1995; Faught 1994; Golledge 1999; Hammond 2008; Hansen 1996; Heijenbrok‐Kal 2006; Huston 1993; Knudsen 2002; Link 1997; Lubezky 1998; Nederkoorn 2002; Wolfle 2002); MRA in three (Borisch 2003; D'Onofrio 2006; Das 2009); CTA in four (Barlinn 2016; Belsky 2000; Das 2009; Lubezky 1998)
Importance	Diagnostic accuracy of DUS to identify carotid artery stenosis in symptomatic patients can improve the path in defining the best treatment option
Included studies	We included 22 studies, with a total of 4957 carotid arteries, mean sample size of 126 carotid arteries, ranging from 24 to 1011; the mean age of participants was 66.3 years (range 53 to 72 years), and the median proportion of men was 70% of included participants. Eighteen prospective studies, two retrospective and, in two studies, it was unclear whether there was a prospective or retrospective design
Risk of bias and applicability concerns	Risk of bias varied considerably across the included studies; we considered nine studies as being at high risk of bias and one as having unclear concern in the patient selection domain, mostly due to failure to include all people with a negative screen or poorly reported patient selection methods; four studies were judged as having a high risk of bias in the index test domain, mostly because of no prespecified thresholds; two as being at high risk of bias and seven at unclear risk of bias in the reference standard domain, as the studies were not blinded or blinding was not described; and the risk of bias in the flow and timing domain was high in 14 studies because not all patients were included in the analysis and it was unclear in another two. Applicability concerns were generally low; six studies were judged as having high concern on the patient selection domain mostly because of previous testing.
Limitations	Seventeen studies were judged as having high risk of bias in at least one domain. The use of velocity criteria with prespecified thresholds and time we accepted between the index test and reference standard (four weeks) led to a lot of studies' exclusions. There were also a lack of data on some carotid stenosis categories and reference standards.
Reference Standard	Studies	Carotid arteries	Summary sensitivity (95% confidence interval)	Summary specificity (95% confidence interval)	Consequences in a cohort of 1000
					Prevalence of the range of stenosis (median) *	Implications *	Quality and Comments
DSA
< 50%	4	1495	0.63 (0.48 to 0.76)	0.99 (0.96 to 0.99)	0.46	460 out of 1000 patients will have < 50% carotid artery stenosis. Of these, 291 (63%) would be correctly diagnosed and receive appropriate clinical treatment and 169 (27%) would receive unnecessary further investigation with another imaging method. Other 532 patients would receive appropriate further investigation, and eight would have no other tests performed and miss a chance for the right diagnosis and the possibility of carotid revascularization .	Limited number of studies Risk of bias: High or unclear in most domains
50‐99%	5	1536	0.97 (0.95 to 0.98)	0.70 (0.67 to 0.73)	0.51	510 out of 1000 patients will have 50‐99% carotid artery stenosis. Of these, 495 (97%) would receive appropriate further investigation with another imaging method, and 15 (3%) would not have any other tests performed and would miss a chance to receive the right diagnosis and the possibility of carotid revascularization. Overall, 147 would receive unnecessary further investigation with another imaging method, and 343 would receive no further investigation and appropriate clinical treatment	Limited number of studies Risk of bias: High or unclear in most domains
50‐69%	1	313	0.28 (0.17 to 0.41)	0.90 (0.85 to 0.93)	0.19	Meta‐analyses not conducted
70‐99%	9	2770	0.85 (0.77 to 0.91)	0.99 (0.96 to 0.99)	0.45	451 out of 1000 patients will have 70‐99% carotid artery stenosis. Of these, 383 (85%) would receive appropriate carotid artery revascularization and 68 (15%) would miss or delay the chance to carotid revascularization. Another 8 would receive inappropriate carotid artery revascularization and 542 would receive appropriate clinical treatment.	Limited number of studies Risk of bias: Low risk in all domains in 2 studies
Occluded	7	1212	0.91 (0.81 to 0.97)	0.95 (0.99 to 0.76)	0.18	180 out of 1000 patients will have carotid artery occlusion. Of these, 164 (91%) would receive appropriate clinical treatment. Another 41 would be false‐positive diagnosed with carotid occlusion and not have other tests performed, and miss a chance of the correct diagnosis and carotid revascularization. Other consequences would depend on the range of stenosis.	Limited number of studies Risk of bias: Low risk in all domains in 1 study Two studies only included patients with occlusion on DUS. Sensitivity analyses excluding them had impact on the results of specificity: 0.98 (95% CI: 0.97 to 0.99).
CTA
70‐99%	2	685	Range: 0.57 to 0.94	0.87 to 0.98	0.18	Meta‐analyses not conducted
Occluded	3	833	0.95 (0.80 to 0.99)	0.91 (0.99 to 0.09)	0.60	600 out of 1000 patients will have carotid artery occlusion. Of these, 570 (95%) would receive appropriate clinical treatment. Another 41 would be false‐positive diagnosed with carotid occlusion and not have other tests performed, and miss a chance of the correct diagnosis and carotid revascularization. Other consequences would depend on the range of stenosis	Limited number of studies Risk of bias: High or unclear in most domains 1 study only included patients with occlusion on DUS
MRA
50‐99%	1	31	0.88 (0.70 to 0.98)	0.60 (0.15 to 0.95)	0.84	Meta‐analyses not conducted
70‐99%	2	102	Range: 0.54 to 0.99	Range: 0.89 to 0.78	0.61	Meta‐analyses not conducted
CI: confidence interval; CTA: computed tomography angiography; DSA: digital subtraction angiography; DUS: duplex ultrasound; MRA: magnetic resonance angiography; TIA: transient Ischemic attack
* We calculated prevalence from the included studies by the reference standard. The prevalence values used to illustrate the review findings as absolute frequencies are the median from the included studies.
CAUTION: The results on this table should not be interpreted in isolation from the results of the individual included studies contributing to each summary test accuracy measure. These are reported in the main body of the text of the review.

Background

Stroke is the third leading cause of death worldwide (Brott 2011; Flumignan 2017; Virani 2021), and probably the most important cause of long‐term disability (CDC 2001; Eliasziw 1994; Strong 2007). Approximately 15 million people have a stroke annually, of which 5 million die as a result of the event and another 5 million remain disabled (Mackay 2004). The estimated direct and indirect costs of care for stroke patients in the USA in 2017 were USD 49.8 billion (Virani 2021). Stroke is considered a devastating disease from the point of view of the patient and the health system.

There are two main categories of stroke: ischemic and hemorrhagic. Approximately 87% of all strokes are ischemic, the main causes of which are carotid artery stenosis, hypertension, and cardiac arrhythmia (Virani 2021). Carotid artery stenosis is responsible for approximately 8% of all strokes and occlusion is judged to be responsible for 3.5% (Flaherty 2013). Patients with carotid artery stenosis are at high risk of a new stroke episode (Easton 2009; Hillen 2003; Moore 1995). The estimated risk of recurrence after a first ischemic episode is 6.4% during the first two to three days, 19.5% within seven days, and 26.1% within 14 days after the initial neurologic event (Tsantilas 2015). In addition, the chances of dying from a subsequent stroke are much higher.

The most important reason for identifying individuals with symptomatic carotid stenosis is the chance to proceed with carotid artery revascularization to prevent a new ischemic episode of stroke or death (Morris 2017). The NASCET 1991 trial found that the two‐year risk of ipsilateral stroke for participants with 70% to 99% carotid stenosis was 26% in those undergoing clinical treatment and 9% in those treated surgically, and the risk was reduced from 22.2% to 15.7% after five years among participants with moderate stenosis (50% to 69%).

Carotid revascularization can be performed by conventional or endovascular surgical treatment and aims to re‐establish adequate blood flow by removing significant stenosis in the vessel. There is strong evidence that carotid endarterectomy should be performed within two weeks of the neurologic event, and urgent revascularization may be considered for stable individuals who have a limited area of infarction with a large penumbra (Fonseca 2021; Rerkasem 2020; Ricotta 2011; Rothwell 2004; Vasconcelos 2016). Important guidelines recommend carotid revascularization be performed as early as possible after the neurologic index event in patients with symptomatic carotid stenosis (≥ 50%) (ESVS Writing Group 2018; Hobson 2008; NICE 2017). The value of revascularization decreases over time: three months after the event, revascularization has no more benefit to the patient than it has to an asymptomatic patient (NCC‐CC 2008; Rothwell 2004). The diagnosis should be confirmed and the severity of extracranial carotid stenosis estimated to perform the correct treatment.

Duplex ultrasound (DUS) is a widely available, non‐invasive, and cost‐effective test, which is usually the test of choice for identifying carotid stenosis and characterizing the severity of the lesion. It is currently still used primarily as a screening and selection test for patients who will undergo more expensive and invasive tests, such as computed tomography angiography (CTA), magnetic resonance angiography (MRA), or digital subtraction angiography (DSA). This review seeks to establish the diagnostic value of DUS for the diagnosis of extracranial carotid stenosis in symptomatic patients. We aim to define whether an individual with symptomatic carotid stenosis should undergo carotid endarterectomy based on DUS alone. In addition, we assess whether DUS is accurate to identify carotid occlusion and patients with non‐significant carotid stenosis who should receive clinical management. This review also contributes to the best decision‐making when clinicians face patients who have an iodine allergy or kidney failure and cannot undergo CTA, MRA, or DSA but who would benefit from carotid revascularization.

Target condition being diagnosed

Carotid artery stenosis is an atherosclerotic lesion that narrows the carotid artery. The deposit of cholesterol plaques on the vessel walls leads to their narrowing and usually occurs in regions of bifurcations, branches, or curvatures, all places of flow disturbance. Although many factors related to the patient and the characteristics of the plaque are studied, the most important parameter in choosing the therapeutic option is still the degree of carotid artery stenosis.

Extracranial carotid artery stenosis can be clinically classified as mild (< 50%), moderate (50% to 69%), severe (70% to 99%), and occlusion (100%) (Grant 2003). Each threshold has an influence on treatment choices for the patient.

Individuals with sudden ipsilateral visual loss, transient ischemic attack (TIA), and ischemic stroke associated with significant (50% to 99%) carotid stenosis within 180 days are considered symptomatic and may require some type of revascularization procedure. Besides, symptomatic individuals with < 50% carotid artery stenosis should receive the best medical management available, and other sources of the stroke should be investigated. Patients with carotid artery occlusion should also receive medical management (ESVS Writing Group 2018; Flumignan 2017; Ricotta 2011).

Index test(s)

DUS is a widely available, low‐cost, truly non‐invasive technique; it is well tolerated by patients and thus ideal for screening and diagnosing atherosclerotic plaque. DUS presents high sensitivity and specificity for diagnosing internal carotid artery (ICA) stenosis in numerous studies, although the results can vary among laboratories and operators (ESVS Writing Group 2018; Souza 2005; Surur 2013; Ventura 2015; Wardlaw 2006a). Currently, DUS is the modality of choice for the initial evaluation of carotid artery disease (ESVS Writing Group 2018; Flumignan 2017; Ricotta 2011). DUS combines B‐mode ultrasonography for morphological images and pulse‐wave Doppler spectrum analysis for flow velocity measures. DUS usually evaluates anatomic images of cervical portions of the common carotid artery (CCA), ICA, and external carotid artery (ECA) and measures their blood flow velocity. DUS can directly measure the luminal diameter of the artery or stenotic section; but rather, its diagnosis relies on blood flow velocity as an indicator of the degree of stenosis.

In 1987, the first validated classification of stenosis based on objective velocity criteria, known as the ‘Strandness Criteria’, was published (Taylor 1987). Since then, different criteria for the classification of carotid stenosis have been developed, and there is still substantial variability from laboratory to laboratory. In 2003, the American Society of Radiology held a conference and standardized the ultrasound criteria to determine stenosis (Grant 2003). They recommended duplex velocity criteria (measurements of internal carotid artery [ICA] peak systolic velocity [PSV] and end‐diastolic velocity [EDV] as well as the ICA/common carotid artery [CCA] PSV ratio) and morphological characteristics (Table 2). The classification by Grant 2003 remains the most used and recommended criteria in clinical practice (AbuRahma 2008; AbuRahma 2011; ESVS Writing Group 2018; Ricotta 2011).

1. DUS criteria for internal carotid stenosis.

Consensus panel based on Grant 2003
Degree of stenosis (%)	Primary parameters		Additional parameters
	ICA PSV (cm/sec)	Plaque estimate (%)*	ICA/CCA PSV ratio	ICA EDV (cm/sec)
Normal	< 125	None	< 2.0	< 40
< 50%	< 125	< 50	< 2.0	< 40
50% to 69%	125 to 230	≥ 50	2.0 to 4.0	40 to 100
≥ 70% but less than near occlusion	> 230	≥ 50	> 4.0	> 100
Near occlusion	High, low or undetectable	Visible	Variable	Variable
Total occlusion	Undetectable	Visible, no detectable lumen	Not applicable	Not applicable
*Plaque estimate (diameter reduction) based on DUS B‐mode and on additional color mode ultrasound

CCA: common carotid artery
DUS: duplex ultrasound
EDV: end diastolic velocity
ICA: internal carotid artery
PSV: peak systolic velocity

The disadvantages of DUS include limited visualization of the proximal CCA and distal ICA and technical difficulties related to the patient's physical condition (e.g. obesity, heart failure, postoperative status). Also, there are no widely acknowledged standardized criteria for pseudo‐occlusion on ultrasound (Fonseca 2021). Furthermore, contrast‐enhanced ultrasound is increasingly being used to evaluate patients with known or suspected atherosclerosis; it can help identify carotid plaque ulcerations, differentiate occlusion from pseudo occlusion, identify carotid dissection, and identify intraplaque neovascularization (Rafailidis 2017). With improved technology, the accuracy of this imaging test has increased significantly over time.

Clinical pathway

Evaluation of a patient with suspected symptomatic carotid stenosis should start with a complete history of the patient's comorbidity and risk factors for atherosclerotic disease. There should also be a physical examination because atherosclerotic carotid artery occlusive disease is a systemic disease (ESVS Writing Group 2018). The atherosclerotic carotid disease imaging diagnosis includes four tests: DUS, DSA, CTA, and MRA. These tests are used alone or in combination.

Patients who present with neurologic symptoms from non‐disabling stroke or TIA should undergo a non‐invasive diagnostic method in the initial evaluation (Brott 2011; ESVS Writing Group 2018; Flumignan 2017; NCC‐CC 2008; Ricotta 2011). Non‐invasive tests include DUS, CTA, and MRA. Patients presenting any degree of extracranial carotid stenosis should be treated with antiplatelet and lipid‐lowering therapy; carotid revascularization should be considered for those presenting significant stenosis (Brott 2011; ESVS Writing Group 2018; Hobson 2008; NICE 2017; Orrapin 2017; Ricotta 2011). Rapid imaging of the carotid artery is essential because there is a short time window for effective stroke prevention in patients presenting significant carotid artery stenosis. Although treatment is beneficial until 180 days after the first neurologic episode, current guidelines recommend that carotid intervention should be performed as soon as possible, ideally before 14 days (ESVS Writing Group 2018).

DSA was considered the gold standard to assess extracranial stenosis of carotid vessels, but it is an invasive method and carries a risk of morbidity or even mortality (ACAS 1995; Davies 1993; Hankey 1990). Its main limitations make this test unsuitable as a screening modality and rarely required for preoperative imaging (unless there are discrepancies on non‐invasive tests). CTA and MRA are replacing DSA. They usually use contrast agents and allow important additional evaluations of the aortic arch, supra‐aortic trunks, distal ICA, and intracranial vessels; this information is mandatory in stenting cases.

In clinical practice, the initial study is usually a bilateral carotid DUS to determine whether carotid stenosis contributes to the patient's symptoms (Brott 2011; ESVS Writing Group 2018; Ricotta 2011). After the first test, the treatment can be defined based solely on this initial test if it is reliable (Ricotta 2011). However, a second look by a different examiner or subsequent confirmation of results with DSA, CTA, or MRA for therapeutic programming is usual and recommended (ESVS Writing Group 2018).

The clinical pathway can vary depending on the center, and a recent guideline suggested that CTA is the most cost‐effective diagnostic method for patients at high risk of carotid artery stenosis in whom early revascularization could be performed (Kleindorfer 2021). Although current clinical guidelines recommend DUS as a first‐line imaging modality, studies have shown a significant misclassification rate before carotid endarterectomy (Collins 2005; Johnston 2001). Moreover, many authors draw attention to the low quality of the studies that have determined the accuracy of non‐invasive tests. Most guideline recommendations are based on old studies of questionable quality (Wardlaw 2006a).

Prior test(s)

In symptomatic patients (ischemic attack, amaurosis fugax, or ischemic stroke), DUS is recommended as the initial test because it is safe, inexpensive, and widely available. Therefore, individuals should not have any formal testing completed before DUS.

Role of index test(s)

DUS has been accepted by some investigators in qualified laboratories as a satisfactory method to determine the severity of carotid stenosis, being the basis of clinical decisions (Howard 2017). However, its use as the only imaging modality prior to performing carotid endarterectomy has been the subject of some controversy. In clinical practice, it is used primarily for screening and selecting patients for other non‐invasive and confirmatory tests, such as CTA or MRA.

Nevertheless, the accuracy of DUS remains a point of discussion (ESVS Writing Group 2018; Souza 2005; Surur 2013; Ventura 2015; Wardlaw 2006a).

Alternative test(s)

Digital subtraction angiography

Digital subtraction angiography was considered the gold standard against all other imaging modalities in individuals with extracranial cerebrovascular disease, even with its risks. Measurement of carotid stenosis is usually done using the NASCET 1991 method. The ECST 1998 method is avoided because it may overestimate carotid stenosis (Figure 1). The cut‐off points of 50% and 70% carotid artery stenosis with the NASCET method have been shown to be equivalent to approximately 75% and 85% for the ECST method, respectively (Nicolaides 1996). The major DSA limitations that make it inappropriate as a screening modality include its cost and associated risks, specifically of stroke and death. Studies have reported a 4% risk of TIA or minor stroke, a 1% risk of major stroke, and even a small (1%) risk of death (Davies 1993; Hankey 1990). Given its invasive characteristics, DSA has now been replaced by other effective, non‐invasive diagnostic methods, and DSA should be reserved for patients in whom non‐invasive imaging methods are contraindicated or inconclusive. In this review, we will consider the NASCET method for determining carotid stenosis by using DSA compared with DUS (Figure 1).

1.

Longitudinal view of carotid bifurcation with methods of measuring carotid stenosis at angiography

A : narrowest ICA diameter
B : normal distal cervical ICA diameter
C : estimated original diameter at the site of the most stenosis
CCA : common carotid artery
ECA : external carotid artery
ECST : European Carotid Surgery Trial
ICA : internal carotid artery
NASCET : North American Symptomatic Carotid Endarterectomy Trial

Contrast‐enhanced magnetic resonance angiography

MRA is another option to provide images of the carotid artery by different techniques, with or without contrast enhancement. The sensitivity of contrast‐enhanced MRA is high, and contrast should be used in all examinations for carotid stenosis diagnosis. Essentially, MRA uses the information of a powerful magnetic field, radiofrequency waves, and a computer program to create highly detailed imaging from different human tissues, including vessels and blood. In a systematic review of published studies on DUS and MRA, using DSA as the gold standard, MRA was found to be both sensitive and specific for detecting carotid stenosis, with a pooled sensitivity of 95% and a pooled specificity of 90% for the diagnosis of 70% to 99% carotid artery stenosis (Nederkoorn 2003). The classification of stenosis is according to the NASCET method (Figure 1).

The notable strengths of contrast‐enhanced MRA are its relative insensitivity to arterial calcification and lack of exposure to ionizing radiation. The limitations of contrast‐enhanced MRA include overestimation of stenosis, the inability to discriminate between subtotal and complete arterial occlusion, and the risk of nephrogenic systemic fibrosis when patients with pre‐existing renal dysfunction are exposed to high doses of gadolinium (Brott 2011). Furthermore, a substantial fraction of patients cannot be examined, such as patients who have claustrophobia, extreme obesity, or incompatible implanted devices such as pacemakers or defibrillators, and MRA is not a readily available method.

Computed tomography angiography

CTA is a validated tool for non‐invasive assessment of the degree of carotid artery stenosis (Daolio 2019; Duddalwar 2004). The rapid acquisition of spiral CTA images allows excellent timing with contrast administration and provides quality images that are less susceptible than MRA to overestimating the severity of carotid stenosis. As with MRA, CTA provides anatomic imaging from the aortic arch through the circle of Willis and the brain parenchyma, with multiplanar reconstruction and analysis allowing evaluation of even very tortuous vessels. Vessel wall imaging is an advantage of CTA and MRA over DSA because the latter detects only the flow (i.e. the contrast in blood). The classification of stenosis is measured according to the NASCET method (Figure 1). However, there are acknowledged drawbacks to CTA, such as the need for intravenous contrast and potential contrast nephrotoxicity, the ionizing radiation dosage, and calcification artefacts.

Rationale

Symptomatic patients with extracranial carotid stenosis should be evaluated rapidly and revascularization planned. If symptomatic patients undergo endarterectomy based on DUS alone, they will not be exposed to ionizing radiation or potentially nephrotoxic contrast materials. In addition, it will be much more cost‐effective to the health system. It must also be considered that there are many places where access to DSA, CTA, or MRA is limited, a factor that could delay treatment, whereas DUS is widely available in hospitals around the world. On the other hand, the decision of the best path to identify carotid stenosis should consider the risk of missing a potentially treatable stenosis (i.e. false‐negative result), which could lead the patient to a new and potentially worse ischemic episode, and the risk of performing surgery unnecessarily based on a false‐positive result.

Doubts about the diagnostic value regarding DUS have previously been published (Moore 1995), and other authors have also questioned its value (Collins 2005). Currently, various guidelines suggest performing DUS as the first diagnostic method, with additional imaging required when DUS is non‐diagnostic (ESVS Writing Group 2018; NCC‐CC 2008; Ricotta 2011). Others suggest that two non‐invasive methods should be performed before endarterectomy and, if only DUS is to be performed, then it should be repeated with a second operator to confirm the result (ESVS Writing Group 2018).

The complexity of diagnostic tests associated with significant variability in the estimates of their accuracy in the literature and studies without standardization of methodology increases the difficulty of standardizing the best diagnostic path for patients with neurologic symptoms suspected of carotid stenosis. Recommendations from different societies are often based on individual studies and old reviews. Knowing the limitations and accuracy of DUS in these patients and evaluating the methodology applied for these determinations play a fundamental role in decision‐making in clinical practice (ESVS Writing Group 2018; NCC‐CC 2008; Ricotta 2011). Understanding diagnostic tests goes beyond knowing their accuracy: it requires identifying their risks, benefits, consequences, and the correct interpretation of results to offer the best therapeutic planning to the patient.

Objectives

To assess the accuracy of DUS in symptomatic patients (sudden visual loss, hemispheric TIA, and ischemic stroke) with suspected extracranial carotid artery stenosis verified by DSA, MRA, or CTA.

Secondary objectives

We planned to assess and evaluate in subgroup analyses any method that could improve accuracy in addition to duplex: microbubble contrast, Power Doppler or similar, and color mode. However, due to the lack of data on contrast and Power Doppler, we only performed subgroup analysis for the color resource in the 70% to 99% range of carotid artery stenosis.

Methods

Criteria for considering studies for this review

Types of studies

We included cross‐sectional or diagnostic test accuracy (DTA) cohort studies assessing DUS against an acceptable reference standard (DSA, MRA, or CTA). We included both prospective and retrospective studies. We included both blinded and non‐blinded studies and investigated the effect of excluding non‐blinded studies by means of sensitivity analyses. We considered a study to be blinded if the examiner of one method did not know the result of the other test. Case reports and case‐control studies were not considered eligible for inclusion because they often overestimate the accuracy that a test has in clinical practice (Rutjes 2005). We excluded studies with an excessively long period (more than four weeks) of time between the index and reference tests, due to changes in the patient’s stenosis and risk of clinical degradation over time and the definition of a symptomatic patient (NASCET 1991). The timing of revascularization of symptomatic internal carotid artery stenosis has been changing over the years. It is still accepted that treatment is beneficial until 180 days after the first neurological episode, but current guidelines already recommend that carotid intervention should be performed as soon as possible, ideally before 14 days (ESVS Writing Group 2018). Therefore, we found four weeks between tests a reasonable time for carotid imaging.

Participants

Symptomatic patients with suspected carotid artery stenosis. Individuals with sudden visual loss, hemispheric TIA, and ischemic stroke associated with carotid stenosis are considered symptomatic (Rothwell 2004). We accepted studies in which at least 70% of included participants were symptomatic.

We excluded participants who did not receive DUS, those for whom the time between the index test(s) and the alternative test(s) was too long (more than four weeks), or those who had had a disabling stroke (modified Rankin Score ≥ 3) because the presence of a severe neurological impairment is known to limit the accuracy of diagnostic techniques (Bonita 1988; Rankin 1957).

Index tests

For DUS, we considered B‐mode identification and velocity‐based estimation of carotid stenosis with or without additional resources (e.g. microbubble contrast, Power Doppler or similar, and color mode). We considered the classification of carotid stenosis with DUS defined with validated duplex velocity criteria (measurements of ICA PSV, EDV, and the ICA/CCA PSV ratio) and morphological characteristics. We used the velocity criteria statement and the parameter priorities of Grant 2003 (Table 2).

Target conditions

Extracranial carotid stenosis can be clinically classified as mild (< 50%), moderate (50% to 69%), severe (70% to 99%), and occlusion (100%) (NASCET 1991). The data from studies should be consistent with this definition or conversion should be possible. Symptomatic carotid stenosis is defined as when an individual presents with sudden ipsilateral visual loss, hemispheric TIA, or ischemic stroke within three months associated with carotid stenosis (Rothwell 2004).

Reference standards

We accepted DSA, MRA, or CTA as reference standards. Due to risks associated with its use, DSA is no longer routinely performed for diagnosis in many centers (ESVS Writing Group 2018). However, until the end of the 20th century, catheter‐based angiography was the test used to measure carotid stenosis in the majority of carotid endarterectomy trials. Carotid stenosis should be classified according to the NASCET method (or conversion should be possible) (Figure 1). As current guidelines support the investigation of carotid stenosis with less invasive methods such as MRA and CTA, we also accepted any of these as standard reference methods, and we presented the results separately.

Search methods for identification of studies

Electronic searches

On 15 February 2021, the Cochrane Stroke Group Information Specialist searched the following electronic databases combining topic‐related and DUS terms:

• Cochrane Register of Diagnostic Test Accuracy Studies (CRDTAS); the full list of the databases, journals, and conference proceedings that have been searched, as well as the search strategies used, are described in the 'Specialised register' section on Cochrane Stroke's website;

• Cochrane Central Register of Controlled Trials (CENTRAL) (the Cochrane Library, latest issue) (Appendix 1);

• MEDLINE Ovid (from 1946 to present) (Appendix 2);

• Embase Ovid (from 1974 to present) (Appendix 3);

• ISI Web of Science: Science Citation Index Expanded (SCI‐EXPANDED) and Conference Proceedings Citation Index‐Science (CPCI‐S) (from 1900 to present) (Appendix 4);

• Database of Abstracts of Reviews of Effects (DARE) (Appendix 5);

• Health Technology Assessment (HTA) Database and International HTA Database; database.inahta.org (Appendix 5);

• Latin American and Caribbean Health Science Information (LILACS) and Índice Bibliográfico Español de Ciencias de la Salud (IBECS) (from 1982 to present) (Appendix 6).

We developed the MEDLINE search strategy with the help of the Cochrane Stroke Group Information Specialist, and we adapted it for the other databases, where necessary Appendix 2).

Searching other resources

We searched the following trial registries (15 February 2021) for details of ongoing and unpublished trials:

• US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov (www.clinicaltrials.gov) (Appendix 7);

• World Health Organization International Clinical Trials Registry Platform (ictrptest.azurewebsites.net/Default.aspx).

We checked the bibliographies of the included trials for additional references to relevant studies and used the Science Citation Index Cited Reference Search for forward tracking of important articles. We also contacted specialists in the field, manufacturers, and the authors of the included studies for any unpublished data.

Data collection and analysis

Selection of studies

Three review authors (NC, LCUN, and RLGF) independently screened and applied the selection criteria to the titles and abstracts identified as a result of our search strategy. We excluded duplicates and studies that did not meet the inclusion criteria. We retrieved the full‐text articles for reports deemed relevant, and two review authors (NC and RLGF) independently assessed the full‐text articles for inclusion or exclusion, and identified and recorded the reasons for exclusion. Any disagreements were resolved through discussion with the author team (JCCBS, CDQF, RS, LCUN, and VV). We included studies as of 1980 because that was when DUS technology began to be applied in clinical practice.

Data extraction and management

Two review authors (NC and RLGF) independently extracted the data from the included studies using a standard form. Any disagreements were resolved by discussion until consensus was established. When necessary, a third review author was consulted (LCUN). When necessary, we contacted the study authors for missing data. We sent data requests to study authors of studies not included in meta‐analyses before excluding a study due to insufficient data. We collected data on details of the included study (authors, study origin, year and language of publication, study design); characteristics of participants (age and gender); index test and definition of criteria used to determine the grade of stenosis; tests carried out prior to the index test; reference standard and definition of criteria used to determine the grade of stenosis; and numerical results (number of true positives, false positives, true negatives, and false negatives). When possible, we extracted 2 × 2 data directly. Alternatively, we reconstructed 2 × 2 tables by entering data on sensitivity, specificity, the total number of participants, and the proportion of diseased participants in the Review Manager 5 diagnostic accuracy calculator (RevMan 2020). We also extracted details of test threshold(s) used for interpretation of the results and the data on the technical aspects of DUS and the reference standards.

Assessment of methodological quality

We adopted the Quality Assessment of Diagnostic Accuracy Studies (QUADAS‐2) tool to assess the methodological quality of the included studies (Whiting 2006; Whiting 2011). Any disagreements were resolved by discussion; if disagreement persisted, all review authors were consulted. We presented the outcome data of the methodological quality assessment in Table 3 summarizing the number of studies with low, high, or unclear risk of bias for each of the four domains (patient selection, index test(s), reference standard, and flow and timing). We used Review Manager 5 to construct methodological quality summary graphs (RevMan 2020). We planned to conduct sensitivity analyses excluding studies at high risk of bias. We considered the overall risk of bias of an included study as low if there was no high‐risk judgement in the four main domains: patient selection, index test, reference standard, and flow and timing.

2. QUADAS‐2 'Risk of bias' and applicability judgements.

Quality Assessment of Diagnostic Accuracy Studies‐2 ( QUADAS‐2)
Patient Selection	A. Risk of bias	Signaling question 1: was a consecutive or random sample of patients enrolled?	Yes: it is described that the sample was consecutive or a random sample. No: it is described that the sample was not consecutive or a random sample. Unclear: the method of sampling is ambiguous.
		Signaling question 2: was a case‐control design avoided?	Yes No
		Signaling question 3: did the study avoid inappropriate exclusions?	Yes: the study included all symptomatic patients. No: the study excluded patients with neurological symptoms. Unclear: the study's exclusion criteria allow for inappropriate exclusions.
		Could the selection of participants have introduced bias?	RISK: High Low Unclear
	B. Concerns regarding applicability	We will include individuals with symptomatic carotid stenosis (i.e. those with sudden visual loss, hemispheric TIA, and ischemic stroke within 3 months associated with carotid stenosis). Patients may or may not have been previously tested. We will describe included participants (symptoms, prior testing, presentation, intended use of index test, and setting).
		Is there concern that the included participants do not match the review question?	CONCERN: High Low Unclear
Index tests(s)	A. Risk of bias	Index test: DUS, i.e. B‐mode identification (morphological analysis) and velocity‐based estimation of carotid artery stenosis with or without color mode We will describe the index test and how it was conducted and interpreted.
		Signaling question 1: were the index test (DUS) results interpreted without knowledge of the results of the reference standard?	Yes: it is described that the index test was performed and interpreted in a blind manner. No: the results of the reference standard were known to the DUS operator. Unclear: it is not reported.
		Signaling question 2: if a threshold was used, was it prespecified? We will use the velocity criteria statement reported in Grant 2003 .	Yes: the threshold used to define positive stenosis was prespecified. No: threshold was not described or was determined after analyzing the results. Unclear: the threshold that was used to define positive stenosis and how it was chosen is unclear.
		Could the conduct or interpretation of the index test have introduced bias?	RISK: High Low Unclear
	B. Concerns regarding applicability	Is there concern that the index test, its conduct, or interpretation differ from the review question?	CONCERN: High Low Unclear
Reference standard	A. Risk of bias	Due to risks associated with its use, DSA is no longer routinely performed in many centers. We will therefore accept as reference standards any one of the following: DSA, MRA, or CTA. We will describe the reference standard test and how it was conducted and interpreted.
		Signaling question 1: is the reference standard likely to correctly classify the target condition? Does the study report that either standards DSA, MRA, or CTA was performed for all participants? Are the reference standard results reported as NASCET 1991 method or is conversion possible ( Figure 1 )?	Yes: reference standard was described and performed for all included participants. No: the test was not performed in all included participants. Unclear: it is not described if the test was performed to all included participants.
		Signaling question 2: were the reference standard results interpreted without knowledge of the results of the index test? Was the person classifying the reference standard results unaware of the DUS results?	Yes: the person performing the reference standard test results was unaware of the DUS test results. No: the person performing the reference standard test results was aware of the DUS test results. Unclear: not reported
		Could the reference standard, its conduct, or its interpretation have introduced bias?	RISK: High: the reference standard was not read blind to the index test, or participants received the reference standard according to the results of the index test. Low: all included participants received the reference standard, and it was performed in a blind manner. Unclear: not reported
	B. Concerns regarding applicability	Is there concern that the target condition as defined by the reference standard does not match the review question?	CONCERN: High Low Unclear
Flow and timing	A. Risk of bias	We will describe any participants who did not receive the index test(s) and/or reference standard or who were excluded from the 2 x 2 table.
		We will describe the time interval and any interventions between index test(s) and reference standard.
		Signaling question 1: was there an appropriate interval between index test and reference standard?	Yes: the time interval between DUS and reference standard was less than 4 weeks. No: the time interval between DUS and reference standard was more than 4 weeks. Unclear: the time interval between DUS and reference standard was not reported or reported as median time.
		Did all patients receive the same reference standard?	Yes No Unclear: not reported
		Were all patients included in the analysis?	Yes No Unclear: not reported
		Could the patient flow have introduced bias?	RISK: High Low Unclear

CEMRA: contrast‐enhanced magnetic resonance angiography
CTA: computed tomography angiography
DSA: digital subtraction angiography
DUS: duplex ultrasound
MRA: magnetic resonance angiography
TIA: transient ischemic attack

Statistical analysis and data synthesis

We performed the analyses following Chapter 10 of the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (Macaskill 2010). We primarily used Cochrane's Review Manager 5 software for baseline analyses (RevMan 2020); we used R software (R Project 2018) for additional analyses and plots, when necessary. Since all included studies reported data using the carotid arteries as the unit of analysis, we also considered the number of carotid arteries as our unit of analysis submitted to both the index test (DUS) and the reference standard. Carotid stenosis should be classified according to the NASCET method (or conversion should be possible) in the reference standard tests, and we adopted the threshold described by Grant 2003 (Table 2) to the index test.

We extracted or derived data from the included studies for each imaging test and each range of carotid stenosis and generated 2 × 2 contingency tables of true‐positive cases, false‐positive cases, false‐negative cases, and true‐negative cases. We considered severe (70% to 99%) and moderate (50% to 69%) carotid artery stenosis as positive and analyzed each of these ranges separately; we also analyzed < 50% carotid stenosis and carotid occlusion. If more than one test was used as a reference test, we constructed a 2 × 2 table for each one, comparing it with DUS. We calculated sensitivity and specificity with 95% confidence intervals (CIs) for each test in each study. We used forest plots to display the sensitivity and specificity estimates measured in each study and to illustrate the variation in estimates among studies.

When at least three studies were evaluating the same range of stenosis and the same reference standard and reported consistent test accuracy estimates, we pooled sensitivity and specificity using the bivariate random‐effects method. This method is recommended for studies using the same positivity threshold (Reitsma 2005). In the bivariate model, the combination of two normally distributed outcomes, the logit‐transformed sensitivities and specificities, while acknowledging the possible correlation between them, leads to the bivariate normal distribution. The parameters of the bivariate model are estimated in a single model to incorporate the possible correlation between sensitivities and specificities. We calculated the Chi ² test for equality to assess the heterogeneity of sensitivity and specificity among studies. We also calculated Spearman’s correlation coefficient to investigate the presence of the threshold effect (correlation between sensitivity and specificity estimates), considering a correlation coefficient of ‐0.60 as indicative of the threshold effect. From the bivariate model, we used Review Manager 5 (RevMan 2020), to plot estimates of sensitivity and specificity from each study and to generate summary receiver operating characteristic (sROC) curves. We calculated a 95% confidence region and a prediction region around the summary estimates from the parameters of the bivariate model and added it to the plot to illustrate the precision in which the estimate was combined (region of an average) and to illustrate the probable range of values that would be expected in 95% of future studies. The combined estimates of likelihood ratios and diagnostic odds ratios (DOR) were obtained by using the Zwindermann & Bossuyt procedure (Zwinderman 2008). This procedure uses the adjustment parameters of the bivariate model to generate sensitivity samples and false positive rates and calculate the 95% CI. In this case, we use the number of 10,000 iterations.

All analyses were performed with the aid of the 'mada' package (Doebler 2017) implemented in the R program (R Project 2018).

We summarized findings with absolute values on 1000 tested participants with the estimated number of false positives (undue treatment) and false negatives (missing appropriate treatment).

Investigations of heterogeneity

We performed meta‐regression analyses to explore potential sources of heterogeneity among the studies by adding one covariate at a time to the bivariate model. A P value less than 0.05 was considered to indicate a significant effect. For the categorical covariates that influence the heterogeneity in the sensitivity and specificity estimates, we performed subgroup analyses if the number of studies made it meaningful to add parameters to the models.

In the protocol, we planned to investigate the potential sources of heterogeneity of the generation of technology; characteristics of the participant population (age and gender); additional ultrasound resources (color mode and Power Doppler, or similar); use of contrast‐enhanced DUS (microbubbles) versus DUS; and time of publication. As evident from the forest and ROC plots, there was considerable between‐study heterogeneity in the test accuracy estimates. However, due to the small number of studies, it was not possible to perform all the planned analyses. We added mean age of the participant population and the prevalence of the disease as covariates in each comparison to analyze potential sources of heterogeneity. We were unable to perform meta‐regression analyses for the participants’ gender, generation of technology, and use of contrast‐enhanced DUS (microbubbles) due to a lack of data.

Sensitivity analyses

We intended to conduct several sensitivity analyses to compare the diagnostic accuracy by investigating the effect of excluding studies at high risk of bias and, in particular, non‐blinded studies. Due to the lack of suitable data (small number of studies in each category), sensitivity analyses were limited. It was possible to perform a sensitivity analysis to examine the impact of blinding for all carotid stenosis ranges for the DUS versus DSA comparison, except for < 50% carotid artery stenosis. We did not conduct sensitivity analyses by excluding studies at high risk of bias because there were insufficient data in each category of stenosis. We considered the overall risk of bias of an included study as low if there was no high‐risk judgement in the four main domains: patient selection, index test, reference standard, and flow and timing. For the 70% to 99% range of carotid artery stenosis, we included studies with similar thresholds, but not exactly the prespecified ones. Hence, we decided to perform a sensitivity analysis excluding all studies that did not exactly use the speed parameter as specified in Table 2. We also performed sensitivity analyses in the occlusion category by excluding Hammond 2008 and Lubezky 1998 , which only included patients that had already been diagnosed with carotid artery occlusion on DUS and, therefore, had no false negative test and low rates of specificity.

Assessment of reporting bias

We did not assess reporting bias because the relevant methods are not well developed for systematic reviews of DTA studies.

Results

Results of the search

Excluded studies

The results of the literature searches are outlined in Figure 2 . We initially identified 36,419 studies and removed 11,332 duplicates. Hence, 25,087 records remained for possible eligibility. After reading the title and the abstract of these records, we excluded 24,434 of them, as they did not meet the inclusion criteria, leaving 653 full‐text studies for eligibility assessment. After the full‐text evaluation, we excluded a further 289 articles that were not relevant to this review, and we excluded another 342 with one or more of the following reasons.

2.

Study flow diagram

• Studies did not assess or did not provide data on DUS accuracy for symptomatic carotid stenosis, even though it was performed (19).

• Less than 70% of the participants included were symptomatic (75).

• Studies did not define the proportion of symptomatic patients (132).

• Preliminary paper of DUS technique described (subjective visual impression of the degree of stenosis) or no objective criteria to estimate stenosis (55).

• Time between the index test and the alternative test was not specified or was more than four weeks (58).

• Accuracy was determined by comparison with the surgical specimen (2).

• Case‐control design (1).

Included studies

We included 22 studies that met our prespecified inclusion criteria. These studies had a total of 4957 carotid arteries, with a mean sample size of 126, ranging from 24 to 1011. The mean age of participants was 66.3 years (range 53 to 72 years), and the mean proportion of men was 70% of included participants. Five studies did not provide the participants’ demographic details (D'Onofrio 2006; Eliasziw 1995; Faught 1994; Hammond 2008; Knudsen 2002), and Chua 2007 described the male‐to‐female sex ratio as 2.9:1. From these 22 included studies, 15 were conducted in Europe (5 in Germany, 3 in the UK, 2 in the Netherlands, 2 in Italy, 1 in Denmark, 1 in France, and 1 in Sweden), 2 in Asia (1 in China and 1 in Singapore), 3 in North America (Faught 1994; Huston 1993 in the USA, and Eliasziw 1995 included patients from 50 North American centers), and 2 in Israel. We present a summary of the characteristics of the included studies in Table 4 .

3. Summary table of included studies.

Name	Study location	Ultrasound technology	Microbubbles contrast	Reference Standard	Quantitative analysis	DUS threshold	Carotids included	Number of participants	Mean age
Anzidei 2012	Italy	Aplio XV device (Toshiba Medical Systems, Japan) or Mylab 70 (Esaote Biomedica, Genoa, Italy)	No	DSA	< 50%, ≥ 50‐99% and occlusion	NASCET + PSV 125‐130 to ≥ 50%	335	170	69
Barlinn 2016	Germany	Aplio MX Toshiba‐ SSA‐780a System®, Toshiba Medical Systems, Germany	No	CTA	< 70%, ≥ 70‐99% and occlusion	(DEGUM criteria) ≥ 50%: ≥ 200 cm/s and ≥ 70%: ≥ 300 cm/s Table 11	593	303	72
Belsky 2000	Israel	Acuson 128, Acuson, Mountain View CA	No	CTA	< 70%, ≥ 70‐99% and occlusion	70–99%: PSV ≥ 250 and EDV ≥ 100	92	46	70
Borisch 2003	Germany	Sonoline Elegra 5.0 system (Siemens)	No	DSA, MRA	< 70%, ≥ 70‐99% and occlusion to DSA. < 70% and ≥ 70% for MRA	≥ 70%: PSV ≥ 250 /Ratio (r)ICA/CCA > 3	71	39	67.4
Chua 2007	Singapore	Diasonics Spectra (Diasonics Inc, Milpatas, California)	No	DSA	No	PSV ICA/ICCA ≥ 1.5 to 50% and ≥ 3.1 to ≥ 70%	188	94	64
Colquhoun 1992	UK	Acuson 128 duplex scanner	No	DSA	No	PSV ≥ 120: ≥ 50% and PSV ≥ 250 for ≥ 80%	53	50	53
Cui 2018	China	Esaote North America, Inc., Indianapolis, IN, USA	No	DSA	No	Table 2 ; Grant 2003	216	54	63
D'Onofrio 2006	Italy	Sequoia 512, Acuson/Siemens	No	DSA/MRA	≥ 50‐99% and ≥ 70‐99%	≥ 60% PSV ≥ 130 and EDV ≥ 40. ≥ 80% PSV ≥ 250 and EDV ≥ 100	41 DSA / 31 MRA	32	Not described
Das 2009	Germany	GE Vivid 7	No	CTA/MRA	No	(DEGUM criteria) ≥ 50%: ≥ 200 cm/s and ≥ 70%: ≥ 300 cm/s Table 11	30	15	69
Bray 1995	France	Not specified	No	DSA	No	≥ 50%: PSV ≥ 130 cm/s. ≥ 70%: PSV ≥ 250 cm/s	128	64	62
Eliasziw 1995	North America	Not specified	No	DSA	< 70%, ≥ 70‐99%	≥ 70%: PSV ≥ 250	1011	1011	Not described
Faught 1994	United States	QUAD I Angiodynograph (Quantum Medical Systems, Issaquah, Wash.) until the latter part of 1989, after which the Quantum 2000 (Quantum Medical Systerns)	No	DSA	< 50%, ≥ 50‐99%, < 70% and ≥ 70‐99%	PSV < 110 cm/s for stenosis 0‐29%/PSV 111‐130 cm/s for stenosis 30‐49%/PSV > 130 cm/s, EDV ≥ 100 for stenosis 50‐69%/PSV ≥ 230 cm/s, EDV ≥ 100 cm/s for stenosis 70‐99%	770	405	Not described
Golledge 1999	UK	Ultramark 9, HDI, Advanced Technology Laboratories, Wash	No	DSA	≥ 70‐99%	EDV ≥ 90 cm/s	100	50	71
Hammond 2008	UK	Acuson 128 XP10	No	DSA	Occlusion	No flow	24	30	Not described
Hansen 1996	Sweden	Acuson XP 10	No	DSA	No	y = 0.54.e 0.021 x (y = PSV ICA and x = the degree of stenosis expressed as the diameter reduction in %)	162	81	68
Heijenbrok‐Kal 2006	Netherlands	Ultramark 9 HDI or HDI 3000 (311 participants). Diasonics Master Series (39 participants)	No	DSA	< 50%, ≥ 50%, 50‐69%, ≥ 70%, near occlusion and occlusion	Table 2 ; Grant 2003	313	350	67
Huston 1993	United States	Acuson 128	No	DSA	< 50%, ≥ 50‐99% and occlusion	PSV < 125 cm/s = < 50% PSV ≥ 125 cm/s and EDV < 135 cm/s = 50‐79% PSV ≥ 125 cm/s and EDV ≥ 135 cm/s ≥ 80%	77	50	67
Knudsen 2002	Denmark	Siemens Sonoline Elegra	No	DSA	No	≥ 70%: PSV ≥ 150 cm/s, EDV ≥ 90 cm/s and PSV ratio ≥ 2.8	129	65	Not described
Link 1997	Germany	Philips P700	No	DSA	70‐99% and occlusion	PSV ≥ 200 cm/s	56	28	63
Lubezky 1998	Israel	HD13000 system	No	CTA, DSA	Occlusion	No flow	148 CTA/54 DSA	148	70
Nederkoorn 2002	Netherlands	Not specified	No	DSA	< 70%, ≥ 70‐99% and occlusion	0‐29%: PSV < 150 cm/s 30‐49%: PSV 150‐190 cm/s 50‐69% PSV 190‐270 cm/s 70‐99%: PSV ≥ 270 cm/s Occlusion: no flow	313	350	67
Wolfle 2002	Germany	Siemens (Elegra)	No	DSA	≥ 70‐99%	70‐99%: PSV ≥ 230 cm/s and EDV ≥ 70 cm/s	94	47	68

CCA: common carotid artery
CTA: computed tomography angiography
DEGUM: The German Society of Ultrasound in Medicine
DSA: digital subtraction angiography
EDV: end diastolic velocity
ICA: internal carotid artery
MRA: magnetic resonance angiography
NASCET: North American Symptomatic Carotid Endaterectomy Trial
PSV: peak systolic velocity

Eighteen studies used a prospective method for participant recruitment (Anzidei 2012; Borisch 2003; Bray 1995; Chua 2007; Colquhoun 1992; Cui 2018; Das 2009; D'Onofrio 2006; Eliasziw 1995; Hammond 2008; Hansen 1996; Heijenbrok‐Kal 2006; Huston 1993; Knudsen 2002; Link 1997; Nederkoorn 2002; Wolfle 2002; Golledge 1999), two used a retrospective method (Barlinn 2016; Belsky 2000), and in two studies it was unclear whether there was a prospective or retrospective design (Faught 1994; Lubezky 1998).

An effort was made to group the results from studies into clinically relevant categories described in Grant 2003 that serve as the basis for treatment decisions and were prespecified in Table 2 in our protocol (Cassola 2018). Details on the reported cut‐offs are presented in the Characteristics of included studies tables. When the criteria used to determine stenosis were too different from the duplex velocity criteria proposed in our protocol or when there was insufficient data to complete a 2 × 2 table for at least one category of stenosis, we described the results narratively without meta‐analysis. Seven studies, therefore, were not included in our quantitative analysis, and were described only narratively (Bray 1995; Chua 2007; Colquhoun 1992; Cui 2018; Das 2009; Hansen 1996; Knudsen 2002).

We focussed our review on symptomatic participants, but we also considered for inclusion studies with up to 30% of asymptomatic participants, we included six studies with mixed populations: Bray 1995 (18% presenting carotid bruit); Colquhoun 1992 (12% presenting non‐specific complaints); Faught 1994 (23%, authors did not describe why these patients were included); Hansen 1996 (11%, this study included only patients before carotid endarterectomy and asymptomatic patients previously undergone an endarterectomy on the symptomatic side, were operated on because of a contralateral asymptomatic severe stenosis); Lubezky 1998 (22%, this study evaluated only occlusion, had an unclear design); and Wolfle 2002 (27%, authors did not describe why these patients were included).

Chua 2007 was a prospective study of 188 carotid arteries in which the authors compared DUS and DSA. However, the calculated data on sensitivity and specificity were based on the ICA/CCA PSV ratio criterion (PSV ratio 3.1 for ≥ 70% ICA stenosis).

Colquhoun 1992 compared DUS to DSA in 53 carotid arteries, but the criteria to determine stenosis on DSA was ECST, and conversion to NASCET was not possible with the available data.

Cui 2018 was a prospective study that compared DUS and DSA in 54 participants but classified stenosis in the ICA and CCA, counting four vessels in each participant and presenting the results grouped. In this way, each participant was counted twice in the analysis.

Das 2009 was a prospective study of 30 internal carotid arteries that compared DUS to MRA and CTA. However, it provided a graphical representation of the results, and it was impossible to extract them into a 2 × 2 table with individual data.

Bray 1995 was a prospective study that compared DUS to DSA in 128 carotid arteries but provided insufficient data to complete a 2 × 2 table with results from each category of stenosis.

Hansen 1996 was a prospective study of 162 arteries comparing DUS with DSA. The degree of stenosis on DSA was calculated by measuring the smallest diameter in the stenotic zone compared with the diameter of the normal CCA proximal to the stenosis. It was not possible to convert this into the NASCET grade of stenosis.

Knudsen 2002 was a prospective study of 129 arteries comparing DUS to DSA. However, the threshold used to classify a ≥ 70% ICA stenosis was PSV ≥ 150 cm/s, EDV ≥ 90 cm/s, and ICA/CCA PSV ratio ≥ 2.8, which we considered too different from our pre‐established thresholds.

Nineteen studies used DSA as the reference standard (Anzidei 2012; Borisch 2003; Bray 1995; Chua 2007; Colquhoun 1992; Cui 2018; D'Onofrio 2006; Eliasziw 1995; Faught 1994; Golledge 1999; Hammond 2008; Hansen 1996; Heijenbrok‐Kal 2006; Huston 1993; Knudsen 2002; Link 1997; Lubezky 1998; Nederkoorn 2002; Wolfle 2002). Two of these studies also presented a comparison between DUS and MRA (Borisch 2003; D'Onofrio 2006), and Lubezky 1998 also presented results from DUS versus CTA. A total of four studies compared DUS and CTA (Barlinn 2016; Belsky 2000; Das 2009; Lubezky 1998), and three presented a comparison of DUS versus MRA (Borisch 2003; Das 2009; D'Onofrio 2006). There were insufficient data to perform a meta‐analysis of MRA as the reference standard (only two studies included). From the sixteen studies included in the quantitative analysis, when possible, we extracted data and completed a 2 × 2 table for each of the categories we proposed. We were able to include the most studies in the 70% to 99% carotid artery stenosis as well as occlusion categories. Nine studies (2770 carotid arteries) presented DUS versus DSA results for 70% to 99% carotid artery stenosis (Borisch 2003; D'Onofrio 2006; Eliasziw 1995; Faught 1994; Golledge 1999; Heijenbrok‐Kal 2006; Link 1997; Nederkoorn 2002; Wolfle 2002), and two studies presented results from DUS versus CTA for 685 carotid arteries (Barlinn 2016; Belsky 2000). Seven studies presented results for occlusion with DSA as the reference standard (Anzidei 2012; Borisch 2003; Hammond 2008; Heijenbrok‐Kal 2006; Huston 1993; Link 1997; Lubezky 1998; Nederkoorn 2002). Only Heijenbrok‐Kal 2006 presented 50% to 69% carotid stenosis results; therefore, it was impossible to perform a meta‐analysis.

The list and details of the included studies are presented in the Characteristics of included studies tables.

Methodological quality of included studies

Risk of bias varied considerably across the included studies. We summarized the results of the methodological quality of the included studies in Figure 3 and Figure 4.

3.

Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies

4.

Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study

Only Eliasziw 1995 and Link 1997 were judged as being at low risk of bias in all domains. Chua 2007 and Lubezky 1998 were classified as having high risk of bias in all domains.

Patient selection

In terms of risk of bias, 10 studies were judged as being unclear regarding patient selection (Barlinn 2016; Bray 1995; Cui 2018; Das 2009; Faught 1994; Golledge 1999; Heijenbrok‐Kal 2006; Huston 1993; Knudsen 2002; Wolfle 2002), mostly because the recruitment method and the sampling procedures were unclear. Eight studies were considered as being at high risk of bias (Anzidei 2012; Belsky 2000; Borisch 2003; Chua 2007; D'Onofrio 2006; Hammond 2008; Hansen 1996; Lubezky 1998). The primary potential source of bias in this domain was the failure to include all people with a negative screen. Mainly because they only enrolled patients with known disease, selected participants based on previous examinations or participants already referred to the institution for preoperative evaluation can result in greater estimates of diagnostic accuracy. Another potential source of bias was the exclusion of difficult‐to‐diagnose patients (i.e. extensive calcified carotid plaques). Anzidei 2012 only included patients with > 30% carotid artery stenosis on DUS. Belsky 2000 selected patients who were candidates for carotid endarterectomy of either one or both ICA. Borisch 2003 included patients referred for preoperative imaging. Chua 2007 excluded occlusion of one or both ICA and atypical flow patterns within vessels, such as low velocities in near‐occlusion, and extensive calcified plaques resulting in long segments of acoustic shadowing. D'Onofrio 2006 only included patients with ultrasonographic findings of > 50% carotid artery stenosis. Hammond 2008 only evaluated patients with an apparent carotid occlusion on DUS. Hansen 1996 only included patients already with a planned carotid endarterectomy. Lubezky 1998 only included patients with carotid occlusion diagnosed by DUS. Only four studies were at low risk of bias in this domain (Colquhoun 1992; Eliasziw 1995; Link 1997; Nederkoorn 2002).

Index test

Four studies were at high risk of bias based on the judgements made about the index test because all of them had no prespecified thresholds (Chua 2007; Golledge 1999; Hansen 1996; Lubezky 1998). We judged three studies as having unclear risk of bias for the index test (Belsky 2000; Borisch 2003; Das 2009), and all other included studies as low risk of bias.

Reference standard

We judged Chua 2007 and Lubezky 1998 as having high risk of bias in the reference standard domain because the study personnel was not blinded to the results from DUS. We judged seven studies as being at unclear risk of bias for this domain (Anzidei 2012; Belsky 2000; Cui 2018; Das 2009; Hammond 2008; Hansen 1996; Huston 1993), mostly because we did not know if the result of the reference standard was interpreted without the knowledge of the result of DUS. All other included studies were judged as having low risk of bias.

Flow and timing

Fourteen studies had a methodological concern due to flow and timing. In 11 of them, not all participants were included in the analysis (Anzidei 2012; Barlinn 2016; Borisch 2003; Bray 1995; Chua 2007; Colquhoun 1992; Hammond 2008; Heijenbrok‐Kal 2006; Huston 1993; Knudsen 2002; Nederkoorn 2002), and in two studies the participants did not receive the same reference standard (D'Onofrio 2006; Lubezky 1998). We judged two studies as having unclear risk of bias for this domain (Das 2009; Hansen 1996), and all other included studies as low risk of bias.

Applicability concerns

We analyzed the applicability concerns regarding patient selection, index test, and reference standard. Five included studies were judged as having low concern in all three domains (Barlinn 2016; Chua 2007; Eliasziw 1995; Link 1997; Nederkoorn 2002;).

Patient selection was the domain where most issues were found. Only six studies were judged as being of low concern (Barlinn 2016; Chua 2007; Cui 2018; Eliasziw 1995; Link 1997; Nederkoorn 2002); eight studies were judged as being of unclear concern (Colquhoun 1992; Das 2009; Faught 1994; Golledge 1999; Heijenbrok‐Kal 2006; Huston 1993; Knudsen 2002; Wolfle 2002); and eight studies were judged as being of high concern (Anzidei 2012; Belsky 2000; Borisch 2003; Bray 1995; D'Onofrio 2006; Hammond 2008; Hansen 1996; Lubezky 1998), mostly because of prior testing used for patient selection or because they described the included population as patients referred to surgery or referred to DSA.

There were no studies whose authors declared a conflict of interest.

Findings

The findings are collected in Table 1 .

We were able to formally compare five ranges of stenosis for DUS versus DSA (< 50%, 50% to 69%, 50% to 99%, 70% to 99%, and occlusion), and occlusion for DUS versus CTA. We did not perform meta‐analyses of studies in carotid artery stenosis categories and different reference standards for which two or fewer studies were included.

Duplex ultrasound versus digital subtraction angiography

Carotid artery stenosis of < 50%

For this category, there were four studies involving 1495 carotid arteries in 975 participants (Anzidei 2012; Faught 1994; Heijenbrok‐Kal 2006; Huston 1993). Sensitivity varied from 0.36 to 0.76, and specificity varied from 0.96 to 0.98 (Figure 5). Using the bivariate model, we estimated a summary sensitivity of 0.63 (95% CI 0.48 to 0.76) and a summary specificity of 0.99 (95% CI 0.97 to 1.0). The summary receiver operating plot (sROC) along with the summary point is illustrated in Figure 6 . The prevalence of < 50% carotid artery stenosis ranged from 14% to 70%.

5.

Forest plot of paired sensitivity and specificity estimated for studies assessing < 50% carotid artery stenosis with DSA as reference standard

6.

Summary ROC Plot of studies assessing < 50% carotid artery stenosis with DSA as reference standard

A meta‐regression analysis showed that the year of publication, the participants' age, and disease prevalence did not impact the accuracy estimates (sensitivity and specificity) (Table 5). All studies had a prospective design. There were not enough studies to perform subgroup analysis.

4. Meta‐regression analysis for < 50% carotid artery stenosis.

Covariate	P value of sensitivity	P value of specificity
Prevalence	0.237	0.015
Year of publication	0.830	0.069
Participants age	0.033	0.952

Carotid artery stenosis of 50% to 69%

Only one study with 330 participants (313 analyzed) and 313 carotid arteries was included (Heijenbrok‐Kal 2006). Sensitivity in this study was 0.28 (95% CI 0.17 to 0.41) and specificity was 0.9 (95% CI: 0.85 to 0.93). We did not perform a meta‐analysis on this combination, for reasons outlined above.

Carotid artery stenosis of 50% to 99%

We included five studies (1536 carotid arteries in 1007 participants) in this stenosis range (Anzidei 2012; D'Onofrio 2006; Faught 1994; Heijenbrok‐Kal 2006; Huston 1993). Sensitivity varied from 0.95 to 0.99 and specificity from 0.67 to 0.76 (Figure 7). Using the bivariate model, we estimated a summary sensitivity of 0.97 (95% CI 0.95 to 0.98) and a summary specificity of 0.70 (95% CI 0.67 to 0.73).The sROC plot along with the summary point is illustrated in Figure 8. The prevalence of 50% to 99% carotid artery stenosis ranged from 19% to 72% in the included studies.

7.

Forest plot of paired sensitivity and specificity estimated for studies assessing 50‐99% carotid artery stenosis with DSA as reference standard

8.

Summary ROC Plot of studies assessing 50‐99% carotid artery stenosis with DSA as reference standard

A meta‐regression analysis showed that the year of publication, the participants' age, and disease prevalence did not impact the accuracy estimates (sensitivity and specificity) (Table 6). All studies, except Anzidei 2012, had a prospective design.

5. Meta‐regression analysis for 50‐99% carotid artery stenosis.

Covariate	P value of sensitivity	P value of specificity
Prevalence	0.959	0.377
Year of publication	0.786	0.346
Participants age	0.231	0.308

We performed sensitivity analyses for lack of blinding of the index test interpreters to reference standard results or vice versa by excluding two non‐blinded studies (Anzidei 2012; Huston 1993). We found no impact on the results based on the likelihood ratio test (Chi² = 1.00, P = 0.61).

Carotid artery stenosis of 70% to 99%

We were able to include the most studies for carotid artery stenosis of 70% to 99%, namely 2770 carotid arteries (in 2312 participants) from nine studies (Borisch 2003; D'Onofrio 2006; Eliasziw 1995; Faught 1994; Golledge 1999; Heijenbrok‐Kal 2006; Link 1997; Nederkoorn 2002; Wolfle 2002) and 2770 carotid arteries were included in analysis. Our prespecified threshold for classifying this range of stenosis was an ICA PSV of 230 cm/s, but we accepted Borisch 2003, D'Onofrio 2006, and Eliasziw 1995 all of which used an ICA PSV of 250 cm/s, Nederkoorn 2002 that used an ICA PSV of 270 cm/s, and Link 1997 that used an ICA PSV of 200 cm/s because we considered all of them similar thresholds.

Sensitivity varied from 0.68 to 0.97 and specificity varied from 0.67 to 0.97 (Figure 9). Using the bivariate model, the summary sensitivity was 0.85 (95% CI 0.77 to 0.91) and the summary specificity was 0.98 (5% CI 0.74 to 0.90). The sROC plot along with the summary point is illustrated in Figure 10. The prevalence of 70% to 99% carotid artery stenosis ranged from 17% to 72% in included studies.

9.

Forest plot of paired sensitivity and specificity estimated for studies assessing 70‐99% carotid artery stenosis with DSA as reference standard

10.

Summary ROC Plot of studies assessing 70‐99% carotid artery stenosis with DSA as reference standard

A meta‐regression analysis to explore heterogeneity showed that prevalence (Chi² = 12.32, P = 0.00211) and year of publication (Chi² = 14.57, P = 0.000684) impacted accuracy estimates, with a trend for higher estimates of summary specificity for higher prevalence and a trend for higher estimates of summary sensitivity for more recent publications. The participants’ age did not impact the estimates of accuracy (Table 7).

6. Meta‐regression analysis for 70‐99% carotid artery stenosis.

Covariate	P value of sensitivity	P value of specificity
Prevalence	0.080	< 0.005
Year of publication	< 0.005	0.229
Participants age	0.899	0.422

We performed sensitivity analyses for lack of blinding of the index test interpreters to reference standard results or vice versa by excluding one non‐blinded study (Borisch 2003). We found no impact on the results (likelihood ratio test: Chi² = 0.45, P = 0.80).

A sensitivity analysis restricted to studies that used the exact velocity criteria described in our protocol (PSV ≥ 230 c/s) resulted in a summary sensitivity of 0.90 (95% CI 0.77 to 0.96) and a summary specificity of 0.83 (95% CI 0.59 to 0.95) (Faught 1994; Heijenbrok‐Kal 2006; Wolfle 2002), but we found no impact on the results (likelihood ratio test: Chi² = 1.62, P = 0.45).

Subgroup analysis for additional ultrasound resources (only the color resource was analyzed) showed no statistically significant difference between summary sensitivity or specificity (likelihood ratio test: Chi² = 4.26, P = 0.12).

Carotid artery occlusion

Seven studies (1212 carotid arteries in 1165 participants) compared DUS and DSA for carotid artery occlusion (Anzidei 2012; Borisch 2003; Hammond 2008; Heijenbrok‐Kal 2006; Huston 1993; Link 1997; Lubezky 1998; Nederkoorn 2002). Sensitivity varied from 0.62 to 0.99, and specificity varied from 0.07 to 0.99 (Figure 11). We estimated a summary sensitivity of 0.91 (95% CI 0.81 to 0.97) and a summary specificity of 0.95 (95% CI 0.76 to 0.99). The sROC plot along with the summary point is illustrated in Figure 12. The prevalence of carotid artery occlusion ranged from 14% to 95%. A meta‐regression analysis showed that the year of publication and participants’ age did not impact the estimates of accuracy (sensitivity and specificity), but it did show a trend for higher summary specificity for higher prevalence (Chi² = 18.91, P < 0.001) (Table 8). Link 1997 had a retrospective design; all other studies had a prospective design.

11.

Forest plot of paired sensitivity and specificity estimated for studies assessing carotid artery occlusion with DSA as reference standard

12.

Summary ROC Plot of studies assessing carotid artery occlusion with DSA as reference standard

7. Meta‐regression analysis for carotid artery occlusion.

Covariate	P value of sensitivity	P value of specificity
Prevalence	0.016	< 0.005
Year of publication	0.608	0.714
Participants age	0.817	0.405

We performed sensitivity analyses for lack of blinding of the index test interpreters to reference standard results or vice versa by excluding five non‐blinded studies (Anzidei 2012; Borisch 2003; Hammond 2008; Huston 1993; Lubezky 1998). We found a significant impact on the estimates of sensitivity (likelihood ratio test: Chi² = 7.75, P = 0.020).The summary sensitivity estimate was 0.93 (95% CI 0.87 to 0.96).

We performed sensitivity analyses excluding Hammond 2008 and Lubezky 1998, which only included patients that had already been diagnosed with carotid artery occlusion on DUS and, therefore, had no false negative test and low rates of specificity. There was a significant impact on the results of specificity (likelihood ratio test: Chi² = 16.44, P < 0.001). The summary estimate for specificity was 0.98 (95% CI 0.97 to 0.99).

See Appendix 8 for characteristics of the studies not included in meta analysis.

Duplex ultrasound versus computed tomography angiography

We included three studies for DUS versus CTA analyses: Barlinn 2016 and Belsky 2000 provided data for 70% to 99% carotid artery stenosis as well as for occlusion; Lubezky 1998 only included carotid arteries with occlusion.

Carotid artery stenosis of 70% to 99%

For this range of stenosis, two studies (685 carotid arteries in 354 participants) fulfilled our prespecified criteria (Barlinn 2016; Belsky 2000). Sensitivity varied from 0.57 to 0.94 and specificity varied from 0.87 to 0.98 (Figure 13). The prevalence of 70% to 99% carotid artery stenosis ranged from 1% to 34%.

13.

Forest plot of paired sensitivity and specificity estimated for studies assessing 70‐99% carotid artery stenosis with CTA as reference standard

Carotid artery occlusion

Three studies (833 carotid arteries in 499 participants) compared DUS and CTA for carotid artery occlusion (Barlinn 2016; Belsky 2000; Lubezky 1998). Sensitivity varied from 0.85 to 1.0 and specificity varied from 0.04 to 1.0 (Figure 14). The prevalence of carotid artery occlusion ranged from 4% to 91%. Using the bivariate model, the estimated summary sensitivity was 0.95 (95% CI 0.79 to 0.99) and the summary specificity was 0.91 (95% CI 0.09 to 0.99). The sROC plot along with the summary point is illustrated in Figure 15. The prevalence of 70% to 99% carotid artery stenosis ranged from 17% to 72%. There were not enough studies to perform subgroup analysis.

14.

Forest plot of paired sensitivity and specificity estimated for studies assessing carotid artery occlusion with CTA as reference standard

15.

Summary ROC Plot of studies assessing carotid artery occlusion with CTA as reference standard

Das 2009 also compared DUS with CTA, with a correlation coefficient (r) between CTA and DUS of 0.84 (95% CI 0.69 to 0.92). However, as stated above, the study provided results of comparisons between tests in graphic format and data extraction was not possible. Therefore, we could not establish sensitivity and specificity, even in a narrative form.

Duplex ultrasound versus contrast‐enhanced magnetic resonance angiography

Two studies (102 carotid arteries) were included in analyses of DUS versus MRA (Borisch 2003; D'Onofrio 2006).

Carotid artery stenosis of 50% to 99%

Only D'Onofrio 2006 presented results for 50% to 99% carotid artery stenosis,with a sensitivity of 0.88 (95% CI 0.70 to 0.98) and a specificity of 0.60 (95% CI 0.15 to 0.95). The prevalence was 42%.

Carotid artery stenosis of 70% to 99%

Borisch 2003 and D'Onofrio 2006 provided data for 70% to 99% carotid artery stenosis. The sensitivity varied from 0.54 to 0.99 and specificity varied from 0.78 to 0.89 (Figure 16). The prevalence was 43% to 80%.

16.

Forest plot of paired sensitivity and specificity estimated for studies assessing 70‐99% carotid artery stenosis with MRA as reference standard

Borisch 2003 used DUS, MRA, and DSA to examine both carotid artery bifurcations from 39 consecutive participants (7 women and 32 men; age range 41 to 80 years; mean age, 67.4 ± 8.4 years) with clinically suspected symptomatic carotid artery stenoses that were referred for preoperative imaging. In this study, four radiologists evaluated the results, and the findings are presented with pooled data (4 observers × 71 vessels = 284 evaluations). We included the DSA results in the meta‐analysis since the authors reported the stenosis measurements from four radiologists (inflated the numbers in the 2 × 2 table); the results were divided by four to reflect the actual number of participants. But there were not enough studies included with MRA as the reference standard. For detecting 70% to 99% carotid stenosis, the reported sensitivity was 100% and specificity was 81.4%. Total agreement between MRA and DUS was achieved in 80% of evaluations (227 of 284). The results of other categories of stenosis are not presented.

D'Onofrio 2006 examined 21 participants with DUS, MRA, and DSA, including 41 carotid arteries in the analysis (1 participant had previous endarterectomy). The authors divided the carotid stenosis into four categories: < 39% (insignificant), 40% to 59% (borderline lesion), 60% to 79% (significant lesion), and 80% to 99% (very significant lesion). We chose to include in the analysis the results from 60% to 99% in our category of 50% to 99% carotid artery stenosis and the results from 80% to 99% in our category of 70% to 99% carotid artery stenosis because the velocities used for detecting these parameters are similar to those we propose (Appendix 9). The authors found poor agreement between DUS and MRA because DUS overestimated measurements in the lower stenosis categories while MRA overestimated the higher ones. For detecting 70% to 99% carotid artery stenosis, the reported sensitivity was 58.8% and the specificity was 88.8%. For detecting 50% to 99% carotid artery stenosis, the sensitivity was 88.5% and the specificity was 50%.

Das 2009 included 15 participants with symptomatic stenosis of the ICA. All participants underwent CTA, MRA, and DUS. The authors used the DEGUM criteria to classify carotid stenosis (Appendix 9). The correlation coefficient (r) between CTA and MRA was 0.83 (95% CI 0.68 to 0.92) and the correlation coefficient (r) between DUS and MRA was 0.83 (95% CI 0.66 to 0.91). The study provided results of comparisons between tests in graphic format and data extraction was not possible. Therefore, we could not establish sensitivity and specificity, even in a narrative form.

Discussion

Summary of main results

We aimed to assess the diagnostic accuracy of DUS for diagnosing carotid artery stenosis in symptomatic patients compared with DSA, MRA, or CTA as reference standards. The main results are shown in the Table 1. We included 22 studies (4957 carotid arteries). We did not include seven studies in our quantitative analysis, only describing them narratively (Chua 2007; Colquhoun 1992; Cui 2018; Das 2009; Bray 1995; Hansen 1996; Knudsen 2002). We included 15 studies in the quantitative analysis with results grouped into the categories we proposed in our protocol (Cassola 2018).

The following is a summary of the results for which statistical significance could be determined (we pre‐set the test consequence graphic as suggested by Whiting 2018):

For DUS versus DSA

• < 50% carotid artery stenosis (four studies, 1495 carotid arteries): The estimated summary sensitivity of DUS was 0.63 (95% CI: 0.48 to 0.76) and the estimated summary specificity was 0.99 (95% CI: 0.96 to 0.99). In a hypothetical cohort of 1000 patients (median prevalence of included studies 46%), 460 patients would have < 50% carotid artery stenosis. Of these, 291 (63%) would be correctly diagnosed and receive appropriate clinical treatment and 169 (27%) would receive unnecessary further investigation with another imaging method. Other 532 patients would receive appropriate further investigation, and eight would have no other tests performed and miss a chance for the right diagnosis and the possibility of carotid revascularization (Figure 17).

• 50% to 99% carotid artery stenosis (five studies, 1536 carotid arteries): The estimated summary sensitivity of DUS was 0.97 (95% CI 0.95 to 0.98) and the estimated summary specificity was 0.70 (95% CI 0.67 to 0.73). In a hypothetical cohort of 1000 patients (median prevalence of included studies 51%), 510 patients would have 50% to 99% carotid artery stenosis. Of these, 495 (97%) would receive appropriate further investigation with another imaging method, and 15 (3%) would not have any other tests performed and would miss a chance to receive the right diagnosis and the possibility of carotid revascularization. Overall, 147 would receive unnecessary further investigation with another imaging method, and 343 would receive no further investigation and appropriate clinical treatment (Figure 18);

• 70% to 99% carotid artery stenosis (nine studies, 2770 carotid arteries): The estimated summary sensitivity of DUS was 0.85 (95% CI 0.77 to 0.91) and the estimated summary specificity was 0.98 (95% CI 0.74 to 0.90). In a hypothetical cohort of 1000 patients (median prevalence of included studies 45%), 451 patients would have 70% to 99% carotid artery stenosis. Of these, 383 (85%) would receive appropriate carotid artery revascularization and 68 (15%) would miss or delay the chance to carotid revascularization. Another 8 would receive inappropriate carotid artery revascularization and 542 would receive appropriate clinical treatment. (Figure 19);

• occlusion (seven studies, 1212 carotid arteries): estimated summary sensitivity of DUS was 0.91 (95% CI: 0.81 to 0.97) and the estimated summary specificity was 0.95 (95% CI: 0.76 to 0.99), respectively. In a hypothetical cohort of 1000 patients (median prevalence of included studies was 18%), 180 will have carotid artery occlusion. Of these, 164 (91%) would receive appropriate clinical treatment. Another 41 would be false‐positive diagnosed with carotid occlusion and not have other tests performed, and miss a chance of the correct diagnosis and carotid revascularization. Other consequences would depend on the range of stenosis (Figure 20);

17.

DSA < 50%: Hypothetical cohort of 1000 symptomatic patients assessed for carotid artery stenosis. We considered pretest probability the median prevalence of the included studies (0.46). tp: true positive; fp: false positive; tn: true negative; fn: false negative

18.

DSA 50‐99%: Hypothetical cohort of 1000 symptomatic patients assessed for carotid artery stenosis. We considered pretest probability the median prevalence of the included studies (0.51). tp: true positive; fp: false positive; tn: true negative; fn: false negative

19.

DSA 70‐99%: Hypothetical cohort of 1000 symptomatic patients assessed for carotid artery stenosis. We considered pretest probability the median prevalence of the included studies (0.45). tp: true positive; fp: false positive; tn: true negative; fn: false negative

20.

DSA Occlusion: Hypothetical cohort of 1000 symptomatic patients assessed for carotid artery stenosis. We considered pretest probability the median prevalence of the included studies (0.18). tp: true positive; fp: false positive; tn: true negative; fn: false negative

For DUS versus CTA

• occlusion (three studies, 833 carotid arteries): The estimated summary sensitivity of DUS was 0.95 (95% CI 0.80 to 0.99) and the estimated summary specificity was 0.91 (95% CI 0.09 to 0.99). In a hypothetical cohort of 1000 patients (median prevalence of included studies was 60%), 600 patients would have carotid artery occlusion. Of these, 570 (95%) would receive appropriate clinical treatment. Another 41 would be false‐positive diagnosed with carotid occlusion and not have other tests performed, and miss a chance of the correct diagnosis and carotid revascularization. Other consequences would depend on the range of stenosis (Figure 21).

21.

CTA occlusion: Hypothetical cohort of 1000 symptomatic patients assessed for carotid artery stenosis. We considered pretest probability the median prevalence of the included studies (0.60). tp: true positive; fp: false positive; tn: true negative; fn: false negative

For DUS versus MRA, we found data in two categories, but there were not enough data for meta‐analysis.

Strengths and weaknesses of the review

The strength of this review is that we adhered to the recommended review methods and performed an extensive search of the literature without language restrictions. Therefore, we reviewed a large number of publications. We followed the standard recommendations of the Cochrane DTA (methods.cochrane.org/sdt/) and our previously published protocol (Cassola 2018) to avoid bias in the review process.

The use of velocity criteria with prespecified thresholds is both a strength and a weakness of this review. We used the most common validated criteria currently being used in most centers (Table 2). In 2011, the Society for Vascular Surgery® (SVS) and the European Society for Vascular Surgery in 2017 included the same criteria in their published guidelines (ESVS Writing Group 2018; Ricotta 2011). In 2014, the Intersocietal Accreditation Commission (IAC) endorsed the same criteria (IAC 2014), but in May 2021, it published some suggested changes for the ≥ 50% carotid artery stenosis criteria (Gornik 2021). Unfortunately, we had to exclude many studies because their thresholds were too different from those proposed in our protocol or did not describe thresholds. We believe that assessing the accuracy of DUS without a prespecified threshold would lead to unrealistic estimates of accuracy and even more heterogeneity among studies because the same velocity criteria can be used to classify carotid artery stenosis of 50% or 70% depending on the center performing DUS. Higher velocity criteria tend to decrease sensitivity and increase specificity. Therefore, different cut‐off velocity thresholds should achieve different estimates of sensitivity and specificity. We provided separate summary estimates of sensitivity and specificity for each proposed category of stenosis.

One significant limitation of our review concerns the issue of reproducibility. Most of the studies did not provide information regarding DUS operator experience, so we could not include any analysis on this characteristic. The diagnostic test interpretation is operator dependent, and multiple factors can affect the accuracy of the measurements, including the correct examination protocol and conditions inherent to the patients, such as hemodynamic factors and the presence of collateral flow through the circle of Willis or the ophthalmic artery.

Another issue is that we used DSA, MRA, or CTA as the reference standard. DSA is still considered the gold‐standard test for carotid artery stenosis. Still, in current practice, its use for diagnostic purposes has been largely supplanted by non‐invasive angiographic modalities (CTA, MRA). Thus, we decided to include CTA and MRA as reference standards. A diagnostic accuracy study should include all patients suspected of having the target condition and for whom the test would be considered (Leeflang 2009). The ideal population in this review would be all patients with neurologic symptoms (stroke, TIA, or sudden visual loss). We expected that recent studies comparing DUS and DSA would not include all symptomatic patients due to the invasive characteristic of DSA and the risk of complications. Therefore, we accepted CTA and MRA as reference tests to include studies in which all symptomatic patients were evaluated. It was frustrating to find little evidence comparing DUS to these less invasive diagnostic tests that are more frequently used in clinical practice. We also recognize that interobserver variation exists for all the reference tests, but with an acceptable agreement (Bucek 2007; Lenhart 2002; Saba 2008).

The estimates of a diagnostic accuracy test (i.e. sensitivity and specificity) are not fixed properties. They describe the behavior of a test under specific conditions, and they typically change at different segments of the disease spectrum and with varying disease prevalence (Leeflang 2012). Therefore, accuracy reviews should consider using the test in clinical practice when defining the population of interest to be studied. Methodological problems in patient inclusion criteria from the studies discussed above apparently influenced an overestimated estimate of prevalence values. For example, we found a median prevalence of 51% for 50% to 99% stenosis and 45% for 70% to 99% stenosis, well above what we see in clinical practice. It is difficult to quantify the effect of this on the systematic review results. The literature provides conflicting results of prevalence effects on DTA estimates (Leeflang 2009; Whiting 2004). In our review, the meta‐regression analyses adding prevalence as a covariate did not impact the accuracy estimates for carotid artery stenosis < 50% and 50% to 99%. However, there was increased specificity with higher prevalence in the 70% to 99% category. In the occlusion category, studies that included only patients with occlusion diagnosed by DUS had a significant impact, reducing the specificity estimates.

We found few studies from each category of stenosis. Many studies provided limited information about the mechanism of enrolling participants into the study. Many of them only included patients with known diseases, and many of them were retrospective. Because DUS is the recommended initial diagnostic test for assessing carotid stenosis, it is likely that some prospective studies and probably all retrospective studies focussed on participants who showed some degree of disease based on an initial assessment with DUS and who received further investigation with more invasive or expensive techniques. A sensitivity analysis would be possible if the included studies provided enough information from previous examinations that had been performed.

In 2006, a systematic review was carried out to compare noninvasive imaging in the diagnosis of symptomatic carotid (Wardlaw 2006a; Wardlaw 2006b) and already discussed the lack of quality evidence over carotid artery diagnostic methods and the need for well‐designed studies on this topic. One main difference was that our review evaluated DUS versus CTA and MRA also as gold standards (besides DSA). Sadly in our review, we found that many years later, we still have poor‐quality studies regarding the accuracy of noninvasive diagnostic imaging of the carotid artery.

The time we accepted between the index test and reference standard (four weeks) is another crucial factor in this review: we excluded some studies that exceeded this interval. We know that there is a progression of the plaque, especially the evolution of the patient's condition, with a risk of new neurologic events shortly after the first symptoms. Currently, it is already considered that the patient should be treated early after a cerebral ischemic event, ideally within an interval of up to 2 weeks, with a reduction in the relative risk with surgical treatment after this period (ESVS Writing Group 2018; Vasconcelos 2016). Considering that this review dealt with symptomatic patients, we believe that the interval of up to four weeks for the complete diagnostic investigation of the patient was a reasonable period.

Finally, it was impossible to perform meta‐analysis for all ranges of stenosis and all reference standards proposed due to the small number of studies contributing to this data.

Applicability of findings to the review question

The findings of this review apply to patients presenting neurologic symptoms and suspected carotid artery stenosis. However, the results cannot be considered definitive because of the small number of included studies in each stenosis category. Many of the included studies were at high or unclear risk of bias and there was heterogeneity among the studies. Using the QUADAS‐2 tool, many studies included in the primary analyses had limitations related to patient selection either because of unclear patient selection methods or the authors had selected only patients with known disease (previous test performed). We also have concerns regarding flow and timing: many studies did not include all patients in the analysis, or the patients did not receive the same reference standard.

The 'generation of technology' is not a surrogate for 'date of publication' because a recent publication can use an old DUS device. However, the year of publication can indirectly correlate with the 'generation of technology'. We could not assess the generation of technology for this review version, but we evaluated the implication of the year of publication with meta‐regression. Most of the included studies did not report the assessor's proficiency, and, therefore, we could not assess this evidence.

Authors' conclusions

Implications for practice.

Ultrasound is undoubtedly an exam with a crucial role in the diagnosis of patients with symptomatic carotid stenosis. Understanding its complexity and limitations helps better fit it into clinical practice and offer the most cost‐effective treatment to the patient. The findings of this review provide evidence that DUS is accurate at discriminating between the presence or absence of significant carotid artery stenosis (< 50% or 50% to 99%). Therefore, there is evidence to support the use of DUS as the first choice modality for the detection of carotid stenosis. Evidence suggests that no further imaging may be necessary to detect the presence of carotid artery stenosis in cases of DUS detecting > 50% carotid stenosis, given the high value of sensitivity for this category. Nonetheless, if the result is < 50% and clinical suspicion of carotid stenosis is high, another diagnostic test could add clinical information.

The results of this review indicate that DUS sensitivity and specificity for 70% to 99% carotid artery stenosis are high, but clinicians should exercise caution in using DUS as the single preoperative diagnostic method. It could be applicable, especially in centers that do not have immediate access to more sophisticated vascular imaging techniques, and the appropriate treatment time window would be lost. Our results showed that in a cohort of 1000 patients (with a high prevalence of carotid stenosis), 8 would receive inappropriate carotid revascularization treatment. When there is lower prevalence of the disease, this number can increase. Proceeding with additional diagnostic tests could improve the accuracy of the carotid stenosis diagnostic, however we could not assess the accuracy of the DUS as a confirmatory test after a first positive test.

The values of sensitivity and specificity for detecting occlusion of the carotid artery are high. However, the quality of the included studies is low and the consequences of a false‐positive result are severe, often leading the patient to miss the chance of carotid revascularization. Therefore, there appears to be a good case for a confirmatory test for these patients.

We found little evidence regarding the accuracy estimates of DUS versus MRA or CTA as reference standards.

The low methodological quality of the studies may reduce the reliability of the conclusion. There were many studies at high risk of bias, and most of them had concerns regarding their applicability, mainly due to the patient selection domain. Therefore, clinicians will have to decide whether additional imaging is necessary after DUS bearing in mind the time when this imaging is performed, and the potential benefits of performing a surgical treatment within a short time.

Implications for research.

Regardless of the positive findings of this review, more studies with high methodological quality of DUS accuracy would improve clinical decisions in patients with symptomatic carotid stenosis. In future studies, study selection criteria require careful attention: appropriate inclusion and exclusion criteria, and a standardized and replicable threshold to determine carotid stenosis. We recognize the challenge of performing a DTA study including all patients with neurologic symptoms. Frequently, smaller centers receive these symptomatic patients. There should be criteria for their referral to specialized centers where more invasive or expensive tests such as DSA, CTA, or MRA are available. Thus, future studies could consider assessing the accuracy of DUS as a confirmatory test in patients previously diagnosed with carotid stenosis based on initial tests.

Although DSA was considered the gold standard for diagnosing carotid artery stenosis, due to the risks related to the procedure, in current clinical practice it is usually reserved for select situations. This change in practice may be the main reason most studies fail to include patients who cover the entire spectrum of neurologic disease. Those patients receiving DSA usually have already been tested with a less invasive technique (usually DUS). Therefore, the validation of the accuracy of DUS using DSA as a reference test can be difficult or impossible because of ethical concerns. Future studies should also include comparisons of DUS versus CTA or MRA because these are the diagnostic tests performed in the clinical practice pathway (the 'new gold standards'). In particular, the criteria regarding patients with 50% to 69% carotid artery stenosis requires attention to determine the potentiality of using DUS to identify this situation accurately.

History

Protocol first published: Issue 11, 2018

Notes

Parts of the methods section of this protocol are based on a standard template established by Cochrane.

Appendices

Appendix 1. CENTRAL search strategy

IDSearchHits
#1MeSH descriptor: [Carotid Arteries] this term only682
#2MeSH descriptor: [Carotid Artery, Common] this term only207
#3MeSH descriptor: [Carotid Artery, External] this term only12
#4MeSH descriptor: [Carotid Artery, Internal] this term only205
#5{or #1‐#4}1067
#6MeSH descriptor: [Arteriosclerosis] this term only957
#7MeSH descriptor: [Atherosclerosis] this term only1094
#8MeSH descriptor: [Constriction, Pathologic] this term only640
#9{or #6‐#8}2667
#10#5 and #9200
#11MeSH descriptor: [Carotid Artery Diseases] this term only454
#12MeSH descriptor: [Carotid Artery Thrombosis] this term only18
#13MeSH descriptor: [Carotid Stenosis] this term only608
#14MeSH descriptor: [Carotid Artery Injuries] explode all trees20
#15((carotid near/5 (steno* or thrombo* or disease* or arter* or atherosclero* or atheroma* or narrow* or plaque* or occlus* or occlud* or constrict* or emboli* or block*))):ti,ab,kw4908
#16{or #10‐#15}4908
#17MeSH descriptor: [Ultrasonography] this term only4610
#18MeSH descriptor: [Ultrasonography, Doppler] this term only560
#19MeSH descriptor: [Ultrasonography, Doppler, Duplex] explode all trees875
#20MeSH descriptor: [Ultrasonography, Doppler, Pulsed] this term only67
#21(((duplex or color or colour or doppler) near/3 (ultrasound or ultrasonograph* or ultrasonic* or scan*))):ti,ab,kw (Word variations have been searched)4088
#22(((duplex or color or colour or doppler) near/3 (sonograph* or echograph* or echosound or echoscop* or echogram* or sonogram* or doptone))):ti,ab,kw (Word variations have been searched)880
#23((CDUS or DUS)):ti,ab,kw (Word variations have been searched)343
#24{OR #17‐#23}9262
#25#16 and #24812

Appendix 2. MEDLINE Ovid search strategy

1. carotid arteries/ or exp carotid artery, common/

2. arteriosclerosis/ or atherosclerosis/

3. constriction, pathologic/

4. 2 or 3

5. 1 and 4

6. carotid artery diseases/ or carotid artery thrombosis/ or carotid stenosis/

7. exp carotid artery injuries/

8. (carotid adj5 (steno$ or thrombo$ or disease$ or arter$ or atherosclero$ or atheroma$ or narrow$ or plaque$ or occlus$ or occlud$ or constrict$ or emboli$ or block$)).tw.

9. 5 or 6 or 7 or 8

10. Ultrasonography/

11. ultrasonography, doppler/ or ultrasonography, doppler, duplex/ or ultrasonography, doppler, color/ or ultrasonography, doppler, pulsed/

12. (duplex or color or doppler).tw.

13. (ultrasound or ultrasonograph$ or ultrasonic$ or scan$).tw.

14. (sonograph$ or echograph$ or echosound or echoscop$ or echogram$ or sonogram$ or doptone).tw.

15. (CDUS or DUS).tw.

16. 10 or 11 or 12 or 14 or 15

17. 9 and 16

Appendix 3. Embase Ovid search strategy

1. carotid artery/ or exp common carotid artery/ or external carotid artery/
2. arteriosclerosis/ or arteriolosclerosis/ or atherosclerosis/ or atheroma/ or atheromatosis/ or atherosclerotic plaque/ or brain atherosclerosis/ or carotid atherosclerosis/
3. ligation/
4. 2 or 3
5. 1 and 4
6. exp carotid artery obstruction/ or carotid artery disease/
7. carotid artery injury/ or carotid atherosclerosis/
8. (carotid adj5 (steno$ or thrombo$ or disease$ or arter$ or atherosclero$ or atheroma$ or narrow$ or plaque$ or occlus$ or occlud$ or constrict$ or emboli$ or block$)).tw.
9. or/5‐8
10. contrast‐enhanced ultrasound/ or interventional ultrasonography/
11. doppler flowmetry/ or color doppler flowmetry/
12. doppler ultrasonography/ or duplex doppler ultrasonography/ or pulsed doppler ultrasonography/
13. ((duplex or colo?r or doppler) adj3 (ultrasound or ultrasonograph$ or ultrasonic$ or scan$)).tw.
14. ((duplex or colo?r or doppler) adj3 (sonograph$ or echograph$ or echosound or echoscop$ or echogram$ or sonogram$ or doptone)).tw.
15. (CDUS or DUS).tw.
16. or/10‐15
17. 9 and 16
18. (exp animal/ or animal.hw. or nonhuman/) not (exp human/ or human cell/ or (human or humans).ti.)
19. 17 not 18

Appendix 4. ISI Web of Science search strategy

# 5
#4 AND #1
Indexes=SCI‐EXPANDED, CPCI‐S Timespan=1900‐2019
# 4
#3 OR #2
Indexes=SCI‐EXPANDED, CPCI‐S Timespan=1900‐2019
# 3
TS=((duplex or color or colour or doppler) NEAR/3 (sonograph* or echograph* or echosound or echoscop* or echogram* or sonogram* or doptone))
Indexes=SCI‐EXPANDED, CPCI‐S Timespan=1900‐2019
# 2
TS=((duplex or color or colour or doppler) NEAR/3 (ultrasound or ultrasonograph* or ultrasonic* or scan*))
Indexes=SCI‐EXPANDED, CPCI‐S Timespan=1900‐2019
# 1
TS=(carotid NEAR/5 (steno* or thrombo* or disease* or arter* or atherosclero* or atheroma* or narrow* or plaque* or occlus* or occlud* or constrict* or emboli* or block*))
Indexes=SCI‐EXPANDED, CPCI‐S Timespan=1900‐2019

Appendix 5. HTA and DARE search strategy

1MeSH DESCRIPTOR Carotid Arteries IN DARE,HTA40
2MeSH DESCRIPTOR Carotid Artery, Common EXPLODE ALL TREES IN DARE,HTA17
3MeSH DESCRIPTOR Carotid Artery Diseases EXPLODE ALL TREES199
4MeSH DESCRIPTOR Carotid Artery Injuries EXPLODE ALL TREES7
5(carotid):TI AND (steno* or thrombo* or disease* or arter* or atherosclero* or atheroma* or narrow* or plaque* or occlus* or occlud* or constrict* or emboli* or block*):TI154
6#1 OR #2 OR #3 OR #4 OR #5241
7MeSH DESCRIPTOR Ultrasonography IN DARE,HTA154
8MeSH DESCRIPTOR Ultrasonography, Doppler IN DARE,HTA56
9MeSH DESCRIPTOR Ultrasonography, Doppler, Duplex EXPLODE ALL TREES IN DARE,HTA41
10MeSH DESCRIPTOR Ultrasonography, Doppler, Pulsed IN DARE,HTA1
11(duplex or colour or color or doppler):TI AND (ultrasound or ultrasonograph* or ultrasonic* or scan*):TI OR (sonograph* or echograph* or echosound or echoscop$ or echogram* or sonogram* or doptone):TI52
12#7 OR #8 OR #9 OR #10 OR #11278
13#6 AND #1214

Appendix 6. LILACS and IBECS search strategy

((mh: (carotid arteries) OR mh: (arterias carótidas) OR mh: (artérias carótidas) OR (arter*
carotid)) OR (mh: (carotid artery diseases) OR mh: (enfermedades de las arterias carótidas)
OR mh: (doenças das artérias carótidas) OR (arter* disease* carotid) OR (arter* disease*
common carotid) OR (arter* disease* external carotid) OR (arter* disease* internal carotid)
OR (atherosclerotic disease* carotid) OR (carotid arter* disorder*) OR (carotid
atherosclerotic disease*) OR (carotid atheroscleros*) OR (aterosclerosis de la carótida) OR
(aterosclerose carotídea) OR (aterosclerose da carótida) OR (c10.228.140.300.200*) OR
(c14.907.253.123*)) OR (mh: (constriction, pathologic) OR mh: (constricción patológica)
OR mh: (constrição patológica) OR (constriction* pathologic*) OR (stenos*) OR (stricture*)
OR (estenos*) OR (estrechamiento patológico) OR (estreitamento patológico)) OR (mh:
(carotid artery injuries) OR mh: (traumatismos de las arterias carótidas) OR mh: (lesões das
artérias carótidas) OR (carotid arteriopath* traumatic) OR (carotid false aneurysm*) OR
(injur* carotid artery) OR (artery trauma carotid) OR (pseudoaneurysm carotid) OR
(seudoaneurisma de la carótida) OR (pseudoaneurisma de la carótida) OR (lesiones de las
arterias carótidas) OR (pseudoaneurisma carotídeo) OR (traumatismos das artérias carótidas)
OR (c10.228.140.300.200.345*) OR (c10.228.140.300.350.500*) OR (c10.900.250.300*)
OR (c14.907.253.123.345*) OR (c14.907.253.535.500*) OR (c26.915.200.200*)) OR (mh:
(atherosclerosis) OR mh: (aterosclerosis) OR mh: (aterosclerose) OR (atheroscleroses) OR
(atherogenesis) OR (aterosclerosis de la carótida) OR (aterosclerose carotídea) OR
(aterosclerose da carótida)) OR (mh: (carotid artery thrombosis) OR mh: (trombosis de las
arterias carótidas) OR mh: (trombose das artérias carótidas) OR (carotid thrombosis) OR
(common carotid artery thrombosis) OR (external carotid artery thrombosis) OR (internal
carotid artery thrombosis) OR (carotid arter* thrombos*) OR (trombosis arterial carotídea)
OR (c10.228.140.300.200.355*) OR (c14.907.253.123.355*) OR (c14.907.253.566.206*)
OR (c14.907.355.590.213.206*))) AND ((mh: (ultrasonography) OR mh: (ultrasonografía)
OR mh: (ultrassonografia) OR (computer echotomography) OR (diagnos* ultrasonic) OR
(diagnos* ultrasound*) OR (ultrasonic tomography) OR (ultrasound imaging*) OR (imaging
ultrasonic) OR (medical sonography) OR (echography) OR (echotomography computer) OR
(sonography medical) OR (echotomography) OR (ecografía) OR (ecotomografía por
computador) OR (sonografía médica) OR (ecografía médica) OR (tomografía ultrasonica)
OR (diagnóstico por ultrasonido) OR (imagen ultrasónica) OR (imagen ultrasonográfica) OR
(imagen de ultrasonido) OR (imagen por ultrasonido) OR (ecotomografía) OR (ecografia)
OR (ecotomografia por computador) OR (sonografia médica) OR (ecografia médica) OR
(tomografia ultrassônica) OR (diagnóstico por ultrassom) OR (imagem ultrassônica) OR
(imagem ultrassonográfica) OR (imagem de ultrassom) OR (imagem por ultrassom) OR
(ecotomografia)) OR (mh: (ultrasonography, doppler, duplex) OR mh: (ultrasonografía
doppler dúplex) OR mh: (ultrassonografia doppler dupla) OR (doppler duplex
ultrasonography) OR (ultrasonografía doppler doble) OR (ultrasonografía dúplex‐doppler))
OR (mh: (ultrasonography, doppler, color) OR mh: (ultrasonografía doppler en color) OR
mh: (ultrassonografia doppler em cores) OR (color doppler ultrasonography)) OR (mh:
(diagnostic imaging) OR mh: (diagnóstico por imagen) OR mh: (diagnóstico por imagem)
OR (imaging diagnostic) OR (imaging medical) OR (imagen clínica) OR (diagnóstico por
imageamento) OR (imageamento diagnóstico) OR (imageamento clínico) OR (imageamento
médico) OR (imagens clínicas) OR (imageologia clínica) OR (imageologia médica) OR
(imagiologia clínica) OR (imagiologia médica) OR (radiodiagnóstico) OR (e01.370.350*)
OR (vs3.003.001.006.005.001*))) AND (instance:&quot;regional&quot;) AND (db:(&quot;LILACS&quot; OR
&quot;IBECS&quot;))

Appendix 7. ClinicalTrials.gov search strategy

( Ultrasound OR ultrasonography ) AND ( carotid artery OR carotid stenosis ) [DISEASE]

Appendix 8. Studies not included in a meta‐analysis

Chua 2007 included 114 symptomatic patients that had undergone DUS and DSA. The objective was to evaluate optimal ultrasonographic criteria for the determination of ICA stenosis of more than or equal to 50%, 60%, and 70%. The authors evaluated the following velocity criteria: ICA PSV, CCA PSV, ICA EDV, CCA EDV, the ICA/CCA PSV ratio, the ICA PSV/CCA EDV ratio, and the ICA/CCA EDV ratio. They found that the ICA/CCA PSV ratio and the ICA PSV/CCA EDV ratio performed superiorly to the other velocity criteria. The data provided are from the area under the ROC curves; the sensitivity and specificity from each criterion were not provided. The ICA/CCA RSV ratio results were for ≥ 50% (ratio of 1.5) sensitivity, specificity, and accuracy: 100%, 85%, and 93%, respectively. For ≥ 60% (ratio of 2.6), sensitivity, specificity, and accuracy were 100%, 94%, and 97%, respectively. For ≥ 70% (ratio of 3.1), sensitivity, specificity, and accuracy were 100%, 91%, and 95%, respectively. The ICA PSV/CCA EDV ratio is rarely used and there are few data in the literature. The authors found the following: for ≥ 50%, a ratio of 3.5 has sensitivity, specificity, and accuracy of 100%, 58%, and 93%, respectively. For both ≥ 60% and ≥ 70%, the same ratio of 10.3 achieved 100% sensitivity, 96% specificity, and 91% accuracy. We were unable to include this study in our analysis because the data from PSV criteria were not provided and the ICA/CCA PSV ratio was too different from our prespecified criteria.

Colquhoun 1992 included 52 patients (99 carotid arteries) referred for DSA by their clinicians. The authors compared DUS versus DSA in different sites of the carotid artery (six segments) – proximal CCA, distal CCA, bulb, proximal ICA, distal ICA, and ECA – and divided stenosis into six grades (no disease, 0% to 24%, 25% to 49%, 50% to 74%, 75% to 99%, and occlusion). They presented the results in tables comparing grades of stenosis for each of the different segments separately. The agreement in grading for each vessel segment was 74.5%, and the sensitivity and specificity could be calculated for each segment separately. In the proximal ICA (segment 4), the estimated sensitivity and specificity for ≥ 50% and ≥ 75% ICA stenosis were 100% and 92%, respectively. This study was not included in the analysis because the authors classified carotid stenosis according to ECST –"DSA measurement of stenosis: the degree of stenosis for each area was assessed using calipers to measure the width of the lumen at the site of maximal stenosis and expressing this as a percentage of the true lumen, predicted by extrapolation from above and below the stenosis". Moreover, we could not convert the ranges of stenosis from the study to the NASCET classification. Some authors believe that because the results from measurements in NASCET and ECST models are approximately linear (Rothwell 1994), the conversion could be made. Because the results are given in ranges and the literature differs on the best mathematical model (Eliasziw 1994a), we were unable to perform such a conversion for this study.

Cui 2018 included 54 patients and examined four vessels – the bilateral CCA and the bilateral ICA – with DUS and DSA. Thus, the results were presented pooled (216 vessels). It was not possible to separate the CCA and ICA results, and therefore we did not include the results from this study in the analysis. The diagnostic accuracy of DUS found in this study, based on the diagnostic results of DSA, was 74.1% (160/216 vessels). For 1% to 49% stenosis, the sensitivity and specificity were 78.2% and 79.1%, respectively. For 50% to 69% stenosis, the sensitivity and specificity were 50% and 93.4%, respectively. For 70% to 99% stenosis, the sensitivity and specificity were 100% and 98.1%, respectively.

Bray 1995 included 64 patients with cerebrovascular disease referred to DSA who had undergone DUS in the 24h prior to DSA. Patients were submitted both to standard duplex sonography and colour Doppler sonographic imaging (CDI); all images were examined separately by two randomly selected radiologists. The authors found high concordance between radiologists in DSA evaluation. This study also assessed stenosis by diameter and area and compared the two ultrasonic methods. The results were the same in carotid artery stenoses > 70%; for the other grades, CDI performed better than standard duplex sonography. For combined hemodynamic (velocity criteria) and morphological data on carotid stenosis, no significant differences were found between techniques. For ≥ 70% carotid artery stenosis, sensitivity and specificity were 85% and 97%, respectively, for standard duplex sonography, and 85% and 96%, respectively, for CDI. In minor stenoses, the accuracy was slightly lower: 83% and 84% for CDI and standard duplex sonography, respectively. The velocity criteria described are very similar to those we prespecified in our protocol, but the data presented were insufficient to complete a 2 × 2 table for any category.

Hansen 1996 included 81 patients (162 arteries) that were examined with DUS and DSA. Stenosis on DUS was graded from the peak systolic velocity according to the equation: y = 0.54 * e ^{0.021 * x} , where y is the peak systolic velocity in the ICA in m/s and x is the degree of stenosis expressed as the diameter reduction in percentage. In addition, a new equation was created for comparison with this original one. Carotid stenosis was determined by using DUS: diameter reduction in percent = [{(b × a) / b } × 100], where a is the smallest diameter in the stenotic zone and b is the diameter of the normal CCA proximal to the stenosis. The authors presented the comparison of DUS velocity criteria and DSA graphically; hence, we were unable to extract data (too many overlapping points). The authors discussed whether DSA is necessary for preoperative evaluation. They found that DSA was recommended in 14 of the 162 arteries examined (8.5%): in 11 of these arteries, DUS showed occlusion of the ICA (DSA confirmed occlusion in 10), and for the other three arteries the reason was poor DUS quality. The authors concluded that DSA did not change management in any of the patients. Although this study fulfilled the participant inclusion criteria, the methods used to determine stenosis based on DUS and DSA were too different from those prespecified in our protocol for quantitative analysis. Moreover, the authors did not provide sufficient data to establish values of sensitivity and specificity, even in a narrative form.

Knudsen 2002 is a short report that included 65 patients with suspected symptomatic high‐grade ICA (unclear how these patients were recruited); all patients underwent DUS and DSA. The authors found an overall agreement of 88%; for detecting ≥ 70% carotid artery stenosis, the reported sensitivity was 94% and the specificity was 86%. For detecting < 70% carotid artery stenosis, the calculated sensitivity was 86.6% and the specificity was 97.1%. We did not include this study in the meta‐analysis because ≥ 70% carotid artery stenosis was characterised by an ICA PSV ≥ 150 cm/s, an ICA EDV ≥ 90 cm/s, and a ICA/CCA PSV ratio > 2.8. These values are too different from those prespecified in our protocol.

Appendix 9. Carotid stenosis threshold criteria from included studies

Criteria used by Bray 1995

Standard duplex		Stenosis %	Colour Doppler
Systolic velocity cm/s	Spectral analysis
> 250	± Negative low frequencies of high energy	70‐100	Marked colour fading. Severe lumen narrowing. ± post‐stenosis flow reversal. ± mosaic pattern
> 130	± Low frequency of high energy	50‐69	Moderate color fading. Moderate lumen narrowing. ± mosaic pattern
110‐130	± Minimal spectral broadening	30‐49	Laminar flow or color fading during systole
< 110	Normal spectrum	0‐29	Laminar flow. Red flow in systole and diastole

Criteria used by Eliasziw 1995

Stenosis category (%)	ICPSV (v) cm/s	ICPFC (f) kHz	Ratio (r) ICA/CCA
< 70	< 250	< 8	< 3
70‐79	250 ≤ v < 375	8 ≤ f < 12	3 ≤ r < 4,5
80‐89	375 ≤ v < 500	12 ≤ f < 16	4,5 ≤ r < 6
90‐99	≥ 500	≥ 16	≥ 6

Footnotes

CCA: common carotid artery
f: frequency
ICA: internal carotid artery
ICPFC: internal carotid peak frequency change
ICPSV: internal carotid peak systolic velocity
r: ratio
v: velocity

Criteria used by Arbeille 1995

Diameter stenosis category	Peak systolic velocity	End diastolic velocity	Systolic velocity ratio (ICA/CCA)	Diastolic velocity ratio (ICA/CCA)
0%	110 cm/s	40 cm/s	1.8	2.6
1%‐39%	110 cm/s	40 cm/s	1.8	2.6
40%‐59%	130 cm/s	40 cm/s	1.8	2.6
60%‐79%	130 cm/s	40 cm/s	1.8	2.6
80%‐99%	250 cm/s	100 cm/s	2.5	5.5
100%	N/A	N/A	N/A	N/A

Footnotes

CCA: common carotid artery
ICA: internal carotid artery
N/A: not applicable

Criteria adapted and translated from Arning 2010

Defined by NASCET (% +‐ 5%)	10%	20‐40%	50%	60%	70%	80%	90%	occlusion
Defined by ECST (% +‐ 5%)	45%	50‐60%	70%	75%	80%	90%	95%	occlusion
Main criteria (1‐5)
1. B‐mode image	+++	+
2. Color Duplex image	+	+++	+	+	+	+	+	+++
3. Intrastenotic PSV (cm/s)			~200	~250	~300	350‐400	100‐500	‐
4. Post‐stenotic PSV (cm/s)					> 50	< 50	< 30	‐
5. Collaterals and precursors (periorbital arteries/ACA)					(+)	++	+++	+++
Supplementary criteria (6‐10)
6. Pre‐stenotic EDV reduction					(+)	++	+++	+++
7. Post‐stenotic flow disturbances			+	+	++	+++	(+)
8. Intrastenotic EDV (cm/s)			< 100	< 100	> 100	> 100
9. Perivascular tissue vibration				(+)	++	++
10. PSV ratio (ICA/CCA)				≥ 2	> 2	≥ 4	≥ 4

Footnotes

Criteria 1: non‐stenosing plaques (up to 10% according to NASCET) will be shown in the B‐image, documentation of width, length and morphology of vessel wall changes
Criteria 2: evidence of the minor stenosis (local aliasing effect) in contrast to the non‐stenosing plaque, illustration of the direction of flow in the case of moderate and severe stenoses and evidence of vascular occlusion
Criteria 3: apply to stenoses with a length of 1‐2 cm and only limited in multi‐vessel processes
Criteria 4: measurement at the most distal extracranial position outside the zone with jet stream and flow disturbances
Criteria 5: possibly only one of the collateral connections affected; if an extracranial examination is carried out alone, the value of the findings is lower and requires careful interpretation because of frequent anatomical variants
Criteria 6: indirect stenosis criteria, in the case of high‐grade, hemodynamically relevant stenosis (≥ 70% according to NASCET), the flow volume decreases; this leads to a decrease in the flow velocity if the vessel cross‐section remains constant. Diastolic reduction of the flow velocity (increased pulsatility) recognizable, while the systolic flow velocity still remains the same
Criteria 7: not always pathological; should only be considered relevant for stenosis diagnosis together with other stenosis criteria
Criteria 8: when PSV cannot be measured with sufficient accuracy, EDV maximum flow velocity can be used here as an additional criterion
Criteria 9: the 'confetti' symbol is only recognizable when the PRF is set low
Criteria 10: is useful for the assessment of carotid artery tandem stenosis, hyperfusion and primary (constitutional) narrow vessels

ACA: anterior cerebral artery
CCA: common carotid artery
DEGUM: Deutsche Gesellshaft für Ultraschall in der Medizin
ECST: European Carotid Surgery Trial
EDV: end diastolic velocity
ICA: internal carotid artery
NASCET: North American Symptomatic Carotid Endaterectomy Trial
PSV: peak systolic velocity

Data

Presented below are all the data for all of the tests entered into the review.

Tests. Data tables by test.

Test	No. of studies	No. of participants
1 DSA < 50%	4	1495
2 DSA 50‐99%	5	1536
3 DSA 50‐69%	1	313
4 DSA 70‐99%	9	2708
5 DSA Occlusion	8	1243
6 CTA 70%‐99%	2	685
7 CTA Occlusion	3	833
8 MRA 70‐99%	2	102
9 MRA 50‐99%	1	31

1. Test.

DSA < 50%

2. Test.

DSA 50‐99%

3. Test.

DSA 50‐69%

4. Test.

DSA 70‐99%

5. Test.

DSA Occlusion

6. Test.

CTA 70%‐99%

7. Test.

CTA Occlusion

8. Test.

MRA 70‐99%

9. Test.

MRA 50‐99%

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Anzidei 2012.

Study characteristics
Patient Sampling	416 symptomatic patients (amaurosis fugax, stroke, transient ischemic attack (TIA) or reversible ischemic neurological deficit) and suspected carotid artery stenosis underwent DUS. Only patients with stenoses on DUS > 30% with irregular atheroma, were sent for combined evaluation with CTA and MRA and only those with an indication for treatment after theses studies underwent DSA Exclusion criteria: Patients that had contraindications to MR, CT or DSA or had already undergone surgical or endovascular treatment for carotid stenosis
Patient characteristics and setting	170 patients (included in analysis), 108 male/62 female, mean age 69 ± 6.5 (range 62‐90 years) risk factors: 17 diabetes mellitus, 34 hypertension, 13 dyslipidemia, 18 current smoker, 11 former smoker Only patients with known disease
Index tests	DUS Image production: Aplio XV or Mylab 70 with dedicated software for the vascular study and a 5‐12 MHz multiband linear transducer Contrast: No Criteria used to determine grade of stenosis: quoted "The degree of stenosis was determined. Measure of the residual lumen at the point of maximum narrowing and peak velocity (125–130 cm/s) were visualized at the level of the stenosis."
Target condition and reference standard(s)	Reference standard: DSA Target condition: symptomatic carotid stenoses Criteria used to determine grade of stenosis: NASCET criteria Complications: There were 11 cases (6.5%) of complications following the DSA procedure (one cerebral ischemia, four pseudoaneurysms and six hematomas at the puncture site); eight (4%) patients suffered moderate‐to‐severe adverse reactions to the iodinated contrast agent.
Flow and timing	416 with symptomatic carotid stenoses underwent DUS. 205 patients underwent CTA and MRA and 170 patients underwent DSA (only those that were treated). In two cases, MRA examination was not considered of diagnostic quality due to motion artefacts. Interval for performing all four techniques was 7 ± 3 days.
Comparative
Methods	Study design: Prospective accuracy cohort study. Unclear whether consecutive recruitment Study location: Italy Year and language of publication: Published in 2011 in English and Italian Study period: May 2006 and May 2010 Participants enrolled: 416 patients underwent DUS; 205 patients underwent CTA and MRA; 170 patients underwent DSA (underwent treatment). Carotids included in analyses: 335
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Barlinn 2016.

Study characteristics
Patient Sampling	Retrospectively evaluated 346 consecutive patients with acute cerebral ischemia who were admitted to a tertiary stroke center. Patients were eligible if their diagnostic workup included DUS and CTA performed within 5 days of each other Exclusion criteria: quote "Patients who underwent acute revascularization therapy of the extracranial ICA prior to completion of both diagnostic studies were excluded from our analysis".
Patient characteristics and setting	All patients were symptomatic, 346 patients with acute Ischemic stroke (n = 284) or transient ischemic attack (n = 62) 303 acute cerebral Ischemic patients included in analyses Mean age, 72 ± 12 years. 58% men and 42% women; median baseline National Institutes of Health Stroke Scale score, 4 [IQR 7] No information on risk factor
Index tests	DUS . Image production: Duplex ultrasonography (Toshiba Aplio MX SSA‐780a System®, Toshiba Medical Systems, Germany) with a 7.5–10‐MHz linear array transducer was used for examinations of the extracranial carotid arteries. Contrast: No Criteria used to determine grade of stenosis: DEGUM ultrasound criteria (Appendix 9Arning 2010)
Target condition and reference standard(s)	Reference standard: CTA Target condition: assessment of extracranial ICA steno‐occlusive disease in patients with acute cerebral ischemia Criteria used to determine grade of stenosis: NASCET criteria Complications: Not described
Flow and timing	43 patients (12%) were not eligible for the final analysis due to the following reasons: ultrasonographic assessment after acute revascularization therapy, n = 13; elapsed time between DUS and CTA > 5 days, n = 23; and only one vascular imaging modality assessable (e.g. streak artefacts from dental implants on CTA), n = 7. The median elapsed time between DUS and CTA was 1 (IQR, 2) day
Comparative
Methods	Study design: Retrospective design; participants identified retrospectively based on availability of complete records, if their diagnostic workup included DUS and CTA performed within 5 days Study location: Germany Year and language of publication: Published in 2016 in English Study period: from January 2012 to December 2012 Participants enrolled: 303 patients Carotids included in analyses: 593 DUS and CTA carotid artery pairs available for comparison
Notes	Only included retrospectively patients that had undergone the index and the reference tests; unclear if DUS results were used to select patients to CTA
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Belsky 2000.

Study characteristics
Patient Sampling	46 randomly chosen patients with symptoms of transient hemispheric attacks or unilateral visual disturbances were enrolled. No further details of patient sampling and recruitment were reported. The patients initially underwent duplex sonography due to suspected cerebral vascular disease. Patients who were candidates for carotid endarterectomy of either one or both the internal carotid arteries were then sent for CTA. Exclusion criteria: Not described
Patient characteristics and setting	All patients were symptomatic. 92 internal carotid arteries from 46 patients. 30 men and 16 women, ranging from 16 to 80 years of age (median, 70 years) No other patient characteristics were described.
Index tests	DUS Image production: Imaging was conducted by color DUS with either a 7‐MHz linear array transducer (Acuson 128, Acuson, Mountain View CA) or a broadband 5–12 MHz linear array transducer (ATL 3000 HDI, Advanced Technology Laboratories, Bothell WA) Contrast: No Criteria used to determine grade of stenosis: The degree of stenosis found on duplex sonography was categorized as mild, moderate and severe, according to the peak systolic and end diastolic velocities (PSV and EDV) on the internal carotid artery, measured in cms; mild stenosis, 0–29%: PSV < 125 and EDV < 40; moderate stenosis: PSV ≥ 125 and EDV ≥ 40; severe stenosis, 70–99%: PSV ≥ 250 and EDV ≥ 100. Occlusion was determined when flow through the internal carotid artery could not be registered by color DUS.
Target condition and reference standard(s)	Reference standard: CTA Target condition: assessment of extracranial ICA steno‐occlusive disease in patients with acute cerebral ischemia Criteria used to determine grade of stenosis: NASCET criteria Complications: Not described
Flow and timing	Only patients with both tests (index and reference standard) were included. There were no exclusions described. CTA examinations were performed up to 1 month post‐duplex ultrasound.
Comparative
Methods	Study design: Retrospective design; participants identified retrospectively based on availability of complete records Study location: Israel Year and language of publication: Published in 2000 in English Study period: between January 1996 and December 1998 Participants enrolled: 46 patients Carotids included in analyses: 92
Notes	Only patients who were candidates for carotid endarterectomy of either one or both the internal carotid arteries were then sent for CTA.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Unclear
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

Borisch 2003.

Study characteristics
Patient Sampling	39 consecutive patients with clinically suspected symptomatic carotid artery stenoses, referred to our institution for preoperative imaging, were included. Exclusion criteria: Patients with known contraindications for contrast‐enhanced MR angiography or DSA
Patient characteristics and setting	All patients were symptomatic: amaurosis fugax (n = 10), single or recurrent transient Ischemic attack (n = 14), and stroke in the previous 8 weeks (n = 15) 39 patients included. 7 women and 32 men; age range, 41–80 years; mean age, 67.4 ± 8.4 years) Risk factors not described
Index tests	DUS Image production: Sonoline Elegra 5.0 system (Siemens), with a 7.5‐MHz linear array transducer Contrast: No Criteria used to determine grade of stenosis: previously published criteria described in Eliasziw 1995; Appendix 9
Target condition and reference standard(s)	Reference standard: DSA and MRA Target condition: assessment of clinically suspected symptomatic carotid artery stenoses Criteria used to determine grade of stenosis: NASCET criteria Complications: No neurologic events occurred
Flow and timing	Both carotid artery bifurcations were examined with contrast‐enhanced MR angiography, duplex sonography, and selective DSA within 10 days. 7 carotids were excluded from statistical analysis: in 3 patients, the carotid artery could not be visualized selectively on DSA images; in 4 patients, the insonation of the stenosis was not possible because of sonographic attenuation due to echogenic plaque in the artery wall.
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: Germany Year and language of publication: Published in 2003 in English Study period: August 1999 and July 2002 Participants enrolled: 39 Carotids included in analyses: 71
Notes	The data presented refer to numbers that are pooled data (4 observers × 71 vessels = 284 evaluations)
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	No
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Unclear
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Bray 1995.

Study characteristics
Patient Sampling	Patientes referred to the radiology department for DSA of the supra‐aortic vessels were consecutively included. Exclusion criteria not described
Patient characteristics and setting	64 patients, 53 men and 11 women, mean age 62.6 +‐ 13.5 years 28 had an established stroke, 19 had only transient Ischemic attacks (TIA). There were 12 with a cervical bruit, and 5 had a cerebral hemorrhage. Risk factors not described
Index tests	DUS Image production: The device has not been specified but performed with a 7 MHz linear probe. Contrast: No Criteria used to determine grade of stenosis: Appendix 9
Target condition and reference standard(s)	Reference standard: DSA Target condition: suspected for carotid disease patients Criteria used to determine grade of stenosis: quantitative measurement of the ratio between the diameters of the residual lumen of the stenosis and the presumably normal carotid artery was made, usually 4 cm beyond the carotid bulb. Complications: Not described
Flow and timing	7 other patients were excluded from the consecutive series because of incomplete or delayed investigations. DUS were performed in the 24 h prior to angiography. Did not describe prior testing, but included only patients referred to DSA of supra‐aortic vessels
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: France Year and language of publication: Published in 1995 in English. Study period: Not described Participants enrolled: 64 patients. Carotids included in analyses: 128
Notes	Did not describe whether patients had undergone any screening imaging test before being included in the study
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Chua 2007.

Study characteristics
Patient Sampling	Evaluated 114 patients who presented within 120 days of the onset of ischemic symptoms (transient ischemic attack or non‐disabling stroke), 20 were excluded Exclusion criteria: Patients with cardiac embolism and prior ipsilateral carotid endarterectomy were excluded. Occlusion of one or both ICA and atypical flow patterns within vessels, such as low velocities in near‐occlusion, and extensive calcified plaques resulting in long segments of acoustic shadowing
Patient characteristics and setting	94 patients ranged from 53 to 76 years (mean, 64; standard deviation, 8.8). The male‐to‐female sex ratio was 2.9:1 No information on risk factors
Index tests	DUS Image production: Diasonics Spectra (Diasonics Inc, Milpatas, California) using a 7.5‐MHz transducer Contrast: No Criteria used to determine grade of stenosis: ROC curve analysis
Target condition and reference standard(s)	Reference standard: DSA Target condition: evaluate optimal criteria for determination of ICA stenosis in symptomatic patients Criteria used to determine grade of stenosis: NASCET criteria Complications: None
Flow and timing	20 were excluded from the study because of the occlusion of one or both ICA and atypical flow patterns within vessels, such as low velocities in near‐occlusion, and extensive calcified plaques resulting in long segments of acoustic shadowing. Carotid duplex ultrasonography and DSA within 1 month of each other
Comparative
Methods	Study design: Prospective, not consecutive, accuracy cohort study Study location: Singapore Year and language of publication: Published in 2007 in English Study period: January 1995 to December 2003 Participants enrolled: 94 Carotids included in analyses: 188
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	No
If a threshold was used, was it pre‐specified?	No
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	No
Could the reference standard, its conduct, or its interpretation have introduced bias?		High risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Colquhoun 1992.

Study characteristics
Patient Sampling	Patients included were referred by their clinicians for DSA examination of the carotid arteries, and then the ultrasound examinations were performed. Did not mention if previous tests were performed before inclusion
Patient characteristics and setting	42 (84%) of patients included were symptomatic: 27 were referred with transient ischemic attacks, two with amaurosis fugax, 15 with stroke and six with non‐specific complaints. 50 patients, 30 men, mean age 58 years (44‐70) and 20 women, mean age 53 years (38‐68) Risk factors not described Included patients were referred for DSA, but previous testing was not described.
Index tests	DUS Image production: Acuson 128 duplex scanner with color flow mapping using a 5 MHz small parts linear array probe (L538) Contrast: No Criteria used to determine grade of stenosis: When possible, the percentage diameter stenosis was measured directly from the image with electronic caliper. Peak systolic velocity of 120 cm/s in the internal carotid artery was taken to indicate > 50% diameter stenosis, and > 250 cm/s with diastolic velocities > 100 cm/s was taken to indicate > 80% diameter stenosis.
Target condition and reference standard(s)	Reference standard: DSA Target condition: suspected for carotid disease patients Criteria used to determine grade of stenosis: The degree of stenosis for each area was assessed using calipers to measure the width of the lumen at the site of maximal stenosis and expressing this as a percentage of the true lumen, predicted by extrapolation from above and below the stenosis. Complications: No complications were recorded.
Flow and timing	Exclusions: Two patients did not attend for ultrasound and in one patient the DSA image from one side was spoilt by a swallowing artefact. Results were therefore obtained from 99 carotid arteries. Also excluded from analysis: 46 of the 99 carotid arteries that were successfully scanned, but no abnormality was detected by either DSA or ultrasound Exams were performed within four weeks.
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: Newcastle, UK Year and language of publication: Published in 1992 in English Study period: Not described Participants enrolled: 50 patients Carotids included in analyses: 53
Notes	Excluded from analysis: carotids in which either DSA and DUS found no abnormalities
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Cui 2018.

Study characteristics
Patient Sampling	54 patients presenting with carotid stenosis were admitted to the First Affiliated Hospital of Xinxiang Medical University (Xinxiang, China). Unclear how they were recruited
Patient characteristics and setting	All patients where carotid stenosis was suspected to arise from vascular diseases in the head and neck. In addition, all patients had experienced transient Ischemic attack and other neurological symptoms. The patients consisted of 32 males and 22 females aged between 37 and 82 years, with a mean age of 63.06 ± 13.21 years. Risk factors not described
Index tests	DUS Image production: A Color Doppler Ultrasography (CDUS) diagnostic instrument (Esaote North America, Inc., Indianapolis, IN USA) with a probe frequency range of 5‐12 MHz Contrast: No Criteria used to determine grade of stenosis: Grant 2003
Target condition and reference standard(s)	Reference standard: DSA Target condition: patients where carotid stenosis was suspected to arise from vascular diseases in the head and neck Criteria used to determine grade of stenosis: NASCET criteria Complications: Not described
Flow and timing	No patients were excluded from analysis. All patients underwent CDUS, CE MRA and DSA examinations within 1 week of diagnosis.
Comparative
Methods	Study design: Prospective cohort study Study location: China Year and language of publication: Published in 2017 in English Study period: from January 2012 to January 2014 Participants enrolled: 54 patients. Carotids included in analyses: 216
Notes	Considered 216 carotid arteries because investigators counted the common carotid artery and the internal carotid artery as separated vessels for analysis Did not describe whether patients had undergone any screening imaging test before being included in the study
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

D'Onofrio 2006.

Study characteristics
Patient Sampling	32 consecutive patients with symptoms of carotid artery disease (transient ischemic attack, minor disabling ischemic stroke, or amaurosis fugax) and ultrasonographic findings of stenosis > 50% of the internal carotid artery. In 21 cases, the carotid bifurcation was studied with CDUS, CE‐MRA and DSA and in 11 cases with Doppler US and CE‐MRA only. Exclusion criteria: Not described
Patient characteristics and setting	All patients were symptomatic, all with proven carotid stenosis on DUS. Not described
Index tests	DUS Image production: Doppler US was carried out by the same operator on the same ultrasound unit (Sequoia 512, Acuson/Siemens, Germany) Contrast: No Criteria used to determine grade of stenosis: described in Arbeille 1995 (Appendix 9)
Target condition and reference standard(s)	Reference standard: DSA and MRA Target condition: patients with symptoms of carotid artery disease Criteria used to determine grade of stenosis: NASCET criteria Complications: Not described
Flow and timing	1 carotid was not included in analysis because of previous endarterectomy. Unclear why only 31 arteries included in comparison between DUS and MRA All exams were performed within 1 week.
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: Italy Year and language of publication: Published in 2005 in English and Italian Study period: Not described Participants enrolled: 32 patients Carotids included in analyses: 41 for comparison between DUS and DSA and 31 for comparison between DUS and MRA
Notes	Did not describe whether patients had undergone any screening imaging test before being included in the study
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Das 2009.

Study characteristics
Patient Sampling	15 patients with symptomatic stenosis of the internal carotid artery were included in this study. All patients underwent one Dual‐Source CTA, MRA and DUS of the supra‐aortic arteries, to assess the degree of stenosis of the internal carotid
Patient characteristics and setting	15 symptomatic patients were included, 12 men and 3 women Average age was 69 years (range 53–79 years) Risk factors not described
Index tests	DUS Image production: GE Vivid 7 (GE Healthcare, Milwaukee, USA) with a 7 Mhz probe Contrast: No Criteria used to determine grade of stenosis: DEGUM criteria (Appendix 9)
Target condition and reference standard(s)	Reference standard: CTA and MRA Target condition: symptomatic stenosis of the internal carotid Criteria used to determine grade of stenosis: NASCET criteria Complications: No
Flow and timing	All three investigations were made within a week.
Comparative
Methods	Study design: Prospective study, unclear whether consecutive recruitment Study location: Germany Year and language of publication: Published in 2009 in English Study period: between April 2007 and March 2008 Participants enrolled: 15 patients Carotids included in analyses: 30
Notes	Did not describe whether patients had undergone any screening imaging test before being included in the study
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Unclear
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Unclear
Could the patient flow have introduced bias?		Unclear risk

Eliasziw 1995.

Study characteristics
Patient Sampling	The study used data collected from the first 3 years of NASCET; 1360 patients were recruited from 50 academic centers across North America. A total of 1011 patients had complete ultrasonographic data. Exclusion criteria: data incomplete or unavailable, ischemic event attributable to a cardiac source of embolism, age over 80 years, presence of significant intracranial vascular disease, and life‐threatening or other disabling conditions
Patient characteristics and setting	All patients were symptomatic; other characteristics of the included patients were not described.
Index tests	DUS Image production: the ultrasound device was not specified; the vast majority of the transducers used were in the 5‐MHz range. Contrast: No Criteria used to determine grade of stenosis: described in Appendix 9
Target condition and reference standard(s)	Reference standard: DSA Target condition: Symptomatic patients with severe (70% to 99%) carotid stenoses Criteria used to determine grade of stenosis: NASCET criteria Complications: stroke rate from angiography was 0.78%
Flow and timing	Ultrasonography was performed concurrently to the angiogram.
Comparative
Methods	Study design: Cross‐sectional study; participants consecutively enrolled Study location: Multicenter in North America Year and language of publication: Published in 1995 in English Study period: from January 1988 through February 1991 Participants enrolled: 1011 patients Carotids included in analyses: 1011 carotids
Notes	Ultrasonography was performed concurrently to the angiogram but was not used in the decision‐making process for entering patients into the study.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

Faught 1994.

Study characteristics
Patient Sampling	Patients from 2 non‐invasive vascular laboratories affiliated with the Southern Illinois University School of Medicine. Did not specify recruitment. Exclusion criteria: inadequate arteriograms and patients with ICA occlusions
Patient characteristics and setting	77% of the patients included were symptomatic; no other characteristics described
Index tests	DUS Image production: quote "Duplex examinations were performed with a QUAD I Angiodynograph (Quantum Medical Systems, Issaquah, Wash.) until the latter part of 1989, after which the Quantum 2000 (Quantum Medical Systerns)". Contrast: No Criteria used to determine grade of stenosis: ROC curve analysis
Target condition and reference standard(s)	Reference standard: DSA Target condition: suspected carotid disease patients Criteria used to determine grade of stenosis: NASCET Complications: Not described
Flow and timing	Arteriograms were performed within 1 month.
Comparative
Methods	Study design: Unclear whether prospective design. Unclear whether consecutive recruitment Study location: Chicago, USA Year and language of publication: Published in 1994 in English Study period: from January 1, 1989 through October 30, 1992 Participants enrolled: 405 patients Carotids included in analyses: 770 arteries
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

Golledge 1999.

Study characteristics
Patient Sampling	Selected symptomatic patients admitted to the hospital
Patient characteristics and setting	50 patients, all symptomatic 62% men and 38% women. Median age 71 years (range from 47 to 84 years)
Index tests	DUS Image production: duplex scan imaging was performed with a 5‐MHz probe (angle of insonation, 60 degrees; Ultramark 9, HDI, Advanced Technology Laboratories, Wash.). Contrast: No Criteria used to determine grade of stenosis: ROC curve analysis
Target condition and reference standard(s)	Reference standard: DSA Target condition: carotid artery stenosis Criteria used to determine grade of stenosis: NASCET Complications: No significant complications
Flow and timing	DSA was performed within 24 hours from DUS.
Comparative
Methods	Study design: Unclear whether prospective design Study location: UK Year and language of publication: Published in 1999 in English Study period: June 1996 to June 1997 Participants enrolled: 50 patients Carotids included in analyses: 100 arteries
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	No
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

Hammond 2008.

Study characteristics
Patient Sampling	Inclusion criteria: patients with non‐disabling neurology symptoms and an apparent carotid occlusion on DUS Exclusion criteria: patients failing to undergo all exams within 14 days; patients who refused to provide informed consent
Patient characteristics and setting	Not described
Index tests	DUS Image production: Acuson 128 XP10 US machine 7 MHz linear array transducer Contrast: yes. Contrast agent: Levovist (Schering, UK) Criteria used to determine grade of stenosis: quote "Vessels were characterised as definitely occluded (if no flow was seen anywhere within the cervical ICA) or definitely patent (if a flow channel was seen throughout the cervical ICA)".
Target condition and reference standard(s)	Reference standard: DSA Target condition: carotid occlusion Criteria used to determine grade of stenosis: quote "Vessels were characterised as definitely occluded if no contrast could be identified in the line of the cervical ICA or if there was significant discontinuity with backfilling of the siphon and distal ICA from the intracranial vessels". Complications: Not described
Flow and timing	DUS, DSA and MRA were performed within 14 days. Quote "Nineteen vessels were excluded on the basis of incomplete imaging due to patient failure to attend (8 vessels), claustrophobia and inability to tolerate MRA (10 vessels), or refusal to consent to DSA (1 vessel). A further 9 vessels were excluded because there was a delay of > 14 days in completing of their imaging." A confident diagnosis could not be made in 5/31 (16%) vessels, so 24 vessels included in analysis
Comparative
Methods	Study design: Prospective accuracy study. Unclear whether consecutive recruitment Study location: UK Year and language of publication: Published in 2008 in English Study period: Between April 2001 and August 2004 Participants enrolled: 30 patients Carotids included in analyses: 24 arteries
Notes	Only evaluated patients with the diagnosis of occlusion or pseudocollusion
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	No
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Hansen 1996.

Study characteristics
Patient Sampling	Patients with a planned carotid endarterectomy sent to be examined with DUS and DSA; unclear where the patients were selected and exams performed previously Exclusion criteria not described
Patient characteristics and setting	81 consecutive patients (22 women and 59 men), aged 49‐83 years (mean 68 years) 89% symptomatic patients: 28 patients (34.5%) had had minor strokes, 32 (39.5%) transient ischemic attacks (TIA), and 12 (15%) amaurosis fugax. 9 (11%); those previously undergoing an endarterectomy on the symptomatic side, were operated on because of a contralateral asymptomatic severe stenosis.
Index tests	DUS Image production: Acuson XP 10 (Acuson, Mountain View, CA, U.S.A.), using either a 7 MHz B‐mode real‐time linear scanner including a 5 MHz pulsed and color‐coded Doppler, or a 5 MHz B‐mode real‐time linear scanner including a 3.5 MHz pulsed and color‐coded Doppler Contrast: No Criteria used to determine grade of stenosis: ROC curve analysis
Target condition and reference standard(s)	Reference standard: DSA Target condition: internal carotid stenosis Criteria used to determine grade of stenosis: quote "Diameter reduction in percent = (b ‐ a)/b * 100. Where a is the smallest diameter in the stenotic zone, and b is the diameter of the normal CCA proximal to the stenosis." Complications: Not described
Flow and timing	All exams were performed within 1 month
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: Sweden Year and language of publication: Published in 1996 in English Study period: Not described Participants enrolled: 81 patients Carotids included in analyses: 162 arteries
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Unclear
If a threshold was used, was it pre‐specified?	No
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Unclear
Could the patient flow have introduced bias?		Unclear risk

Heijenbrok‐Kal 2006.

Study characteristics
Patient Sampling	Patients with amaurosis fugax, transient ischemic attack, or minor stroke from three hospitals. Unclear whether other exams were performed previously Occlusions or pseudo‐occlusions were excluded.
Patient characteristics and setting	350 symptomatic patients 76% male (266) and 24% female (84) Mean age of 67 years (range, 39–88 years)
Index tests	DUS Image production: for 311 patients: Ultramark 9 HDI or HDI 3000 (Advanced Technology Laboratories, Bothell, Wash.); for 39 patients: Diasonics Master Series (GE Medical Systems, Milwaukee, Wis.) Contrast: No Criteria used to determine grade of stenosis: PSV of 230 cm/sec for the diagnosis of 70%–99% stenosis and 125 cm/sec for the diagnosis of 50%–99% stenosis
Target condition and reference standard(s)	Reference standard: DSA Target condition: carotid artery stenosis Criteria used to determine grade of stenosis: NASCET Complications: Not described
Flow and timing	313 patients included in analysis (had both examinations). 323 patients were evaluated with DSA and 330 patients with duplex US. Quote "Values were missing owing to the following reasons: Sometimes it was not feasible to perform both examinations before surgery, some patients withdrew from the study after having undergone one examination, and the examination was not always correctly performed according to our study protocol. Also, occasionally, the PSV was not measured when duplex US was performed. Finally, in seven patients, it was impossible to measure the degree of stenosis because of poor image quality and the poor reliability of the DSA findings". DUS and DSA were performed within 4 weeks.
Comparative
Methods	Study design: Prospective diagnostic study Study location: Netherlands Year and language of publication: Published in 2006 in English Study period: January 1997 through January 2000 Participants enrolled: 313 patients Carotids included in analyses: 313 arteries
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Huston 1993.

Study characteristics
Patient Sampling	52 consecutive patients referred for cerebral angiography were selected, but 2 patients were excluded because they did not complete the MRA examination (claustrophobia and severe back pain). Unclear whether other exams were performed previously
Patient characteristics and setting	50 symptomatic patients (symptoms of hemispheric ischemia) 35 men and 15 women aged 42‐82 years. The median age was 67.
Index tests	DUS Image production: Acuson 128; Acuson, Mountain View, Calif. Contrast: No Criteria used to determine grade of stenosis: stenosis 1‐49% ‐ PSV < 125 cm/s; stenosis 50‐79% ‐ PSV ≥ 125 cm/s and EDV ≤ 135 cm/s; 80‐99% ‐ PSV ≥ 125 cm/s and EDV ≥ 135 cm/s
Target condition and reference standard(s)	Reference standard: DSA Target condition: patients with symptoms of carotid artery disease referred to DSA Criteria used to determine grade of stenosis: NASCET Complications: Not described
Flow and timing	52 patients were included, but 2 did not complete MRA examination and were excluded. 98 carotid bifurcations were evaluated by DSA, but statistical analysis was limited to the 77 arteries that had DUS and DSA performed within 2 weeks.
Comparative
Methods	Study design: Prospective accuracy cohort study Study location: USA Year and language of publication: Published in 1993 in English Study period: Not described Participants enrolled: 52 patients Carotids included in analyses: 77 arteries
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Knudsen 2002.

Study characteristics
Patient Sampling	Included patients with suspected symptomatic high‐grade ICA stenosis but did not specify how they suspected it was high‐grade stenosis
Patient characteristics and setting	65 symptomatic patients Characteristics not described Unclear if there were previous exams performed
Index tests	DUS Image production: Siemens Sonoline Elegra, Siemens Medical System, Washington Contrast: No Criteria used to determine grade of stenosis:stenosis ≥ 70% stenosis was characterized by an ICA PSV ≥ 150 cm/s, an ICA end diastolic velocity ≥ 90 cm/s, and a PSV ratio ≥ 2.8. No flow indicated occlusion.
Target condition and reference standard(s)	Reference standard: DSA Target condition: symptomatic high‐grade ICA stenosis Criteria used to determine grade of stenosis: NASCET Complications: Not described
Flow and timing	DSA and DUS performed within 2 days 65 patients and 129 arteries were included in analysis; unclear why 1 artery was excluded
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: Denmark Year and language of publication: Published in 2002 in English Study period: a 12‐month period 1998 to 1999 Participants enrolled: 65 patients Carotids included in analyses: 129 arteries
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Link 1997.

Study characteristics
Patient Sampling	Patients were evaluated for symptomatic cerebrovascular disease and agreed to all tree examinations (DUS, DSA and CTA). Exclusion criteria not described
Patient characteristics and setting	28 patients included, 10 women (35.7%) and 18 men (64.3%). All patients were symptomatic. Age: 46‐77 years old (mean 63 years)
Index tests	DUS Image production: Philips P700. Color‐coded duplex sonography was performed with 5‐ and 7.5‐MHz linear‐array transducers Contrast: No Criteria used to determine grade of stenosis: Carotid stenosis were classified as mild (0‐29%) when PSV was less than 100 cm/s; moderate (30‐69%) when PSV ranged from 100 cm/s to 200cm/s; and severe (70‐99%) when PSV was greater than 200 cm/s
Target condition and reference standard(s)	Reference standard: DSA Target condition: symptomatic carotid stenosis Criteria used to determine grade of stenosis: NASCET Complications: Not described
Flow and timing	All three examinations were performed within 48 hours. All arteries included in analysis
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: Germany Year and language of publication: Published in 1996 in English Study period: Not described Participants enrolled: 28 patients Carotids included in analyses: 56 arteries
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

Lubezky 1998.

Study characteristics
Patient Sampling	The study included patients with carotid occlusion diagnosed by CDUS. Exclusion criteria: quote "Patients in whom the duplex scan was equivocal, with poor visualization of the ICA, were excluded from the study. Only cases in which a technically satisfactory duplex scanning was obtained were included".
Patient characteristics and setting	148 patients diagnosed with carotid occlusion. 102 male (70%) and 46 female (30%) Age: ranged from 43 to 89 years old (average age was 70 years) 22% were asymptomatic and 88% were symptomatic. Other risk factors for atherosclerosis: 99 patients (67%) had hypertension, 85 (58%) were current or past smokers, 64 (44%) had a history of ischemic heart disease, 56 (38%) had hypercholesterolemia, 44 (30%) had diabetes, and 38 patients (26%) had peripheral vascular disease. Five patients (3%) had a history of neck radiation.
Index tests	DUS Image production: HD13000 system (Advanced Technology Laboratories, Bothell, WA, USA) using a linear 5‐10 MHz, 38 mm transducer Contrast: No Criteria used to determine occlusion: Not specified, only occlusion included
Target condition and reference standard(s)	Reference standard: CTA and DSA Target condition: carotid artery occlusion Criteria used to determine grade of stenosis: only occlusion included Complications: Not described
Flow and timing	Maximum time interval between studies was 30 days.
Comparative
Methods	Study design: Unclear Study location: Israel Year and language of publication: Published in 1998 in English Study period: From 1995 to 1997 Participants enrolled: 148 patients Carotids included in analyses: 148 arteries compared with CTA and 54 arteries compared with DSA
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	No
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Unclear
If a threshold was used, was it pre‐specified?	No
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	No
Could the reference standard, its conduct, or its interpretation have introduced bias?		High risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		High risk

Nederkoorn 2002.

Study characteristics
Patient Sampling	350 consecutive symptomatic patients suspected of having carotid artery stenosis were included. Not described if any previous exams were performed Exclusion criteria: patients with contraindications for MRA
Patient characteristics and setting	350 patients included, 76% male and 24% female. Mean age 67 years (range from 39‐88 years) 100% of the patients were symptomatic. 4% had previous carotid endarterectomy. Other risk factors for atherosclerosis: 49% had hypertension; 15% had diabetes; 51% had cardiac history (angina, myocardial infarction, heart failure, or bypass surgery or PTA); 23% had peripheral arterial disease; 49% were current smokers and 34% were ex‐smokers.
Index tests	DUS Image production: Not described Contrast: No Criteria used to determine stenosis: Carotid stenosis were classified as mild (0‐29%) when PSV was less than 150 cm/s; mild‐to‐moderate (30‐49%) when PSV ranged from 150 cm/s to < 190 cm/s; moderate (50‐69%) when PSV ranged from 190 cm/s to < 270 cm/s; severe (70‐99%) when PSV was greater than 270 cm/s; and occlusion when no flow was detected.
Target condition and reference standard(s)	Reference standard: DSA Target condition: carotid artery stenosis Criteria used to determine grade of stenosis: NASCET criteria Complications: 1.4% minor stroke, 0.3% major stroke and 0.6% mortality
Flow and timing	From the 350 patients, 323 DSA, 330 DUS, and 295 MRA were included in the analysis. 313 arteries were included in DUS versus DSA analysis. Missing data were caused by: impossibility of performing all three exams before surgery, patients withdrawn, tests not performed according to the study protocol, or poor quality of images of some MRA and DSA. Patients underwent DUS, MRA, and DSA examination within a maximum of 4 weeks.
Comparative
Methods	Study design: Prospective, consecutive, accuracy cohort study Study location: University Medical Center Utrecht, University Medical Center Rotterdam, and Enschede Medical Center (Netherlands) Year and language of publication: Published in 2002 in English Study period: January 1997 to November 2000 Participants enrolled: 350 patients Carotids included in analyses: 313 arteries
Notes	Only the symptomatic side was included in the analysis.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

Wolfle 2002.

Study characteristics
Patient Sampling	47 patients were included with suspected severe (≥ 70%) carotid stenosis; unclear how patients were selected Exclusion criteria: patients with claustrophobia or contraindications for MRA
Patient characteristics and setting	47 patients included; 34 (72%) were symptomatic and 13 (28%) were asymptomatic 37 (79%) male and 10 (21%) female The median age was 68 years (ranging from 46‐84 years).
Index tests	DUS Image production: Siemens (Elegra) with a 7,5MHz linear scan Contrast: No Criteria used to determine stenosis: Carotid stenosis was classified as severe (70‐99%) when PSV was greater than 230 cm/s and EDV was greater than 70 cm/s.
Target condition and reference standard(s)	Reference standard: DSA Target condition: ≥ 70% carotid artery stenosis Criteria used to determine grade of stenosis: NASCET criteria Complications: 2.1% stroke rate
Flow and timing	Median time between exams was 2.8 days (SD 2.17)
Comparative
Methods	Study design: Prospective, not consecutive, accuracy study Study location: Germany Year and language of publication: Published in 2002 in German Study period: Not described Participants enrolled: 47 patients Carotids included in analyses: 94 arteries
Notes	Patients were only included with proven carotid stenosis.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Was a case‐control design avoided?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (All tests)
Were the index test results interpreted without knowledge of the results of the reference standard?	Yes
If a threshold was used, was it pre‐specified?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

CCA: common carotid artery
CDUS: color duplex ultrasound
CE: contrast enhanced
CTA: computed tomography angiography
DEGUM: The German Society of Ultrasound in Medicine
DSA: digital subtraction angiography
DUS: duplex ultrasound
EDV: end diastolic velocity
ICA: internal carotid artery
IQR: interquartile range
MRA: magnetic resonance angiography
MR: magnetic resonance
NASCET: North American Symptomatic Carotid Endaterectomy Trial
PSV: peak systolic velocity
PTA: percutaneous transluminal angioplasty
ROC:receiver operating curve
SD: standard deviation
TIA: transient ischemic attack
US: ultrasound

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
AbuRahma 1995	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
AbuRahma 1997	Less than 70% of the patients included were symptomatic and time between index test and alternative test was not specified
AbuRahma 1998	Less than 70% of the patients included were symptomatic
AbuRahma 2011	Less than 70% of the patients included were symptomatic
Ackerstaff 1982	Less than 70% of the patients included were symptomatic
Ackroyd 1984	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Adiga 1984	Study did not provide enough data for construction of a 2 x 2 table and the method of calculating the degree of stenosis
Alexandrov 1993	Study did not define the proportion of symptomatic patients and time between index test and alternatives tests was not specified
Alexandrov 1997a	Study did not define the proportion of symptomatic patients and time between index test and alternatives tests was not specified
Alexandrov 1997b	Study did not define the proportion of symptomatic patients
Alves 1982	Time between index test and alternative test was not specified
Alves 1983	Less than 70% of the patients included were symptomatic, time between index test and alternative test was not specified, and the study did not provide enough information about the method of calculating the degree of stenosis
Ammar 2017	Retrospective study that did not provide any suitable test comparison. The object of the study was if additional imaging studies (over DUS) were necessary for treatment planning
Anderson 1983	Time between index test and alternative test was not specified. An experimental study about the US method; the quantification of stenosis was based on subjective visual impression
Anderson 2000	The DUS examinations were not standardized and there was no description of time between examinations
Appleberg 1982	Study did not define the proportion of symptomatic patients and time between index test and alternatives tests was not specified
Arbeille 1984	Study did not define the proportion of symptomatic patients and the degree of stenosis was determined by a subjective visual impression of the Doppler spectrum analysis
Arbeille 1997	Only DUS was assessed; there was no comparison with CTA or DSA or MRA
Archie 1981	Study did not define the proportion of symptomatic patients and time between index test and alternatives tests was not specified
Arous 2019	Study did not define the proportion of symptomatic patients and time between index test and alternative test was more than four weeks
Auffray‐Calvier 1996	Comparision on MRA and DSA. DUS was performed, but there were no data on DUS accuracy
Azieva 2016	No suitable diagnostic accuracy data
Back 2000	Less than 70% of the patients included were symptomatic
Back 2003	No direct comparison between DUS and MRA or DSA. The study compared MRA and DSA after inconclusive duplex scan
Bain 1998	Time between index test and alternative test was more than four weeks
Ballard 1994	Less than 70% of the patients included were symptomatic and time between index test and alternative test was more than four weeks
Ballard 1997	Less than 70% of the patients included were symptomatic
Ballotta 1999	Less than 70% of the patients included were symptomatic
Bandyk 1985	Time between index test and alternative test was not specified and the degree of stenosis was determined by a subjective visual impression of the Doppler spectrum analysis
Barlinn 2018	Less than 70% of the patients included were symptomatic
Barnes 1976	Study did not define the proportion of symptomatic patients
Barnes 1982	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis). Time between index test and alternative test was not specified
Barry 1987	Less than 70% of the patients included were symptomatic
Bartylla 1997	Study did not define the proportion of symptomatic patients and time between index test and alternatives tests was not specified
Baskett 1976	Preliminary paper on DUS technique. Most of the included population were healthy volunteers
Beckett 1990	Study did not define the proportion of symptomatic patients and time between index test and alternatives tests was not specified
Beebe 1999	Study did not define the proportion of symptomatic patients.Time between index test and alternative test was more than four weeks
Beer 1983	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis)
Beer 1986	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis)
Benhamou 1984	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis) and compared DUS results with postoperative endarterectomy specimens
Berger 1983	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis) and index test was transvenous digital subtraction angiography. Study did not define the proportion of symptomatic patients
Berman 1995	Time between index test and alternatives tests was not specified
Berry 1980	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis). Time between index test and alternative test was not specified
Beutler 1985	Study did not define the proportion of symptomatic patients and the degree of stenosis was determined by a subjective visual impression of the Doppler spectrum analysis
Biasi 1998	Less than 70% of the patients included were symptomatic
Binaghi 2001	Study did not define the proportion of symptomatic patients
Birmpili 2018	Study did not define the proportion of symptomatic patients
Blackshear 1984	No direct comparison of DUS and DSA. Compared systolic peak frequency on DUS with pressure gradient measured at operation
Blackshear 1985	Study did not define the proportion of symptomatic patients
Blackshear 1987	Study did not define the proportion of symptomatic patients
Bladin 1995	Accuracy of DUS was not assessed
Blasberg 1982	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Bloch 1979	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described ("Audible Doppler sounds from the flowmeter were distributed to a speaker and to a stereo tape recorder. A lateral projection image of the common carotid artery and its major branches was produced with this device").
Boccalon 1985	Study did not define the proportion of symptomatic patients
Bone 1976	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and time between index test and alternative test was not specified
Bone 1988	Time between index test and alternative test was not specified
Bonig 2000	Time between index test and alternative test was not specified
Boyko 2018	DUS and other angiographic modalities were performed within 6 months
Boyle 1995	Time between index test and alternative test was more than four weeks. Accuracy of duplex was assessed compared with operative findings
Branas 1994	Study did not define the proportion of symptomatic patients
Braun 2008	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Browman 1995	Study did not define the proportion of symptomatic patients
Bucek 2006	Study did not define the proportion of symptomatic patients
Buijs 1993	Time between index test and alternative test was more than four weeks and less than 70% of the patients included were symptomatic
Bulger 2005	Time between index test and alternative test was more than four weeks and less than 70% of the patients included were symptomatic
Busse 1974	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Busuttil 1996	Less than 70% of the patients included were symptomatic
Caes 1987	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis). Time between index test and alternative test was not specified
Cape 1984	Study did not define the proportion of symptomatic patients
Cappetti 1996	Study did not define the proportion of symptomatic patients
Carnicelli 2013	Study did not define the proportion of symptomatic patients and patients were included if they underwent CTA within 6 months of a DUS
Carpenter 1995	Study did not define the proportion of symptomatic patients
Carpenter 1996	Study did not define the proportion of symptomatic patients
Carroll 1989	Time between index test and alternative test was not specified
Chaix 1985	Subjective criteria to estimate stenosis on DUS and the proportion of symptomatic patients were not specified
Chan 1982	Study did not define the proportion of symptomatic patients and time between index test and alternatives tests was not specified
Chang 1995	Study did not define the proportion of symptomatic patients and time between exams was up to 2 months. Another sample of patients was included and time between exams was up to 6 months
Chang 2002	Less than 70% of the patients included were symptomatic
Chen 1997	Less than 70% of the patients included were symptomatic
Chen 1998	Study did not define the proportion of symptomatic patients
Chervu 1994	Less than 70% of the patients included were symptomatic
Chowdhury 2011	The exact criteria for determination of the degree of stenosis was not specified
Clevert 2006	Less than 70% of the patients included were symptomatic
Clevert 2007	Less than 70% of the patients included were symptomatic
Colhoun 1984	Study did not define the proportion of symptomatic patients
Collins 2005	Time between index test and alternative test was not specified
Colon 1979	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and the study did not define the proportion of symptomatic patients
Connolly 1985	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Cooperberg 1992	Study did not define the proportion of symptomatic patients
Corti 1998	Less than 70% of the patients included were symptomatic (stroke, amayrosis fugax, transient ischemic attack). The exact criteria for determination of the degree of stenosis was not specified
Criswell 1998	Study did not define the proportion of symptomatic patients
Crummy 1979	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and time between index test and alternative test was not specified
Csanyi 1993	Study did not define the proportion of symptomatic patients and the average time between DSA was 24.3 + 21.0 days
Curley 1998	Time between index test and alternative test was not specified
Daiss 1984	Time between index test and alternative test was not specified
Dalotto 1985	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and did not define the proportion of symptomatic patients
Daolio 2019	Time between index test and alternative test was more than four weeks and less than 70% of the patients included were symptomatic
Dawson 1991	Less than 70% of the patients included were symptomatic
Dawson 1993	Less than 70% of the patients included were symptomatic
Dean 2005	The exact criteria for determination of the degree of stenosis was not specified
De la Cruz Cosme 2017	The exact criteria for determination of the degree of stenosis was not specified
De Monti 2003	Study did not define the proportion of symptomatic patients
Dharmasaroja 2018	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Dilley 1986	Study did not define the proportion of symptomatic patients
Dinkel 2001	Time between index test and alternative test was not specified and it stated that most of the participants had symptomatic cerebrovascular disease, but the proportion was not described
Dippel 1999	Time between index test and alternative test was more than four weeks
Dix 2000	Study did not define the proportion of symptomatic patients
Doyle 2012	Study did not define the proportion of symptomatic patients
Doyle 2014	Study did not define the proportion of symptomatic patients
Drevet 1997	Less than 70% of the patients included were symptomatic
Eckmann 1990	Less than 70% of the patients included were symptomatic
Ellis 1996	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Elmore 1998	Study did not define the proportion of symptomatic patients
El‐Saden 2001	Time between index test and alternative test was more than four weeks
Engelhardt 2005	Less than 70% of the patients included were symptomatic
Erdoes 1996	Study did not sufficiently provide data for 2 × 2 table production
Erickson 1989	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Felber 1985	Time between index test and alternative test was more than four weeks and no the criteria used to estimate stenosis was not described
Fell 1981	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Filis 2002	Less than 70% of the patients included were symptomatic
Fillinger 1996	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Finkenzeller 2008	Time between index test and alternative test was not specified
Fischer 1985	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and study did not define the proportion of symptomatic patients
Fischer 1985a	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and study did not define the proportion of symptomatic patients
Fix 1984	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Flanigan 1985	The exact criteria for determination of the degree of stenosis was not specified
Fragata 2006	Time between index test and alternative test was more than four weeks
French‐Sherry 2016	Study did not define the proportion of symptomatic patients
Friese 2001	Study did not define the proportion of symptomatic patients
Fujimoto 2006	Study did not define the proportion of symptomatic patients
Furst 1993	Time accepted between index test and alternative test was more than four weeks
Furst 1999	Case‐control design
Geidel 1991	Time between index test and alternative test was not specified
Geuder 1989	Time between index test and alternative test was not specified and the exact criteria for determination of the degree of stenosis was not specified
Giraldi 1986	Evaluated patients with occlusion of the internal carotid artery for information on the collateral circles (Willis and pre‐Willis)
Glover 1984	Less than 70% of the patients included were symptomatic
Gmelin 1985	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Golledge 1996	Time accepted between index test and alternative test was more than four weeks
Goodson 1987	Time between index test and alternative test was not specified and the exact criteria for determination of the degree of stenosis was not specified
Gortler 1994	Accuracy was determined by comparison with the surgical specimen
Grajo 2007	Time accepted between index test and alternative test was more than four weeks
Grant 1999	Less than 70% of the patients included were symptomatic
Grant 2000	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Griewing 1996	Less than 70% of the patients included were symptomatic
Griffiths 1998	Time between index test and alternative test was not specified and the exact criteria for determination of the degree of stenosis was not specified.
Griffiths 2001	Time between index test and alternative test was not specified and the exact criteria for determination of the degree of stenosis was not specified
Hames 1981	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Hames 1985	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Harward 1986	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Hathout 2005	The average time interval between sonography and arteriography was 2 months and the study did not define the proportion of symptomatic patients
Hathout 2015	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Herring 1984	Study did not define the proportion of symptomatic patients
Hetzel 1993	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Hjelmgren 2018	Evaluated non‐stenotic carotid plaques
Hobson 1980	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Honish 2005	Time between index test and alternative test was not specified
Horrocks 1979	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Howard 1991	Study did not define the proportion of symptomatic patients
Humphrey 1990	Time between index test and alternative test was not specified
Hunink 1993	Study did not define the proportion of symptomatic patients
Huston 1998	Study did not define the proportion of symptomatic patients
Huston 2000	Study did not define the proportion of symptomatic patients
Hutchison 1985	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Hwang 2002	Study did not define the proportion of symptomatic patients
Hwang 2003	Study did not define the proportion of symptomatic patients
Hwang 2003a	Time between index test and alternative test was more than four weeks and the study did not define the proportion of symptomatic patients
Jackson 1985	Less than 70% of the patients included were symptomatic
Jackson 1998	Less than 70% of the patients included were symptomatic
Jacobs 1985	Study did not define the proportion of symptomatic patients
Jogestrand 2002	Time between index test and alternative test was more than four weeks and study did not provide enough data for construction of a 2 x 2 table
Johnson 2000	Time between index test and alternative test was not specified
Johnston 1982	Less than 70% of the patients included were symptomatic
Johnston 1985	Study did not define the proportion of symptomatic patients
Johnston 2001	Less than 70% of the patients included were symptomatic
Jones 1982	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Juhel 1983	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Jung 2000	Less than 70% of the patients included were symptomatic
Jung 2002	Less than 70% of the patients included were symptomatic
Kagawa 1996	Less than 70% of the patients included were symptomatic
Keberle 2001	Study did not define the proportion of symptomatic patients
Keller 1978	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and the study did not define the proportion of symptomatic patients
Kim 2016	Time between index test and alternative test was more than four weeks
Kim 2018	Time between index test and alternative test was more than four weeks
Kirsch 1994	Study did not define the proportion of symptomatic patients
Knox 1982	Study did not define the proportion of symptomatic patients and time accepted between index test and alternative test was more than four weeks
Koga 1983	Less than 70% of the patients included were symptomatic
Koga 2001	Study did not define the proportion of symptomatic patients
Korteweg 2008	Time between index test and alternative test was more than four weeks
Krappel 2002	Study did not define the proportion of symptomatic patients
Krasinski 2009	Only included subjects without hemodynamically significant carotid stenosis and did not describe if they are symptomatic or asymptomatic. The objective was to evaluate potential spatial differences in carotid atherosclerosis measured using 3D MR and US
Kreske 1999	Study did not define the proportion of symptomatic patients
Kuhn 1981	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and study did not define the proportion of symptomatic patients
Kuhn 1984	Less than 70% of the patients included were symptomatic
Labropoulos 1997	Study did not define the proportion of symptomatic patients
Langlois 1983	Study did not define the proportion of symptomatic patients
Lee 1992	Study did not define the proportion of symptomatic patients
Lee 1996	Time between index test and alternative test was more than four weeks
Lefemine 1986	Preliminary paper of DUS technique and the study did not supply information on accuracy data
Leonardo 2003	Less than 70% of the patients included were symptomatic
Levien 1985	Time between index test and alternative test was not specified
Lewis 1980	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Lewis 2002	Time between index test and alternative test was more than four weeks and study did not provide enough data for construction of a 2 x 2 table
Lindegaard 1984	Time between index test and alternative test was not specified
Link 1997a	Time between index test and alternative test was not specified
Long 2001	Less than 70% of the patients included were symptomatic
Lovelock 2003	Study did not define the proportion of symptomatic patients
Ludwig 1984	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. Time between index test and alternative test was not described and the study did not define the proportion of symptomatic patients
Lusby 1981	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis). Time between index test and alternative test was not specified
Macharzina 2018	Less than 70% of the patients included were symptomatic
Macheers 1986	Study did not define the proportion of symptomatic patients
MacKenzie 2002	Study did not define the proportion of symptomatic patients
Makaryus 2009	Less than 70% of the patients included were symptomatic
Manga 1986	Study did not define the proportion of symptomatic patients
Mansour 1995	Less than 70% of the patients included were symptomatic
Marshall 1988	Less than 70% of the patients included were symptomatic
Martin‐Conejero 2007	Less than 70% of the patients included were symptomatic
Matos 2014	Less than 70% of the patients included were symptomatic and time between index test and alternative test was more than four weeks
Mattle 1991	Time between index test and alternative test was not specified and did not define the proportion of symptomatic patients
Mattos 1992	Study did not define the proportion of symptomatic patients
Mattos 1994	Less than 70% of the patients included were symptomatic
Matz 2017	Time between index test and alternative test was not specified and did not define the proportion of symptomatic patients
McLaren 1996	Study did not define the proportion of symptomatic patients
Mitchell 1991	Time between index test and alternative test was not specified and did not define the proportion of symptomatic patients
Mittl 1994	Time between index test and alternative test was not specified
Modaresi 1999	Study did not define the proportion of symptomatic patients
Moll 2000	Time between index test and alternative test was not specified and did not define the proportion of symptomatic patients
Moll 2001	Time between index test and alternative test was not specified and did not define the proportion of symptomatic patients
Moneta 1993	Study did not define the proportion of symptomatic patients
Moore 1986	Study did not define the proportion of symptomatic patients
Moore 1988	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. The study did not define the proportion of symptomatic patients
Muller 2015	Study did not define the proportion of symptomatic patients
Murie 1984	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. Time between index test and alternative test was more than four weeks
Muto 1996	Less than 70% of the patients included were symptomatic
Neale 1994	Study did not define the proportion of symptomatic patients
Neff 2005	Study did not define the proportion of symptomatic patients
Neschis 2001	Study did not define the proportion of symptomatic patients
New 2001	Study did not define the proportion of symptomatic patients and time between index test and alternative test was more than four weeks
Nichtweiss 1987	Time between index test and alternative test was not specified and the method of calculating the carotid stenosis was not described
Nonent 2004	Less than 70% of the patients included were symptomatic
Nonent 2011	Study did not define the proportion of symptomatic patients
Nordal 1993	Time between index test and alternative test was not specified and the method of calculating the carotid stenosis was not described
Norrving 1981	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. Time between index test and alternative test was not specified
Norrving 1985	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Nowak 2007	Same patients from Jogestrand 2002. Time between index test and alternative test was more than four weeks
O'Callaghan 2011	Study did not define the proportion of symptomatic patients
O'Leary 1987	Study did not define the proportion of symptomatic patients
Ohm 2005	Less than 70% of the patients included were symptomatic
Orgles 1999	Study did not define the proportion of symptomatic patients
Paciaroni 2003	The exact criteria for determination of the degree of stenosis was not specified
Padayachee 1982	Less than 70% of the patients included were symptomatic
Padayachee 1997	Study did not define the proportion of symptomatic patients
Paivansalo 1996	Time between index test and alternative test was more than four weeks
Patel 1995	Less than 70% of the patients included were symptomatic ("There were 74 symptomatic carotid bifurcations (42%)")
Patel 2002	Time accepted between index test and alternative test was more than four weeks ("The median time lapse between DUS and the other three imaging techniques was 33 days (range 27 to 185 days)")
Pelz 2015	Less than 70% of the patients included were symptomatic
Petisco 2015	Less than 70% of the patients included were symptomatic
Pfister 2009	Study did not define the proportion of symptomatic patients
Poindexter 1991	Less than 70% of the patients included were symptomatic
Polak 1989	Study did not define the proportion of symptomatic patients
Polak 1992	Less than 70% of the patients included were symptomatic
Polak 1993	MRA and DUS were used in combination. There was no DUS alone accuracy data and time between index test and alternative test was more than four weeks
Portilla 2010	Study did not define the proportion of symptomatic patients
Puzich 1986	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Py 2001	Time between index test and alternative test was more than four weeks
Qureshi 2001	Study did not define the proportion of symptomatic patients
Ratliff 1985	Study did not define the proportion of symptomatic patients
Ricotta 1987	Study did not define the proportion of symptomatic patients
Riles 1992	Study did not define the proportion of symptomatic patients
Rodrigus 1995	Study did not define the proportion of symptomatic patients
Saba 2008	Study did not define the proportion of symptomatic patients
Saba 2010	Study did not define the proportion of symptomatic patients
Sabeti 2004	Less than 70% of the patients included were symptomatic
Saia 1981	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. Study did not define the proportion of symptomatic patients
Samarzija 2018	Study did not define the proportion of symptomatic patients and time between index test and alternative test was more than four weeks
Sameshima 1999	Study did not provide the method of calculating the degree of stenosis and time between index test and alternative test was not specified
Saouaf 1998	Less than 70% of the patients included were symptomatic
Satiani 1988	Study did not define the proportion of symptomatic patients
Savic 2010	Time between index test and alternative test was not specified
Senant 1984	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Serfaty 2000	Less than 70% of the patients included were symptomatic
Shaalan 2008	Less than 70% of the patients included were symptomatic
Shakhnovich 2010	Study did not define the proportion of symptomatic patients
Sillesen 1988	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. Study did not define the proportion of symptomatic patients
Sillesen 1991	Less than 70% of the patients included were symptomatic
Sitzer 1993	Criteria to determine carotid stenosis was not based on velocity criteria and time between tests was not described
Slovut 2010	Study did not define the proportion of symptomatic patients
Soulez 1999	Study did not define the proportion of symptomatic patients
Srinivasan 1995	Time between index test and alternative test was not specified
Staikov 2000	Time between index test and alternative test was more than four weeks
Staikov 2002	Study did not define the proportion of symptomatic patients
Staikov 2004	Time between index test and alternative test was more than four weeks
Stavenow 1987	Less than 70% of the patients included were symptomatic
Stefanini 2012	Time between index test and alternative test was more than four weeks
Steger 1995	Less than 70% of the patients included were symptomatic
Steinke 1990	Less than 70% of the patients included were symptomatic
Steinke 1997	Less than 70% of the patients included were symptomatic
Sumner 1979	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Sumner 1982	Less than 70% of the patients included were symptomatic
Tarnawski 1990	Validation of MRA technique using a pulsatile phantom and in vivo healthy asymptomatic subjects
Tateishi 2013	Study did not define the proportion of symptomatic patients
Tian 2016	Study did not define the proportion of symptomatic patients
Titi 2007	Time between index test and alternative test was not specified
Tokunaga 2016	Time between index test and alternative test was not specified
Tola 2004	Asymptomatic patients
Torvaldsen 1985	Less than 70% of the patients included were symptomatic
Tschammler 1991	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Turnipseed 1982	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described.
Turnipseed 1993a	Time between index test and alternative test was not specified
Utz 1983	Study did not define the proportion of symptomatic patients
Vaisman 1986	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Van Prehn 2008	Study did not define the proportion of symptomatic patients
Vit 2003	Study did not define the proportion of symptomatic patients
Von Arbin 1983	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described. Study did not define the proportion of symptomatic patients
Wardlaw 2005	Time between index test and alternative test was not specified
Weaver 1980	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis). Study did not define the proportion of symptomatic patients
Weaver 1980a	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described (subjective visual impression of the degree of stenosis). Study did not define the proportion of symptomatic patients
Weintraub 1985	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described and compared DUS results with postoperative endarterectomy specimens. Study did not provide enough data for construction of a 2 x 2 table
Wessels 2004	Less than 70% of the patients included were symptomatic
Wetzner 1984	Study did not define the proportion of symptomatic patients
Wikstrom 2002	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Wilkerson 1991	Less than 70% of the patients included were symptomatic
Wilterdink 1996	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Winkelaar 1999	Study did not define the proportion of symptomatic patients
Withers 1990	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Wolverson 1983	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Wolverson 1985	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Worthy 1997	Time between index test and alternative test was not specified
Yiu‐Tong 1985	Less than 70% of the patients included were symptomatic
Young 1992	Time between index test and alternative test was not specified
Young 1994	Did not use a valid method for determining the degree of stenosis on DSA. (quote: "We have relied on experienced radiologists reporting their visual impression of the degree of stenosis present, as we believe that this is the method most commonly used in routine clinical practice.")
Yurdakul 2004	Study did not define the proportion of symptomatic patients and time between index test and alternative test was not specified
Yurdakul 2004a	Asymptomatic patients
Zananiri 1993	Asymptomatic patients
Zanette 1982	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Zanette 1987	Preliminary paper of DUS technique; no objective criteria to estimate stenosis described
Zierler 1990	Asymptomatic patients
Zorzon 1987	Study did not define the proportion of symptomatic patients
Zwicker 1987	Less than 70% of the patients included were symptomatic
Zwiebel 1983	Less than 70% of the patients included were symptomatic
Zwiebel 1985	Less than 70% of the patients included were symptomatic

CTA: computed tomography angiography
DSA: digital subtraction angiography
MR: magnetic resonance
MRA: magnetic resonance angiography
US: ultrasound

Differences between protocol and review

The International Network of Agencies for Health Technology Assessment has taken responsibility for the HTA database, previously managed by the Centre for Reviews and Dissemination (CRD). This was previously done by the Centre for Reviews and Dissemination (CRD).

We aimed only to include participants with symptomatic carotid stenosis but accepted studies in which at least 70% of included participants were symptomatic.

We removed from QUADAS‐2 the question ‘Was the person conducting the test (DUS) sufficiently trained?’ in the index test domain.

We were unable to perform all the sensitivity and subgroup analyses and meta‐regressions we had originally planned and detailed in the protocol, due to the small number of included studies.

Sensitivity analyses have been added to explore the effects of excluding studies that only included patients with known occlusion on DUS.

Two authors joined the review team after the protocol was published (LCUN and NCJ) and contributed to the final version of the review.

Contributions of authors

NC, JCCBS, RS, VV, and RLGF designed the review, registered the review title, and contributed to the design of the review. NC, CDQF, and RLGF developed the search strategies, with additional input from Joshua David Cheyne, Cochrane Stroke Information Specialist. NC, CDQF, and LCUN screened papers against eligibility criteria and appraised the quality of the papers. NC, RLGF, and NCJ extracted study data and performed and interpreted analyses. NC wrote the first draft of the review with contributions from RLGF. All authors reviewed and approved the protocol content prior to submission.

Sources of support

Internal sources

• No sources of support provided

External sources

• Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Brazil

  This study was financed in part by CAPES, finance code 001.

Declarations of interest

NC: none known.

JCCBS: none known.

CDQF: none known.

RS: none known.

VV: none known.

NCJ: none known.

LCUN: none known.

RLGF: none known.

New