Computed tomography angiography or magnetic resonance angiography for detection of intracranial vascular malformations in patients with intracerebral haemorrhage

Abstract

Background

Intracranial vascular malformations (brain or pial/dural arteriovenous malformations/fistulae, and aneurysms) are the leading cause of intracerebral haemorrhage (ICH) in young adults. Early identification of the intracranial vascular malformation may improve outcome if treatment can prevent ICH recurrence. Catheter intra‐arterial digital subtraction angiography (IADSA) is considered the reference standard for the detection an intracranial vascular malformation as the cause of ICH. Computed tomography angiography (CTA) and magnetic resonance angiography (MRA) are less invasive than IADSA and may be as accurate for identifying some causes of ICH.

Objectives

To evaluate the diagnostic test accuracy of CTA and MRA versus IADSA for the detection of intracranial vascular malformations as a cause of ICH.

Search methods

We searched MEDLINE (1948 to August 2013), EMBASE (1980 to August 2013), MEDION (August 2013), the Database of Abstracts of Reviews of Effects (DARE; August 2013), the Health Technology Assessment Database (HTA; August 2013), ClinicalTrials.gov (August 2013), and WHO ICTRP (International Clinical Trials Register Portfolio; August 2013). We also performed a cited reference search for forward tracking of relevant articles on Google Scholar (http://scholar.google.com/), screened bibliographies, and contacted authors to identify additional studies.

Selection criteria

We selected studies reporting data that could be used to construct contingency tables that compared CTA or MRA, or both, with IADSA in the same patients for the detection of intracranial vascular malformations following ICH.

Data collection and analysis

Two authors (CBJ and RA‐SS) independently extracted data on study characteristics and measures of test accuracy. Two authors (CBJ and PMW) independently extracted data on test characteristics. We obtained data restricted to the subgroup undergoing IADSA in studies using multiple reference standards. We combined data using the bivariate model. We generated forest plots of the sensitivity and specificity of CTA and MRA and created a summary receiver operating characteristic plot.

Main results

Eleven studies (n = 927 participants) met our inclusion criteria. Eight studies compared CTA with IADSA (n = 526) and three studies compared MRA with IADSA (n = 401). Methodological quality varied considerably among studies, with partial verification bias in 7/11 (64%) and retrospective designs in 5/10 (50%). In studies of CTA, the pooled estimate of sensitivity was 0.95 (95% confidence interval (CI) 0.90 to 0.97) and specificity was 0.99 (95% CI 0.95 to 1.00). The results remained robust in a sensitivity analysis in which only studies evaluating adult patients (≥ 16 years of age) were included. In studies of MRA, the pooled estimate of sensitivity was 0.98 (95% CI 0.80 to 1.00) and specificity was 0.99 (95% CI 0.97 to 1.00). An indirect comparison of CTA and MRA using a bivariate model incorporating test type as one of the parameters failed to reveal a statistically significant difference in sensitivity or specificity between the two imaging modalities (P value = 0.6).

Authors' conclusions

CTA and MRA appear to have good sensitivity and specificity following ICH for the detection of intracranial vascular malformations, although several of the included studies had methodological shortcomings (retrospective designs and partial verification bias in particular) that may have increased apparent test accuracy.

Plain language summary

Computed tomography angiography or magnetic resonance angiography for detecting blood vessel abnormalities in patients with intracerebral haemorrhage

Blood vessel abnormalities are the leading cause of bleeding in the brain (known as intracerebral haemorrhage) in young adults. Early detection of blood vessel abnormalities may improve outcome if treatment can prevent bleeding recurrence. This review looked at different tests used to identify blood vessel abnormalities in the brain. Intra‐arterial digital subtraction angiography (IADSA) is the standard test used and involves positioning a tube, introduced through a blood vessel in the groin, into blood vessels near the brain. Dye is directly injected into the brain's blood vessels using this tube. Computed tomographic angiography (CTA) and magnetic resonance angiography (MRA) are newer tests that may be done without any injections (MRA) or only through an injection into the arm (CTA and MRA). This review investigated the accuracy of CTA or MRA, or both, compared with IADSA after intracerebral haemorrhage. We found eight studies (involving 526 participants) that compared CTA with IADSA and three studies (involving 401 participants) that compared MRA with IADSA. Both CTA and MRA appear to have good accuracy when compared with IADSA. However, the studies were small and were limited in many cases by their design. Further research that looks at accuracy, practicality, and costs is needed.

Summary of findings

Summary of findings'. 'Summary of results. Results of studies on computed tomography angiography.

Population	Individuals with acute intracerebral haemorrhage (with or without extension into other intracranial spaces) who are well enough to undergo computed tomography angiography (CTA) or magnetic resonance angiography (MRA) and catheter intra‐arterial digital subtraction angiography (IADSA), and have no contraindications to either test
Setting	Ideally, the intended setting is a population‐based representation of all patients with intracerebral haemorrhage. However, the included studies were all restricted to secondary and tertiary care from Asia (5 CTA, 2 MRA), North America (3 CTA), and Europe (1 MRA)
Index test	CTA or MRA
Reference standard	Catheter IADSA
Number of studies	8 cross‐sectional studies that evaluated CTA and IADSA in the same patient population and 3 cross‐sectional studies that evaluated MRA and IADSA in the same patient population
Test	Studies	Cases	Total	Summary sensitivity (95% CI)	Summary specificity (95% CI)
CTA	8	180	526	0.95 (0.90 to 0.97)	0.99 (0.95 to 1.00)
MRA	3	122	401	0.98 (0.80 to 1.00)	0.99 (0.97 to 1.00)
Comparison of CTA and MRA	In studies of variable methods and degrees of bias, the diagnostic accuracy of both CTA and MRA appear comparable to IADSA for detection of intracranial vascular malformations following intracerebral haemorrhage. There was no evidence to suggest a difference in sensitivity or specificity, or both, between CTA and MRA (P value = 0.6). However, there was no study that directly compared the accuracy of CTA and MRA
CAUTION: The results on this table should not be interpreted in isolation from the results of the individual included studies contributing to each summary test accuracy measure. These are reported in the main body of the text of the review

CI: confidence interval

Background

A stroke is a clinical syndrome characterised by a sudden loss of focal brain function causing symptoms that last more than 24 hours or lead to death. Strokes occur either because of inadequate blood supply to the brain (ischaemic stroke) or because of bleeding. Bleeding can be between the membranous layers that surround the brain or into the ventricles of the brain. However, bleeding directly into the brain substance (intracerebral haemorrhage or ICH) is a particularly devastating form of stroke. It accounts for 10% to 15% of all strokes in Caucasian populations (Lovelock 2007) and the one‐month case fatality is approximately 40% (Van Asch 2010).

It is extremely important to make the distinction between ICH and other forms of stroke since the causes, prognosis, and treatment vary according to each condition. No clinical scoring systems have yet been shown to be accurate in distinguishing between these entities. Therefore, timely brain imaging is required to differentiate ICH from ischaemic stroke and the other forms of haemorrhagic stroke (Al‐Shahi Salman 2009). Computed tomography (CT) and magnetic resonance imaging (MRI) have both been used in clinical practice. A Cochrane review of diagnostic test accuracy has confirmed the utility of CT for detecting acute haemorrhagic stroke but has concluded that insufficient information exists to establish the merits of MRI for this purpose (Brazzelli 2009).

If a scan demonstrates ICH, the priority shifts to investigating the cause of the bleed in order to estimate prognosis and direct treatment. Physicians tend to infer the cause of ICH on the basis of the patient's age, co‐morbidities, and ICH characteristics on CT or MRI. For instance, elderly patients with preceding hypertension and ICH in a 'deep' location in the brain are often assumed to have had a bleed unrelated to any underlying structural abnormality, while young patients with 'deep' ICH and no past history of high blood pressure are often investigated for an underlying vascular malformation (Cordonnier 2010). However, these assumptions may be inaccurate (Cordonnier 2010). Additional history, laboratory investigations, and sophisticated radiological tests may help to establish the cause of ICH.

Target condition being diagnosed

Intracranial vascular malformations are of particular importance as they are the leading cause of ICH in young adults (explaining as many as one‐third of cases in young people; Al‐Shahi 2001) and cause an appreciable proportion of ICH in older people (18% (95% CI 13% to 24%) of cases in people aged ≥ 50 in a recent systematic review; Cordonnier 2010). Intracranial vascular malformations include arteriovenous malformations, dural arteriovenous fistulae, and aneurysms (which are the target conditions in this review). Arteriovenous malformations are abnormal tangles of dilated arteries and veins of varying calibre that lack an intervening capillary network. The arteries and veins are connected in a central nidus (Latin nidus, nest). The lack of capillaries results in direct arteriovenous shunting from the high‐pressure arterial system to the low‐pressure venous system, which is not designed to accommodate radical fluctuations in pulse pressure. A dural arteriovenous fistula is similar to an arteriovenous malformation but is a direct, high‐flow fistula between the external carotid circulation and a dural venous sinus or cerebral cortical vein. An aneurysm is a weakened, focal protrusion of the arterial wall that is often found at a branching point of the cerebral blood vessels. These three types of vascular malformations confer appreciable risks of recurrent ICH, can be identified in the acute stages of ICH with brain imaging, and are often amenable to treatment intended to prevent rebleeding.

The traditional reference standard for diagnosing arteriovenous malformations, dural arteriovenous fistulae, and aneurysms is catheter intra‐arterial digital subtraction angiography (IADSA). This involves inserting a catheter into the femoral artery and guiding it back through the arterial circulation to the common, internal, or external carotid arteries or into the vertebral arteries. Pre‐contrast x‐ray images (the 'mask') are taken and then contrast dye is injected from the catheter into the cerebral artery in which it is positioned. Further images are taken during injection and the mask is then subtracted from the post‐contrast images. All that remains should be the blood vessels that were filled with contrast material. Images are taken in quick succession during the injection so that radiologists can evaluate blood flow through the arterial system into the venous system. Arteriovenous malformations are typically visualised on IADSA as tangles of abnormal vessels with enlarged feeding arteries and dilated, tortuous veins within the brain substance. Early venous filling during the arterial phase of contrast injection is characteristic of arteriovenous malformations. Dural arteriovenous fistulae appear as early venous filling during the arterial phase of contrast injection via direct connections between dural arteries and a venous sinus or cortical veins within the dura mater covering of the brain. Aneurysms are visualised on IADSA as contrast‐filled outpouchings that arise from an arterial wall or bifurcation, although thrombosed aneurysms may be more difficult to appreciate if there is limited contrast penetration.

Given the disparity in the prevalence of moya‐moya disease between Asian and non‐Asian populations (Scott 2009), we decided to exclude this condition as ethnicities are likely to vary within and between studies.

Index test(s)

Although IADSA has been used to diagnose these target conditions in the past, CT‐ and MRI‐based techniques are becoming increasingly popular in clinical practice. CT scanners use x‐rays to create a series of cross‐sectional images of the brain. CT is very useful for identifying acute haemorrhage but cannot reliably detect vascular malformations due to its inability to discriminate blood vessels clearly from the surrounding brain substance. MRI uses the electromagnetic properties of protons to produce high‐quality cross‐sectional images of the brain. Blood vessels are better appreciated on MRI as characteristic 'flow voids'.

More advanced CT and MRI techniques have recently been developed to image blood vessels separately from the brain. CT angiography (CTA) and MR angiography (MRA) exploit CT and MRI techniques to create high‐resolution images of the arteries of the brain that can be used to diagnose arteriovenous malformations, dural arteriovenous fistulae, and aneurysms. Both tests combine sophisticated imaging techniques, sometimes involving a precisely timed contrast infusion, to accentuate blood vessels during either the arterial or venous phase of contrast perfusion. Hundreds to thousands of images are obtained and reconstructed in a variety of planes so that blood vessels can be visualised in two and three dimensions.

Neither CTA nor MRA is known to be superior to the other. Currently the choice of test ultimately depends on its availability and the patient's ability to undergo the test; for example, patients may have contraindications to MRA or they may be unable to lie still for IADSA under local anaesthetic. The timing of these tests in relation to the ICH onset may influence diagnostic test accuracy, because the mass effect of ICH may obscure the underlying intracranial vascular malformation. Repeat IADSA, three or more weeks following ICH onset, can detect arteriovenous malformations that were missed on an initial IADSA performed during the acute stages of the bleed (Hino 1998).

Rationale

In patients who present to hospital with ICH, the swift and accurate identification of any underlying intracranial vascular malformation that may be the underlying cause is important. For instance, patients with ICH caused by arteriovenous malformations seem to fare better than their counterparts with 'primary' ICH, even after adjusting for known prognostic factors (Van Beijnum 2009). Early and accurate diagnosis could improve outcome if treatment prevents recurrent haemorrhage.

Catheter IADSA is the conventional reference standard technique for the diagnosis of arteriovenous malformations, dural arteriovenous fistulae, and aneurysms. However, IADSA is not available in all centres and, because a catheter is required for the procedure, it carries a small but significant risk of stroke, infection, haematoma formation at the catheter entry site, and pseudo‐aneurysm formation at the femoral artery puncture site. As a result of their widening availability, lower procedural risks, increasing resolution, and sophisticated reconstruction software, CTA and MRA are often used prior to, or instead of, IADSA in patients with a suspected intracranial vascular malformation following different types of intracranial haemorrhage.

Systematic reviews and meta‐analyses have already been performed to determine how the diagnostic accuracy of CTA and MRA compares with IADSA for the detection of aneurysms following subarachnoid haemorrhage (bleeding between the arachnoid and pial meningeal layers that surround the brain) (White 2000; Chappell 2003; Van Gelder 2003; Westerlaan 2011). When we registered our title and protocol for this Cochrane systematic review, there were no meta‐analyses of the diagnostic test accuracy of CTA and MRA versus IADSA following ICH. However, two systematic reviews have recently emerged (Ma 2012; Wong 2012), although they did not restrict data to the same reference standard (IADSA). Each review used different inclusion criteria and neither analysed MRA. The quality of included studies were not systematically assessed using validated measures such as the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool, heterogeneity among included studies was not analysed in one review (Wong 2012), and the target conditions differed from those evaluated in our analysis. Therefore, we systematically reviewed the literature for studies of the diagnostic accuracy of CTA or MRA compared with IADSA as a reference standard for the detection of intracranial vascular malformations underlying ICH.

Objectives

To evaluate the diagnostic test accuracy of CTA and MRA versus IADSA for the detection of intracranial vascular malformations as a cause of ICH.

Secondary objectives

To investigate the influence of the timing of the index and reference tests on diagnostic test accuracy.

Investigation of sources of heterogeneity

We planned to investigate the differences in diagnostic accuracy in subgroups defined by:

• age;

• ICH volume;

• timing of the index test(s) after symptom onset;

• CT technology (for example non‐helical CT scanners, helical CT scanners, and multi‐detector helical CT scanners) and MRI technology (for example MRI magnet strength and the type of angiographic technique used to image the arteries);

• the coverage of the brain that is investigated by CTA/MRA in each study (e.g. some users may only evaluate the region around the haematoma while others may choose to also evaluate vessels distant from the bleed);

• the number of arterial territories and orientations investigated by IADSA (e.g. the internal carotid artery territory, the external carotid artery territory, the vertebral artery territory, or all three);

• whether three‐dimensional angiography was used;

• differences in the qualifications and experience of the radiologists reporting the index tests and reference standards.

Methods

Criteria for considering studies for this review

Types of studies

Single or comparative test accuracy studies of CTA or MRA, or both (index tests) versus IADSA (reference standard) in patients with radiographically verified ICH were eligible for inclusion as long as participants underwent at least one index test as well as the reference standard. We included retrospective and prospective case series and cohort studies of consecutive patients (Gluud 2005), irrespective of their date of publication, country, or language of origin. There were no restrictions based on the medical setting in which the tests were performed. We set a minimum overall sample size per study of 20 or more participants. We accepted studies that had fewer than 20 individuals undergoing IADSA as long as the overall study size was greater than 20 participants.

Participants

We included all participants with ICH (as defined by study authors), who were investigated with one or more index tests and a reference standard. Participants were included irrespective of the severity of their disease as long as they were stable enough to undergo an index test and a reference standard.

Index tests

We assessed the following index tests.

1. Computed tomography angiography (CTA).

2. Magnetic resonance angiography (contrast enhanced or non‐enhanced) (MRA).

Comparator tests

We sought single test accuracy studies that evaluated CTA or MRA against the IADSA reference standard, as well as comparative studies of CTA versus MRA against the IADSA reference standard.

Target conditions

The target conditions were aneurysms, arteriovenous malformations, or pial/dural arteriovenous fistulae that have caused ICH. We re‐classified patients with moya‐moya as 'non‐diseased' for the purposes of the review.

Reference standards

A single reference standard for the diagnosis of intracranial vascular malformations does not exist. In clinical practice, however, expert assessment of IADSA is considered the most valid test and therefore we only included studies that used this test as the reference standard in the review. If studies had used more than one reference standard, we only included these studies if they provided data on patients who had been investigated with IADSA.

Search methods for identification of studies

Electronic searches

We searched the following electronic bibliographic databases.

• MEDLINE (Ovid) (1948 to August 2013) (Appendix 1).

• EMBASE (Ovid) (1980 to August 2013) (Appendix 2).

• Database of Abstracts of Reviews of Effects (DARE) (www.crd.york.ac.uk/crdweb/) (August 2013).

• Health Technology Assessment Database (HTA) (www.crd.york.ac.uk/crdweb/) (August 2013).

• MEDION (www.mediondatabase.nl/) (August 2013) using the 'Systematic Reviews and Diagnostic Studies' search filter, the ICPC code = 'Neurological' and the signssymp = 'Medical Imaging'.

• ClinicalTrials.gov (August 2013).

• WHO ICTRP (International Clinical Trials Register Portfolio) (August 2013).

We developed comprehensive search strategies for MEDLINE and EMBASE with the help of the Cochrane Stroke Group Information Specialist and adapted these for the other databases. We searched for all relevant studies and did not restrict our searches by date, institution, or language.

Searching other resources

We searched the reference lists and performed a cited reference search for forward tracking of relevant articles on Google Scholar (http://scholar.google.com/) (August 2013) for all articles that we reviewed for eligibility during the full‐text review phase.

Data collection and analysis

Selection of studies

One author (CBJ) reviewed titles and abstracts of the records identified from the electronic searches and excluded obviously irrelevant citations. We then obtained the full copy of the remaining papers and the same author together with a second review author (PMW or RA‐SS) assessed these to identify relevant studies that met the inclusion criteria for the review. If there was any uncertainty about a study's eligibility we reached a decision through discussion. If there was still uncertainty, we contacted the study authors to ask them to provide the relevant information necessary to resolve the uncertainty. We were not blinded to study authors, institution, and study results during the selection process.

Data extraction and management

Two of three review authors (CBJ and PMW or RA‐SS) independently extracted data using a standardised data collection form. We accepted the authors' definitions of a positive index/comparator/reference test result for arteriovenous malformations, dural arteriovenous fistulae, and aneurysms.

The absolute numbers of observations of true positives, false positives, true negatives, and false negatives had to be specified or had to be able to be derived from the available data in order to be formally meta‐analysed. We contacted study authors if this was not possible; if these data could not be obtained, despite contacting the authors, then we excluded the study. We excluded studies if they were a reanalysis or republication of data from a study population that was already included in the review.

We compared the data abstracted by two review authors (CBJ and PMW or RA‐SS) and resolved any disagreements through discussion. We used the extracted raw data to construct 2 x 2 contingency tables for outcomes defined as any of the target conditions detected. We used the 2 x 2 contingency tables to calculate sensitivity and specificity for each index test in all included studies using RevMan 5.2 (RevMan 2012).

We extracted the following additional study‐level attributes.

1. General information: title, journal (including volume and pages), year, institution and country, language, and study design.

2. Population sampling: number of participants screened, number eligible, number enrolled, and number undergoing both the index tests (CTA or MRA, or both) and the reference standard (IADSA).

3. Demographic characteristics: participant age, ethnicity, number with pre‐existing hypertension, number with a prior intracranial haemorrhage.

4. Radiological description of the ICH: volume, location, and extension of the intracerebral haemorrhage.

5. Index test parameters for: (1) CTA: manufacturer and model, equipment (non‐helical CT, helical CT, or multi‐detector CT), slice thickness, pitch, matrix, field of view, coverage, contrast agent, delay to contrast infusion, contrast volume and rate of infusion, reformatting technique and image format; and (2) MRA: manufacturer and model, magnet strength, sequences, contrast agent and rate of infusion, slice thickness, matrix, field of view, and coverage. We also recorded the number of indeterminate scans and the number and types of adverse events.

6. Reference standard parameters for IADSA: manufacturer and model, the arterial territories imaged (unilateral or bilateral internal carotids, unilateral or bilateral external carotids, and unilateral or bilateral vertebral arteries), number of projections (views) per artery, use of three‐dimensional angiography, frame rate, and matrix. We recorded the number of indeterminate results and the number and types of adverse events.

We contacted the authors of studies that might have been subject to differential verification bias because they had used more than one reference standard. We requested the raw data in order to recalculate the 2 x 2 contingency tables according to our definition of the target condition. We constructed the new contingency tables using only IADSA as the reference standard. We therefore excluded participants who underwent a reference standard other than IADSA from the analysis.

There was likely to be inter‐study variation in the positivity threshold due to the fact that scans must be interpreted based on judgement rather than on an explicit quantitative cut‐point (Macaskill 2010). We therefore planned to organise the data according to different thresholds if there were obvious discrepancies in the definition of a positive test result. We planned to exclude uninterpretable scans from the contingency tables and evaluate them in a sensitivity analysis if we encountered this issue.

Assessment of methodological quality

We assessed the methodological quality of each study using a modified version of the QUADAS tool (Whiting 2003). QUADAS appraises study quality by indicating the presence or absence of 14 key criteria through a series of questions that are answered as 'yes', 'no', or 'unclear'. The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy advises omitting three items that relate mainly to the quality of reporting rather than the quality of the methodology.

In addition to the Cochrane recommended QUADAS questions, we have added an extra criterion to evaluate whether those interpreting the index test were of an appropriate level of training that we have defined as either an 'expert' radiologist (a neuroradiologist) or an 'experienced' radiologist (consultant level or equivalent radiologist with five or more years of clinical practice).

This modified 12‐item quality assessment form is available as Appendix 3.

Two review authors (CBJ and RA‐SS) independently assessed study quality and a third review author (PMW) checked the index test and reference standard attributes. Disagreements were mediated through arbitration.

Statistical analysis and data synthesis

We calculated sensitivity and specificity, with 95% confidence intervals (CI), for each index test in all studies. We used RevMan 5.2 to create coupled forest plots to visually evaluate the variation in the estimates of sensitivity and specificity between studies (RevMan 2012). We plotted the results in receiver operator characteristic (ROC) space. Sensitivity was used to define the y‐axis, specificity defined the x‐axis, and each point on the plot represents the proportion of true positives amongst those with our defined target conditions against the proportion of false positives amongst those lacking our defined target conditions for one particular study.

We planned to conduct a meta‐analyses of study‐specific pairs of sensitivity and specificity to calculate pooled estimates of sensitivity and specificity using the bivariate method (Macaskill 2010).

We performed the analyses using RevMan 5.2 (RevMan 2012) and obtained pooled estimates using the Metandi package within the Stata Statistical Software version 12.1 (StataCorp 2011).

Investigations of heterogeneity

We planned to address heterogeneity by adding covariates of interest to the bivariate model. As described in the introduction, covariates included equipment parameters for CTA, MRA, and IADSA, the experience of those reading the index test and reference standard, the timing of the index test with respect to symptom onset, the average participant age, and the average haematoma size.

Sensitivity analyses

We planned sensitivity analyses to evaluate the robustness of our eligibility criteria by exploring the effect of including only adults (as opposed to all age groups) on the pooled sensitivity and specificity results. We also planned to undertake sensitivity analyses for each of the 12 modified QUADAS criteria described in Appendix 3 to determine the effect of poor study quality on the overall results. If feasible, we intended to perform a sensitivity analysis on the effect of excluding uninterpretable index test results by instead re‐classifying them as a negative result. We planned to perform these analyses when more than three studies were available for each strata.

Assessment of reporting bias

We contacted the authors of the studies that were excluded because they did not report specific outcome measures of interest to inquire whether these data were available but had not been published. If data were available and met the inclusion criteria, we included them in our analysis.

We planned to assess publication bias using the method described by Deeks 2005 if there were a sufficient number of studies available.

Results

Results of the search

The MEDLINE and EMBASE searches identified 4472 unique citations. Of these, we considered 37 relevant to the purpose of our review. We retrieved full‐text articles and subsequently excluded 26 following review. We were unsure about whether two articles could be included but ultimately excluded them since the corresponding author did not reply to our requests for additional information (Sasiadek 2000; Sasiadek 2002). We excluded the remaining articles because the index test was not computed tomography angiography (CTA)/magnetic resonance angiography (MRA) (n = 2) (Pott 1992; Kadkhodayan 2012), intra‐arterial digital subtraction angiography (IADSA) was not the reference standard (n = 8) (Awad 1992; Hünerbein 2003; Lee 2007; Elhammady 2008; Sha 2008; Bekelis 2012; Wijman 2012; Kamel 2013), there was no comparison between CTA/MRA and IADSA (n = 1) (Zheng 2012), the article was a review rather than original research (n = 6) (Bowen 2007; Truwit 2007; Kidwell 2010; Campeau 2012; Chandra 2012; Khosravani 2013), the overall study size was smaller than 20 participants (n = 1) (Evans 2005), or the study was not performed on patients presenting with ICH (n = 6) (Fasulakis 2003; Dammert 2004; Griffiths 2006; Gross 2012; Leung 2012; Li 2013).

Eleven studies, each published as an individual report, comprising a total of 927 participants, met our inclusion criteria (Figure 1). Eight studies compared CTA with IADSA (Eshwar Chandra 1998; Murai 1999; Delgado Almandoz 2009; Romero 2009; Yeung 2009; Yoon 2009; Wong 2011; Ma 2012) and three studies compared MRA with IADSA (Wong 2010; Lummel 2012; Zhou 2012). The details of included studies are described in the Characteristics of included studies table.

1.

Study flow diagram.

Methodological quality of included studies

The quality of the 11 included studies varied considerably when evaluated according to our modified QUADAS criteria (Figure 2). Five were prospective (all CTA studies, 354 participants), five were retrospective (three CTA studies, 172 participants; two MRA studies, 218 participants), and the design of one study was unclear (MRA, 183 participants). All studies apart from one failed to provide a formal definition of a positive scan apart from general statements about identification of a vascular or 'secondary' cause for the ICH.

2.

Methodological quality summary: review authors' judgements about each methodological quality item for each included study.

Representative spectrum

CTA

Eight studies comprising a total of 526 participants compared CTA with IADSA. All studies described their inclusion criteria. However, only two studies comprising a total of 133 participants (Figure 2) met our definition of a representative spectrum (Appendix 3).

Sample size for all eight studies ranged from 31 to 210. The mean or median age of participants in each study ranged from 28 to 56 years. Two studies included participants under 16 years of age (Eshwar Chandra 1998; Romero 2009). Two studies did not document mean or median participant age (Delgado Almandoz 2009; Ma 2012). The proportion of females ranged from 33% to 54%. Two studies did not document gender distribution (Delgado Almandoz 2009; Ma 2012). One study reported the proportion of participants with pre‐existing hypertension at the time of ICH (14%) (Romero 2009). No study indicated how many participants had experienced prior ICH.

Three studies documented the types of intracranial haemorrhage at the time of presentation (Romero 2009; Yoon 2009; Wong 2011). The proportion presenting with 'pure' ICH ranged from 10% to 56%. The proportion presenting with ICH and subarachnoid haemorrhage (SAH) ranged from 9% to 39%, the proportion presenting with ICH and intraventricular haemorrhage (IVH) ranged from 10% to 34%, and the proportion presenting with ICH and a subdural haematoma (SDH) ranged from 3% to 17%. Two studies (Romero 2009; Yoon 2009) contained participants who presented with ICH combined with both SAH and IVH (23% and 8% of participants respectively). One study contained participants with ICH and simultaneous SAH, IVH, and SDH (5%) (Romero 2009). Five studies reported the location of the presenting ICH (Eshwar Chandra 1998; Romero 2009; Yeung 2009; Yoon 2009; Wong 2011). The presenting ICH was lobar in 56% to 100% of cases, deep in 0% to 19% of cases, and infratentorial in 0% to 21% of cases. No study reported the mean or median ICH volume at presentation.

MRA

Three studies comprising a total of 401 participants compared MRA with IADSA. The inclusion and exclusion criteria were clearly described and the patient population met our definition of a representative spectrum in two of three studies (Appendix 3). The mean age ranged from 42 years (standard deviation (SD) 15) to 53.4 years (range 14 to 82). The proportion of females ranged from 34% to 54%. Participant withdrawals were explained but there was no documentation about whether adverse events were encountered. All participants could be accounted for in the studies. The proportion with pre‐existing hypertension was 28% and 34% in the two studies that reported these data (Wong 2010; Zhou 2012). It is unclear whether any of the participants had experienced prior ICH.

One study documented the types of intracranial haemorrhage at the time of presentation (Wong 2010). Nineteen patients presented with ICH and concomitant IVH. The presenting ICH was lobar in 53% of cases, deep in 32% of cases, and infratentorial in 15% of cases. The mean or median ICH size at the time of presentation was not reported.

Acceptable reference standard

CTA

We deemed the reference standard to be of unclear quality in four studies (Eshwar Chandra 1998; Murai 1999; Delgado Almandoz 2009; Ma 2012), which did not document the equipment, technical specifications, and the level of expertise of those interpreting the IADSA in sufficient detail. A neuroradiologist interpreted the IADSA in five studies (Romero 2009; Yeung 2009; Yoon 2009; Wong 2010; Ma 2012); the expertise of those interpreting the IADSA was not documented in three studies (Eshwar Chandra 1998; Murai 1999; Delgado Almandoz 2009), and one study used a combination of an expert neuroradiologist and a neurosurgeon (Wong 2011).

MRA

We deemed the reference standard to be of unclear quality in one study (Lummel 2012), which did not document the equipment, technical specifications, and the level of expertise of those interpreting the IADSA in sufficient detail. A neuroradiologist interpreted the IADSA in two studies (Wong 2010; Lummel 2012), while the level of expertise of those interpreting the IADSA was not documented in the third study (Zhou 2012).

Acceptable delay between tests

CTA

Three studies documented time from ICH symptom onset to the CTA (one study had a mean of six hours from symptom onset to CTA (Murai 1999), one studied required that the CTA be performed ≤ 24 hours from ICH onset (Delgado Almandoz 2009), one study required the CTA to be performed ≤ 120 hours from ICH onset (Wong 2011)), and one study documented the time from ICH symptom onset to IADSA (IADSA had to be completed ≤ 144 hours from ICH onset (Wong 2011)). Three studies reported on the mean or median interval from CTA to IADSA. All IADSA examinations were performed within 48 hours of the CTA in one study (Romero 2009), IADSA was performed a median of two days from the CTA in one study (Yeung 2009), and there was a mean of 16.1 hours (SD 9 hours) between the CTA and IADSA in the third study (Yoon 2009).

MRA

The time from ICH symptom onset to both the index test and reference standard ranged from six to 18 weeks in the one MRA study that reported these values (Wong 2010).

Partial verification bias avoided

CTA

Three studies avoided partial verification bias by prospectively recruiting patients with the intention that each participant would undergo both CTA and IADSA (Yoon 2009; Wong 2011; Ma 2012). It was unclear whether the results of the CTA influenced the selection of those undergoing IADSA in one study (Eshwar Chandra 1998). The CTA result influenced the timing of IADSA in one prospective study (Murai 1999), and certainly would have influenced the decision to perform IADSA in the three retrospective studies (Delgado Almandoz 2009; Romero 2009; Yeung 2009).

MRA

Partial verification bias would be expected to be present in all three retrospective studies (Wong 2010; Lummel 2012; Zhou 2012), since the results of the MRA would certainly influence the decision to proceed to IADSA in routine clinical practice.

Differential verification bias

CTA

Three studies used a combination of IADSA and pathology or operative findings as a reference standard (Eshwar Chandra 1998; Delgado Almandoz 2009; Romero 2009). We contacted study authors and extracted data related only to those participants undergoing both CTA and IADSA. This may have introduced a form of selection bias as those undergoing pathological confirmation may have had larger ICH volumes (leading to surgery or death) that could have obscured the target condition on imaging. However, it does remove the influence of differential verification bias. All other studies used IADSA as the exclusive reference standard.

MRA

All studies restricted the reference standard to IADSA.

Incorporation bias

Incorporation bias was avoided in all 11 included studies.

Reference standard result blinded

CTA

Three of five prospective studies explicitly mention that IADSA was interpreted blinded to the index result (Yoon 2009; Wong 2011; Ma 2012). Two neuroradiologists in one retrospective study read the IADSA at least one month after the CTA (Yeung 2009). However, while this would reduce the risk of recall bias, it would not eliminate it altogether. One retrospective study reported that the IADSA scans were reported with knowledge of the CTA results (Romero 2009).

MRA

One retrospective study explicitly mentioned that IADSA was interpreted blinded to the index result (Wong 2010). The blinding status was unclear in one study (Zhou 2012) and the final study involved interpretation of the IADSA with knowledge of the results of the MRA (Lummel 2012).

Index test result blinded

CTA

The index test was interpreted blinded to the results of the IADSA in six studies (Delgado Almandoz 2009; Romero 2009; Yeung 2009; Yoon 2009; Wong 2011; Ma 2012), while no statement to this effect was made in the remaining two studies (Eshwar Chandra 1998; Murai 1999).

MRA

The index test was interpreted blinded to the results of the IADSA in two studies (Wong 2010; Lummel 2012), while no statement to this effect was made in the remaining study (Zhou 2012).

Relevant clinical information

CTA

Five studies required that the CTA and IADSA were interpreted blinded to the participant's clinical information (Delgado Almandoz 2009; Romero 2009; Yeung 2009; Wong 2011; Ma 2012), while the three remaining studies did not address this point (Eshwar Chandra 1998; Murai 1999; Yoon 2009).

MRA

Two studies required that the MRA and IADSA were interpreted blinded to the participant's clinical information (Wong 2010; Lummel 2012), while the one remaining study did not address this point (Zhou 2012).

Uninterpretable results reported

No study documented whether uninterpretable images were encountered.

Withdrawals explained

No study reported whether there were participants who had to withdraw due to adverse events or due to an inability to tolerate the index test or reference standard. All participants could be accounted for in each study but the process by which the final study population was reached was unclear in three studies (Eshwar Chandra 1998; Yeung 2009; Wong 2011).

Index tests interpreted by an expert

CTA

A neuroradiologist interpreted the CTA in six studies. CTA was interpreted by either a neuroradiologist or neurosurgeon in one study (Murai 1999) and the level of expertise of those interpreting the dynaCT scan was not clear in one study (Eshwar Chandra 1998).

MRA

MRA images were interpreted by neuroradiologists or neurosurgeons in two studies (Wong 2010; Zhou 2012) and were interpreted exclusively by neuroradiologists in the final study (Lummel 2012).

Findings

Studies on CTA

Figure 3 shows the paired forest plot for sensitivity and specificity for CTA compared with IADSA. The pooled estimates of sensitivity and specificity were 0.95 (95% CI 0.90 to 0.97) and 0.99 (95% CI 0.95 to 1.00) respectively (Figure 4). Approximately 85% of the estimates of specificity were close to or at the 'ceiling level' (specificity of 1.00). The pooled positive likelihood ratio was 73 (95% CI 19 to 277) and the negative likelihood ratio was 0.06 (95% CI 0.03 to 0.10).

3.

Forest plot of the paired sensitivity and specificity values for the detection of an intracranial vascular malformation following intracerebral haemorrhage using computed tomography angiography (CTA) or magnetic resonance angiography (MRA) compared to a reference standard of catheter intra‐arterial digital subtraction angiography.

4.

Pooled estimates of sensitivity and specificity for computed tomography angiography (black) and magnetic resonance angiography (red) plotted in receiver operator characteristic space of studies compared with catheter intra‐arterial digital subtraction angiography for the detection of intracranial vascular malformations following intracerebral haemorrhage.

The results remained robust in a sensitivity analysis in which six studies that recruited only adult patients, comprising 465 participants, were evaluated separately (Murai 1999; Delgado Almandoz 2009; Yeung 2009; Yoon 2009; Wong 2011; Ma 2012). The pooled estimates of sensitivity and specificity were 0.95 (95% CI 0.89 to 0.98) and 0.99 (95% CI 0.95 to 1.00).

We performed sensitivity analyses when more than three studies for each analysis were available. The results remained robust in the following pre‐specified sensitivity analyses where only those meeting each study criterion were analysed.

• Acceptable reference standard (four studies, 251 participants): sensitivity 0.99 (95% CI 0.79 to 1.00) and specificity 0.98 (95% CI 0.88 to 1.00).

• Delay between tests reported (four studies, 251 participants): sensitivity 0.99 (95% CI 0.79 to 1.00) and specificity 0.98 (95% CI 0.88 to 1.00).

• Partial verification bias avoided (four studies, 323 participants): sensitivity 0.94 (95% CI 0.90 to 0.97) and specificity 0.99 (95% CI 0.96 to 1.00).

• Index test results blinded (six studies, 451 participants): sensitivity 0.96 (95% CI 0.90 to 0.99) and specificity 0.99 (95% CI 0.94 to 1.00).

• Index test interpreted by an expert (seven studies, 482 participants): sensitivity 0.96 (95% CI 0.90 to 0.98) and specificity 0.99 (95% CI 0.95 to 1.00).

We were unable to perform sensitivity analyses for differential verification bias (all studies met the criterion), incorporation bias (all studies met the criterion), relevant clinical information provided to those interpreting the test (no study met this criterion), reporting and reclassifying uninterpretable images as negative (no study met this criterion), and explanation of study withdrawals (all studies met this criterion).

Although there were sufficient data available (more than 85% complete for all studies) to address heterogeneity between studies according to gender, mean or median age, equipment parameters, and experience of those interpreting the CTA, the shortage of available studies (n = 8) precluded formal analysis of heterogeneity. However, there was no indication of heterogeneity based on visual inspection of the forest and ROC plots and the low variance estimates. There were insufficient data to assess heterogeneity according to the equipment parameters of IADSA, the experience of those interpreting the IADSA studies, the timing of the CTA with respect to symptom onset, and average haematoma size.

No studies reported adverse events.

We did not formally evaluate publication bias due to the limited number of studies (Deeks 2005).

Studies on MRA

Figure 3 shows the paired forest plot for sensitivity and specificity for MRA compared with IADSA. The pooled estimates of sensitivity were 0.98 (95% CI 0.80 to 1.00) and for specificity were 0.99 (95% CI 0.97 to 1.00) for the three studies (Figure 4). The corresponding positive likelihood ratio was 91 (95% CI 22 to 376) and the negative likelihood ratio was 0.02 (95% CI 0.002 to 0.234).

We could not perform our pre‐specified sensitivity analyses due to the limited number of studies.

No studies reported adverse events.

We did not formally evaluate publication bias due to the limited number of studies (Deeks 2005).

Comparison of CTA and MRA

Using all included studies, we compared the diagnostic accuracy of CTA and MRA by adding covariate terms for test type to the parameters of the bivariate model to determine the effect of test type on sensitivity and specificity. We used a likelihood ratio test to assess the statistical significance of the difference in sensitivity and specificity between tests by comparing models with and without the covariate terms in the bivariate model. The likelihood ratio test indicated no evidence (P value = 0.6) of a difference in sensitivity or specificity, or both between CTA and MRA (Figure 4; Table 1). Indirect comparisons can be prone to confounding, however, and direct comparisons between CTA and MRA are currently lacking.

Discussion

This systematic review and meta‐analysis identified eight studies comparing the diagnostic accuracy of computed tomography angiography (CTA) with intra‐arterial digital subtraction angiography (IADSA) and three studies comparing magnetic resonance angiography (MRA) with IADSA. The pooled estimates of sensitivity and specificity were 0.95 (95% confidence interval (CI) 0.90 to 0.97) and 0.99 (95% CI 0.95 to 1.00) respectively for CTA and 0.98 (95% CI 0.89 to 1.00) and 0.99 (95% CI 0.98 to 1.00) respectively for MRA. Studies were of variable methodological quality with discrepant inclusion criteria, retrospective designs, and high rates of partial verification bias.

Summary of main results

The Table 1 summarises our findings for the eight included studies comparing CTA with IADSA and the three studies comparing MRA with IADSA. CTA and MRA appear equivalent to IADSA for the detection of intracranial vascular malformations following intracerebral haemorrhage (ICH). However, it is important to consider that sample sizes were typically small and study designs varied. High rates of partial verification bias (50%) complicate evaluation of sensitivity. Inflated estimates of sensitivity would be expected in situations where the index test is used to select participants for the reference standard. Furthermore, differential verification bias had complicated studies that used IADSA, pathology, or a combination of pathology and IADSA as a reference standard; we mitigated this effect by restricting our analyses to participants in whom IADSA had been the reference standard.

The majority of included studies were 'streamlined' to select a patient population with the greatest likelihood of harbouring an underlying vascular malformation or aneurysm. Focused exclusion of patients with alternative diagnoses can falsely elevate specificity since disease mimics are preferentially excluded. Studies excluding patients with pre‐existing hypertension or deep haemorrhages, a population of patients with a lower yield of aneurysms and arteriovenous malformations on IADSA performed following ICH (Cordonnier 2010), would be particularly prone to this phenomenon. In addition, broad selection criteria are required as certain subgroups of patients, such as those older than 50, have been found to have an appreciable yield of vascular malformations and aneurysms on IADSA following ICH despite a common perception to the contrary (Cordonnier 2010).

The timing of the CTA, MRA, and IADSA, both in respect to each other and from the onset of symptoms, would also be expected to exert a major influence on overall estimates of diagnostic accuracy. The haemorrhage could obscure the vascular malformation or aneurysm if the scan is performed too early. Almost half of the studies did not report time from symptom onset to CTA, only one study recorded the time from symptom onset to IADSA, and less than half reported the delay between CTA and IADSA. Likewise, although all three MRA studies reported time from symptom onset to index test, none of them reported the delay between the MRA and IADSA. The optimal timing of scans has not been established but reporting duration and correlating it with accuracy is a crucial step to resolving this uncertainty.

Strengths and weaknesses of the review

We used a comprehensive search strategy on eight major electronic databases without any restrictions on date, language, or country of origin. We identified studies both with and without a diagnostic test accuracy filter incorporated into the search strategy.

We minimised patient overlap by only selecting the most recent study published where there were many from the same patient cohort. Two review authors evaluated selected full studies for inclusion and a combination of two of three review authors (CBJ and PMW or RA‐SS) with complementary expertise (two neurologists and one neuroradiologist) independently extracted data and assessed study quality.

The main limitations pertain to the dearth of studies and quality of the evidence. We identified few studies, methodological quality was variable, sample sizes were small, and there was incomplete reporting of patient and test characteristics.

These issues are common in studies of diagnostic test accuracy and tend to lead to overestimation of diagnostic performance, especially when performed using a narrow spectrum of patients (Lijmer 1999; Rutjes 2006). Adherence to the Standards for Reporting of Diagnostic Accuracy (STARD) guidelines will facilitate optimal design and reporting of future studies (Bossuyt 2003).

We did not identify any comparative test accuracy studies (i.e. studies that compared CTA with MRA using IADSA as the reference standard). Due to limited studies and minimal heterogeneity we were not able to perform in‐depth sensitivity analyses evaluating the contribution of study quality to the overall estimates of sensitivity and specificity. Similarly, we could not evaluate all prespecified features of interest for heterogeneity due to inconsistent reporting. We also could not investigate heterogeneity and publication bias due to insufficient studies and small sample sizes.

We were unable to evaluate practicality and applicability formally due to a lack of information on patient tolerability, the frequency with which uninterpretable scans were encountered, the consequences of false negative diagnoses, and measures of cost‐effectiveness.

Applicability of findings to the review question

Both CTA and MRA appear comparable to IADSA for the detection of intracranial vascular malformations following ICH. However, routine use of CTA and MRA will need to take into account practicality and cost‐effectiveness. CTA is quicker and less invasive than IADSA. Patient tolerability is therefore improved, making it a more practical option for critically ill patients. It is also more widely available and more easily accessible that IADSA. Likewise, MRA is also less invasive and, though not as appropriate for critically ill patients, its adverse event profile compares favourably to CTA since it does not require radiation. Although less available than CTA, it is at least as accessible as IADSA. IADSA is a more demanding procedure requiring operators with highly specialised training in interventional techniques. However, although CTA and MRA are relatively low‐risk procedures, they can cause contrast reactions and the overall risk of adverse events has yet to be formally compared with IADSA. For instance, in certain circumstances, patient radiation exposure may be greater with CTA compared with IADSA (Manninen 2012).

The consequences of a false negative diagnosis have yet to be evaluated. A missed intracranial vascular malformation may predispose the patient to recurrent ICH or may simply necessitate IADSA, which potentially negates the value of CTA or MRA as first‐line imaging modalities. Neuroradiologists in the included studies were routinely denied access to basic clinical information, creating an artificial situation that is not representative of routine clinical practice. A practical benefit to IADSA is that, unlike CTA and MRA, images can be reviewed in 'real time' and adjustments can therefore be made to enhance image quality prior to the completion of the test. A cost‐effectiveness analysis comparing CTA or MRA with IADSA is lacking.

Although there appeared to be very little heterogeneity between studies according to CTA equipment parameters, it is possible that the accuracy of CTA compared with IADSA may further improve with advancing technology. The two earliest studies were performed using single‐slice CT scanners that are now obsolete in high‐income countries (Eshwar Chandra 1998; Murai 1999). The expectation therefore will be that the summary estimate of the diagnostic accuracy of CTA will continue to compare favourably with IADSA as more evidence accrues from studies benefiting from increasingly advanced technology.

Authors' conclusions

Implications for practice.

In conclusion, we evaluated eight studies that compared computed tomography angiography (CTA) with intra‐arterial digital subtraction angiography (IADSA) and three studies that compared magnetic resonance angiography (MRA) with IADSA for the detection of intracranial vascular malformations following intracerebral haemorrhage (ICH). Studies were of variable methodological quality. Our results suggest that CTA and MRA are probably of a comparable diagnostic accuracy to IADSA in patients with ICH. However, these estimates should be interpreted with caution because they are based on a few studies, of variable methodological quality, with small sample sizes. Neither practicality nor cost‐effectiveness was evaluated in the included studies. Additional, well‐designed studies are needed to confirm whether CTA or MRA can be used as the primary imaging modality for diagnosing intracranial vascular malformations following ICH.

Implications for research.

Future research should focus on the diagnostic performance of CTA and MRA in a broad spectrum of patients following ICH. These studies should also investigate the diagnostic test accuracy of technical advances in MRA, such as high‐resolution, contrast‐enhanced and time‐resolved techniques (Hadizadeh 2008; Taschner 2008), and in CTA, such as 4D‐CTA using 320 detector row computed tomography (CT) scanners (Willems 2011; Willems 2012). Additional analyses will be required to evaluate the cost‐effectiveness of these tests, and their impact on patients' overall outcome. We were unable to perform a direct comparison of CTA and MRA (using IADSA as the reference standard) due to limited data, but future studies should address this issue.

Future studies should focus on the implications of false negative and false positive results. These two relative misclassification costs are rarely equal (Mallett 2012). A false negative CTA or MRA result leaves the patient at risk of recurrent haemorrhage if IADSA is not performed. Alternatively, a false positive will usually not result in harm from inappropriate treatment (surgery, radiosurgery, or embolisation) because IADSA is usually done before any procedure. Under these circumstances, a higher sensitivity at the relative expense of specificity may be optimal.

Appendices

Appendix 1. MEDLINE search strategy

1. exp basal ganglia hemorrhage/ or intracranial hemorrhages/ or cerebral hemorrhage/ or intracranial hemorrhage, hypertensive/ or cerebrovascular disorders/
 2. ((brain$ or cerebr$ or cerebell$ or intracerebral or intracran$ or parenchymal or intraparenchymal or intraventricular or infratentorial or supratentorial or basal gangli$ or putaminal or putamen or posterior fossa or hemispher$ or stroke or apoplex$) adj10 (h?emorrhage$ or h?ematoma$ or bleed$)).tw.
 3. 1 or 2 or ICH.tw.
 4. Magnetic Resonance Angiography/
 5. angiography/ or cerebral angiography/
 6. Magnetic Resonance Imaging/
 7. 5 and 6
 8. ((magnetic resonance or MR or MRI or NMR) adj5 (angiogra$ or arteriogra$)).tw.
 9. MRA.tw.
 10. exp Tomography, X‐Ray Computed/
 11. angiography/ or cerebral angiography/
 12. 10 and 11
 13. ((Compute$ tomograph$ or CT or CAT) adj5 (angiogra$ or arteriogra$)).tw.
 14. CTA.tw.
 15. 4 or 7 or 8 or 9 or 12 or 13 or 14
 16. angiography, digital subtraction/
 17. angiography/ and (subtraction technique/ or subtraction.tw.)
 18. ((digital subtract$ or catheter or cerebral or brain) adj5 (angiogra$ or arteriogra$)).tw.
 19. (DSA or IADSA).tw.
 20. 16 or 17 or 18 or 19
 21. 3 and 15 and 20
 22. exp *basal ganglia hemorrhage/di, pa, ra or *intracranial hemorrhages/di, pa, ra or *cerebral hemorrhage/di, pa, ra or *intracranial hemorrhage, hypertensive/di, pa, ra or *cerebrovascular disorders/di, pa, ra
 23. 15 and 22
 24. exp "sensitivity and specificity"/
 25. (sensitiv$ or specificity).tw.
 26. (predictive adj5 value$).tw.
 27. exp diagnostic errors/
 28. ((false adj positive$) or (false adj negative$)).tw.
 29. (observer adj variation$).tw.
 30. (roc adj curve$).tw.
 31. (likelihood adj3 ratio$).tw.
 32. likelihood function/
 33. 24 or 25 or 26 or 27 or 28 or 29 or 30 or 31 or 32
 34. 3 and 15 and 33
 35. 21 or 23 or 34
 36. 20 and 22
 37. 3 and 20 and 33
 38. 35 or 36 or 37

Appendix 2. EMBASE search strategy

1. basal ganglion hemorrhage/ or brain hemorrhage/ or brain ventricle hemorrhage/ or cerebellum hemorrhage/ or brain hematoma/
 2. ((brain$ or cerebr$ or cerebell$ or intracerebral or intracran$ or parenchymal or intraparenchymal or intraventricular or infratentorial or supratentorial or basal gangli$ or putaminal or putamen or posterior fossa) adj10 (haemorrhage$ or hemorrhage$ or haematoma$ or hematoma$ or bleed$)).tw.
 3. 1 or 2 or ICH.tw.
 4. magnetic resonance angiography/
 5. angiography/ or arteriography/ or exp brain angiography/
 6. nuclear magnetic resonance imaging/
 7. 5 and 6
 8. ((magnetic resonance or MR or MRI or NMR) adj5 (angiogra$ or arteriogra$)).tw.
 9. MRA.tw.
 10. computed tomographic angiography/
 11. angiography/ or arteriography/ or exp brain angiography/
 12. computer assisted tomography/ or spiral computer assisted tomography/
 13. 11 and 12
 14. ((compute$ tomograph$ or CT or CAT) adj5 (angiogra$ or arteriogra$)).tw.
 15. CTA.tw.
 16. 4 or 7 or 8 or 9 or 10 or 13 or 14 or 15
 17. conventional angiography/ or digital subtraction angiography/
 18. image subtraction/ or subtraction.tw.
 19. angiography/ or arteriography/ or exp brain angiography/
 20. 18 and 19
 21. ((digital subtract$ or catheter or cerebral or brain) adj5 (angiogra$ or arteriogra$)).tw.
 22. (DSA or IADSA).tw.
 23. 17 or 20 or 21 or 22
 24. 3 and 16 and 23
 25. *basal ganglion hemorrhage/di or *brain hemorrhage/di or *brain ventricle hemorrhage/di or *cerebellum hemorrhage/di or *brain hematoma/di
 26. 16 and 25
 27. "sensitivity and specificity"/
 28. receiver operating characteristic/
 29. diagnostic accuracy/
 30. exp diagnostic error/
 31. observer variation/
 32. "limit of detection"/
 33. "diagnostic test accuracy study".sh.
 34. (sensitivity or specificity).tw.
 35. (predictive adj3 value$).tw.
 36. ((false adj positive$) or (false adj negative$)).tw.
 37. observer variation$.tw.
 38. (roc adj curve$).tw.
 39. (likelihood adj3 ratio$).tw.
 40. or/27‐39
 41. 3 and 16 and 40
 42. 24 or 26 or 41
 43. 23 and 25
 44. 3 and 23 and 40
 45. 42 or 43 or 44

Appendix 3. Modified QUADAS methodological items and operational definitions

Methodological variable	Operational definition/information required from each study
1. Was the spectrum of patients representative of the patients who will receive the test in practice? (Spectrum bias)	Yes: the study population reflects the target population, which we have defined as prospectively identified, consecutive adults (≥ 16 years of age) with acute intraparenchymal ICH (with or without extension into other intracranial spaces) who are well enough to undergo the both tests, have no contraindications to either the index test or reference standard, and who are recruited with the express purpose of determining the diagnostic accuracy of the index test No: the study population does not match these criteria Unclear: there is insufficient information provided to compare the study population to our ideal target population
2. Is the reference standard likely to correctly classify the target condition?	Yes: IADSA was performed by an expert or experienced operator. We define 'expert' as a neuroradiologist and 'experienced' as a consultant level (or equivalent) radiologist with ≥ 5 years of clinical experience. In addition, we will require that, at the very minimum, the relevant arterial territories in the vicinity of the haematoma must have been examined No: IADSA was not performed by an expert or experienced operator or the complete arterial territories surrounding the haematoma were not examined Unclear: insufficient information is available to evaluate this criterion
3. The delay between the index test and reference standard was documented (Disease progression bias)	Yes: the mean or median delay (± standard deviation or interquartile range) between the index test and reference standard is explicitly stated No: the time delay is not reported Of note, there is no 'appropriate' time frame between tests per se. This is partly because we do not know how long the haematoma persists in each case. Residual haematoma could obscure the vascular malformation and therefore studies using longer delays between tests may report varying estimates of accuracy. It is impossible at this point to provide an empirically based 'appropriate' time frame since we do not yet have the data to analyse this question. We aimed to record delay, however, as we wish to establish empirical evidence of what should be an appropriate time frame between tests
4. Did the whole sample or a random selection of the sample, receive verification using a reference standard? (Partial verification bias)	Yes: every patient or a random selection of patients undergoing the index test also received the reference standard No: the index test played a role in selecting who underwent the reference standard Unclear: there is insufficient information available to determine if the index test influenced who underwent the reference standard
5. Did patients receive the same reference standard regardless of the index test result? (Differential verification bias)	Yes: all patients undergoing the index test uniformly received the same reference standard No: all patients underwent the same index test but underwent different reference standards Unclear: there is insufficient detail to tell if every participant in the analysis underwent the same reference standard
6. Was the reference standard independent of the index test? (Incorporation bias)	Yes: the index test does not form a part of the reference standard No: the index test is used in conjunction with IADSA as the formal reference standard Unclear: there is insufficient information to determine whether the index tests formed a part of the reference standard
7.  Were the reference standard results interpreted without knowledge of the results of the index test? (Information bias)	Yes: there is an explicit statement that the reference standard was interpreted blinded to the results of the index test No: there is an explicit statement that the reference standard was not interpreted blinded to the results of the index test Unclear: there is no statement specifically describing how the reference standard was interpreted in relation to the index test
8. Were the index test results interpreted without knowledge of the results of the reference standard? (Information bias)	Yes: there is an explicit statement that the index test was interpreted blinded to the reference standard No: there is an explicit statement that the index test was not interpreted blinded to the results of the reference standard Unclear: there is no statement specifically describing how the index test was interpreted in relation to the reference standard
9. Were the same clinical data available when test results were interpreted as would be available when the test is used in practice? (Information bias)	Yes: there is an explicit statement that the same clinical information was available to those interpreting the index test and reference standard as would be available in routine clinical practice No: there is an explicit statement that clinical information was not made available to those interpreting the index test or reference standard Unclear: there is no statement to this effect
10. Were uninterpretable or intermediate test results reported?	Yes: uninterpretable index test results are reported and explained No: if it is explicitly stated or if it is clear from the analysis that there were unreported intermediate or uninterpretable index tests results that were not explained by the authors Unclear: it is not clear if there were intermediate or uninterpretable index test results and there is no mention of intermediate or uninterpretable results in the text
11. Were withdrawals from the study explained?	Yes: the authors clearly explain why participants withdrew from the study, if there is an explicit statement that there were no withdrawals, or if all patients can be clearly accounted for in the paper No: if it is clear that patients did withdraw but the authors provide no explanation as to why this occurred Unclear: it is impossible to tell whether patients withdrew from the study
12. Were those interpreting the index test of an appropriate level of training?	Yes: there is a clear statement that an 'expert' (a subspeciality neuroradiologist) or an 'experienced' (defined as ≥ 5 years of practice at a consultant level or equivalent) radiologist interpreted the index test No: there is an explicit statement that someone other than a neuroradiologist or an experienced radiologist interpreted the index test results Unclear: there is no statement describing the background of those who interpreted the index test results or, if a radiologist other than a neuroradiologist interpreted the index test, their years of experience are not provided

Data

Presented below are all the data for all of the tests entered into the review.

Tests. Data tables by test.

Test	No. of studies	No. of participants
1 CTA	8	526
2 MRA	3	401

1. Test.

CTA.

2. Test.

MRA.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Delgado Almandoz 2009.

Clinical features and settings	The study enrolled consecutive patients ≥ 18 years of age presenting to a tertiary care emergency centre in Boston, USA, from 1 January 2000 to 1 November 2008, with evidence of intraparenchymal haemorrhage on CT. Patients had to be evaluated with CTA within 24 hours of presentation. Patients with associated subarachnoid haemorrhage in the basal cisterns, loss of grey‐white matter differentiation in a vascular territory suggesting a pre‐established acute ischaemic stroke, a known intracranial vascular malformation or mass lesion, or known probable cerebral amyloid angiopathy according to Boston criteria were excluded
Participants	775 patients were screened of which 210 were eligible and enrolled. Mean participant age was not documented. The gender breakdown was not provided. No details were available about co‐morbidities. No details were provided about ICH location or extension of blood into other intracranial compartments
Study design	Retrospective study of consecutive patients
Target condition and reference standard(s)	The reference standards were IADSA and intraoperative and pathological findings. Data exclusive to IADSA were obtained. IADSA equipment and technical specifications were not documented. A positive result was a vascular abnormality accounting for the haemorrhage
Index and comparator tests	The index test was CTA performed using a 16‐ or 64‐slice LightSpeed GE Healthcare helical CT scanner. The base of the C1 vertebrae to the vertex was scanned using a pitch of 0.5, 1.25 mm slice thickness, and a field of view of 22 cm. Non‐ionic contrast material (65 to 85 mL) was administered by power injector at 4 to 5 mL/s with either a fixed 25‐second delay between onset of contrast material injection and start of scanning (or a delay of 40 seconds if the participant had atrial fibrillation) or by SmartPrep automatic contrast‐bolus triggering technique. A positive result was defined as any underlying vascular abnormality accounting for the haemorrhage
Follow‐up	The study authors did not state whether there were any study withdrawals. The authors did not report whether there were any instances of uninterpretable images. The authors did not report whether any participants suffered an adverse event
Level of expertise of those interpreting the index test	The CTA source and maximum intensity projection images were reviewed by 2 neuroradiologists blinded to the participant's clinical information
Notes	The resulting 1.25 mm thick axial CTA source images were digitally archived and standard maximum intensity projection images of the major intracranial vessels were created by the 3‐D laboratory
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	Patients had to be evaluated with CTA within 24 hours of presentation to be eligible for the study
Acceptable reference standard? All tests	Unclear	The reference standard was IADSA and was described in sufficient detail to replicate. However, the level of expertise of those interpreting the images was not documented
Acceptable delay between tests? All tests	No	The authors did not report the delay between the CTA and IADSA
Partial verification avoided? All tests	No	This is a retrospective study and, in routine practice, the CTA result would influence the decision to proceed to either to IADSA or straight to surgery
Differential verification avoided? All tests	Yes	IADSA, surgical findings, and pathology were used as reference standards. However, data exclusive to IADSA as a reference standard were obtained from the authors
Incorporation avoided? All tests	Yes	CTA did not form a part of the reference standard
Reference standard results blinded? All tests	Unclear	The reference standard was the IADSA final report. It is not clear whether those interpreting the IADSA images were privy to results of the CTA scans
Index test results blinded? All tests	Yes	Those interpreting the CTA were blinded to the IADSA result
Relevant clinical information? All tests	No	Those interpreting the CTA images were blinded to all relevant clinical information including the participant's clinical condition and other imaging reports
Uninterpretable results reported? All tests	Unclear	No comment was made about uninterpretable scans
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	Yes	The CTA source and MIP images were reviewed by two neuroradiologists

Eshwar Chandra 1998.

Clinical features and settings	This study enrolled patients from a tertiary care referral centre in Chandigarh, India, with acute, spontaneous ICH over a 5‐year period from 1990 to 1995. Hypertensive patients with haemorrhage in the thalamus, putamen, internal capsule, and the cerebellum were excluded
Participants	It is unclear how many patients were screened for inclusion or were potentially eligible for enrolment. 45 participants were enrolled. Females comprised 36% (16/45) of the total population. Mean or median age was not documented; age range was 3 to 75 years old. It is unclear how many had a prior intracranial haemorrhage. ICH was lobar in 38 (84%) cases, deep in 4 (9%) cases, and infratentorial in 3 (7%) cases. Mean ICH size at presentation was not documented
Study design	Prospective case series. All participants underwent dynamic CT (CTA) and 44 of 45 underwent IADSA
Target condition and reference standard(s)	The reference standards were IADSA and surgical findings. IADSA equipment and technical specifications were not documented. A positive result was a vascular abnormality accounting for the haemorrhage
Index and comparator tests	The index test was a dynaCT. All were performed using either a Siemens Somatom HiQ S Scanner or a Shimadzu SCT 2000T. These were single‐slice, non‐spiral scanners. Technical specifications were not documented. The study covered the Circle of Willis and the region of the haematoma. A positive result was a vascular abnormality accounting for the haemorrhage
Follow‐up	Study withdrawals and exclusions were explained. There was no statement made about uninterpretable images. There was no statement regarding whether any adverse effects occurred
Level of expertise of those interpreting the index test	The level of experience of those interpreting the dynaCT scans was not documented
Notes	There was no description of post‐processing reconstruction of angiographic images
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	Hypertensive patients with haemorrhage in the thalamus, putamen, internal capsule, and cerebellum were excluded
Acceptable reference standard? All tests	Unclear	The technical specifications and the background of those interpreting the IADSA was not documented. There was no description of the extent to which the arterial territories that surrounded the haematoma were evaluated
Acceptable delay between tests? All tests	No	The delay between tests was not reported
Partial verification avoided? All tests	Yes	CTA did not determine who got IADSA
Differential verification avoided? All tests	Yes	IADSA and histopathological findings were used as reference standards. Data exclusive to IADSA could be extracted
Incorporation avoided? All tests	Yes	dynaCT did not form a part of the reference standard
Reference standard results blinded? All tests	Unclear	There was no statement about whether those interpreting the IADSA images were blinded to the results of the dynaCT test
Index test results blinded? All tests	Unclear	There was no statement about whether those interpreting the dynaCT test were blinded to the results of the IADSA
Relevant clinical information? All tests	Unclear	There was no statement about whether those interpreting the IADSA and dynaCT were provided with all relevant clinical information apart from the results of the competing test
Uninterpretable results reported? All tests	Unclear	There was no statement about whether uninterpretable results were encountered
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	Unclear	The level of experience of those interpreting the dynaCT scans was not documented

Lummel 2012.

Clinical features and settings	Patients with acute ICH identified from an electronic in‐hospital database in Munich, Germany, were included if they had MRI/A imaging within 20 days of symptom onset and had MRI/A and IADSA image quality that was sufficient for evaluation. Patients with traumatic ICH were excluded. However, in the majority of cases, MRA was only performed if CTA was 'normal', meaning that this is likely a dilute sample of patients with vascular malformations
Participants	120 patients were screened, 67 were eligible, and all were enrolled. Mean age was 53.4 years (range 14 to 82 years of age). There were 23 (34%) females and 44 (66%) males. No details were provided about co‐morbidities. ICH was associated with intraventricular haemorrhage in 8 cases and subarachnoid haemorrhage in 5 cases. Haemorrhage was lobar in 36 cases, basal ganglial in 13 cases, and infratentorial in 18 cases
Study design	Retrospective case series. All participants underwent both MRA and IADSA although it appears that IADSA preceded MRA in all cases
Target condition and reference standard(s)	The reference standard was IADSA. The equipment was not described other than stating that a Biplanar Neurostar DSA system was used. The authors did not provide a formal definition but stated that their aim was to identify vascular malformations and other bleeding sources in ICH patients
Index and comparator tests	The index test was MRA. All scans were performed using either a Magnetom Symphony 1.5T unit or a Sign HDxt 3T unit. Magnet strength was either 1.5 or 3 T. Depending on the participant, sequences included T1, contrast‐enhanced T1, T2, proton density, FLAIR, T2*, and DWI. All 67 participants had time of flight imaging and 40 participants had contrast‐enhanced imaging. The study authors did not provide a formal definition but stated that their aim was to identify vascular malformations and other bleeding sources in ICH patients
Follow‐up	Study withdrawals and exclusions were explained. There was no statement made about uninterpretable images. There was no statement regarding whether any adverse effects occurred
Level of expertise of those interpreting the index test	MRA images were evaluated independently by 2 neuroradiologists
Notes	IADSA images were evaluated by the same 2 neuroradiologists. There was no description of post‐processing reconstruction of angiographic images. Interrater reliability for the detection of an IADSA positive lesion was excellent (kappa = 0.93)
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	All patients with non‐traumatic ICH who had an MRI performed within 20 days of symptom onset. However, in the majority of cases, MRA was only performed if CTA was 'normal', meaning that this is likely a dilute sample of patients with vascular malformations
Acceptable reference standard? All tests	Unclear	The technical specifications of the IADSA were not documented. There was no description of the extent to which the arterial territories that surrounded the haematoma were evaluated
Acceptable delay between tests? All tests	No	The delay between tests was not reported
Partial verification avoided? All tests	No	The study was retrospective and in routine practice it would be expected that MRA would influence the choice to proceed to IADSA
Differential verification avoided? All tests	Yes	Every participant underwent IADSA as the reference standard
Incorporation avoided? All tests	Yes	MRA did not form a part of the reference standard
Reference standard results blinded? All tests	No	The same 2 neuroradiologists interpreted the IADSA images in the presence of all participant information following their analysis of MRA images
Index test results blinded? All tests	Yes	MRA was interpreted by 2 neuroradiologists blinded to all participant information including IADSA results
Relevant clinical information? All tests	No	MRA was interpreted by 2 neuroradiologists blinded to all participant information
Uninterpretable results reported? All tests	Unclear	There was no statement about whether uninterpretable results were encountered
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	Yes	MRA was interpreted by 2 neuroradiologists

Ma 2012.

Clinical features and settings	Patients with acute ICH admitted to West China Hospital in Sichuan, China, were included if they were non‐hypertensive (defined by medical history or clinical evidence of normal blood pressure) and were between the ages of 18 and 45 years. Patients with a traumatic ICH or primarily subarachnoid haemorrhage were excluded
Participants	436 patients were screened, 97 were eligible, and 92 were enrolled. 5 eligible patients were excluded because of renal failure (n = 1) or a brain tumour (n = 4). Data regarding age and gender were not provided. No details were provided about co‐morbidities. No details were provided about ICH location or extension of blood into other intracranial compartments
Study design	Prospective case series. All participants underwent both CTA and IADSA
Target condition and reference standard(s)	The reference standard was IADSA. The equipment was not described other than stating a GE LCV + Advantx system was used. A positive test result was defined as a 'vascular lesion'
Index and comparator tests	The index test was CTA. All scans were performed using a GE LightSpeed VCT 64 scanner. The study covered 'rostral from C‐2'. The parameters used included 0.625 mm slice thickness every 0.6 mm and a pitch of 513:1. A total of 125 mL of Omnipaque 300 was administered intravenously at a rate of 4.0 to 5.0 mL/s using a 15‐second prep delay. A positive test result was defined as a 'vascular lesion'
Follow‐up	Study withdrawals and exclusions were explained. There was no statement made about uninterpretable images. There was no statement regarding whether any adverse effects occurred
Level of expertise of those interpreting the index test	CTA images were evaluated independently by 2 neuroradiologists
Notes	There was no description of post‐processing reconstruction of angiographic images. IADSA images were interpreted by 2 endovascular neuroradiologists
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	Patients over the age of 45 and those with pre‐existing hypertension were excluded
Acceptable reference standard? All tests	Unclear	The technical specifications of the IADSA were not documented. There was no description of the extent to which the arterial territories that surrounded the haematoma were evaluated
Acceptable delay between tests? All tests	No	The delay between tests was not reported
Partial verification avoided? All tests	Yes	CTA did not determine who underwent IADSA
Differential verification avoided? All tests	Yes	Every participant underwent IADSA as the reference standard
Incorporation avoided? All tests	Yes	CTA did not form a part of the reference standard
Reference standard results blinded? All tests	Yes	IADSA was reviewed by 2 endovascular neuroradiologists blinded to clinical and CTA data
Index test results blinded? All tests	Yes	CTA was reviewed by 2 neuroradiologists blinded to clinical information
Relevant clinical information? All tests	No	IADSA and CTA were read blinded to all clinical data
Uninterpretable results reported? All tests	Unclear	There was no statement about whether uninterpretable results were encountered
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	Yes	CTA was interpreted by 2 neuroradiologists

Murai 1999.

Clinical features and settings	Patients referred to a tertiary care referral centre in Tokyo, Japan, for investigation of spontaneous ICH. Patients were included if they had an initial CT scan performed within 12 hours of symptom onset and a second CT within 24 hours of onset, a 3D‐CTA performed within 12 hours of symptom onset, and IADSA performed in the 'chronic stage'. Patients who were clinically unstable or who had a history of chronic renal failure or an allergic reaction to iodine were excluded
Participants	92 patients were screened, and 31 were eligible and enrolled. The mean age was 56 years (range 34 to 74). Over half (17/31; 55%) were male. No details were provided about co‐morbidities. No details were provided about ICH location or extension of blood into other intracranial compartments
Study design	Prospective case series. All participants underwent both the index and reference standard
Target condition and reference standard(s)	The reference standard was IADSA. The equipment used to perform IADSA was not described. 1‐ or 2‐vessel angiography was performed using cut‐film technique. A positive test result was the presence of a vascular abnormality as the cause of haemorrhage
Index and comparator tests	The index test was CTA. All scans were performed using a Hitachi W 3000 AD helical single‐slice scanner. The study covered the Circle of Willis and the region of the haematoma. The parameters used included 1 mm slice thickness, a 512 x 512 matrix, a pitch of 1, and a 21 cm field of view. Participants were scanned from the posterior margin of the haematoma through superior margin including the middle cerebral artery and Circle of Willis. A total of 100 mL of Iomeprol (350 mg iodine/mL) was administered intravenously at 2 mL/s using a 25‐second pre‐scanning delay. A positive result was the identification of a vascular abnormality as the cause of haemorrhage
Follow‐up	Study withdrawals and exclusions were explained. There was no statement made about uninterpretable images. There was no statement regarding whether any adverse effects occurred
Level of expertise of those interpreting the index test	CTA images were evaluated by a neuroradiologist
Notes	CTA images were reconstructed into 3 dimensions using a volume‐rendered technique through 'standard scanner software'. We were able to reclassify cases of moya‐moya as 'negatives' after receiving data from the study authors
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	The study population was restricted to only those patients undergoing CT scans at both 12 and 24 hours following symptom onset and a CTA within 12 hours of symptom onset
Acceptable reference standard? All tests	Unclear	The reference standard was IADSA and was described in sufficient detail to replicate. However, the level of expertise of those interpreting the images was not documented
Acceptable delay between tests? All tests	No	The authors did not indicate the delay between the CTA and IADSA
Partial verification avoided? All tests	No	The CTA result determined when the participant underwent IADSA
Differential verification avoided? All tests	Yes	All participants underwent both a CTA and an IADSA
Incorporation avoided? All tests	Yes	CTA did not form a part of the reference standard
Reference standard results blinded? All tests	Unclear	The authors did not state whether the IADSA images were interpreted blinded to the CTA result
Index test results blinded? All tests	Unclear	The authors did not state whether the CTA images were interpreted blinded to the IADSA result
Relevant clinical information? All tests	Unclear	The authors did not state whether all routine clinical information apart from the result of the competing test was available to those interpreting the CTA and IADSA images
Uninterpretable results reported? All tests	Unclear	The authors did not state whether uninterpretable images were encountered
Withdrawals explained? All tests	Yes	The authors explained study exclusions. All enrolled participants were accounted for in the analysis
Index test interpreted by an expert? All tests	Yes	A neuroradiologist reviewed all CTA images

Romero 2009.

Clinical features and settings	This study enrolled patients from a tertiary care referral centre in Boston, USA, with acute, spontaneous ICH who underwent CTA over a 5‐year period from 2002 to 2007. Patients with known vascular anomalies, known brain neoplasms, those with imaging findings suggestive of ischaemic stroke on admission, or those with subarachnoid haemorrhage within the Sylvian fissures or basal cisterns, or both, were excluded
Participants	80 patients were screened for inclusion. 43 patients were eligible and all were enrolled. There were no study withdrawals. Females comprised 54% (23/43) of the total population. The mean age was 28 years (range 4 to 40). 14% (6/43) of patients had pre‐existing hypertension. It is unclear how many had a prior intracranial haemorrhage. 35% (15/43) had pure ICH, 9% (4/43) had ICH with subarachnoid haemorrhage, 21% (9/43) had ICH with intraventricular haemorrhage, 7% (3/43) had ICH with subdural haemorrhage, 23% (10/43) had ICH with subarachnoid and intraventricular haemorrhage, and 5% (2/43) had ICH with subarachnoid, intraventricular, and subdural haemorrhage. ICH was lobar in 27 (63%) cases, deep in 5 (12%) cases, infratentorial in 9 (21%) cases, and mixed in 2 (4%) cases. Mean ICH size at presentation was not documented
Study design	Retrospective case series. All participants underwent both the index and reference standard
Target condition and reference standard(s)	The reference standard was IADSA and surgical and pathological findings. Data exclusive to IADSA were obtained. IADSA equipment and technical specifications were not documented but was there were appropriate references to the literature that would enable replication of the procedure. A positive result was a vascular abnormality accounting for the haemorrhage
Index and comparator tests	The index test was CTA performed using a 16‐ or 64‐slice GE Healthcare helical CT scanner. The skull base to the vertex was scanned using a pitch of 0.7, 3 mm slice thickness, and a field of view of 18 cm. Isovue 370 non‐ionic contrast material (90 to 120 mL) was administered by power injector at 2 to 3 mL/s with either a fixed 25‐second delay between onset of contrast material injection and start of scanning (or a delay of 40 seconds if the participant had atrial fibrillation) or by SmartPrep automatic contrast‐bolus triggering technique. A positive result was defined as any underlying vascular abnormality accounting for the haemorrhage
Follow‐up	The authors did not state whether there were any study withdrawals. The authors did not report whether there were any instances of uninterpretable images. The authors did not report whether any participants suffered an adverse event
Level of expertise of those interpreting the index test	A neuroradiologist evaluated the CTA source and maximum intensity projection images blinded to the participant's clinical information
Notes	The resulting 1.25 mm thick axial CTA source images were digitally archived and standard maximum intensity projection images of the major intracranial vessels were created by the 3‐D laboratory
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	Patients older than 40 were excluded from the study
Acceptable reference standard? All tests	Yes	IADSA was performed by an expert operator and was described well enough to replicate through appropriate references to the literature
Acceptable delay between tests? All tests	Yes	The time between IADSA and CTA was ≤ 48 hours
Partial verification avoided? All tests	No	This is a retrospective study and, in routine practice, the CTA result would be expected to influence the decision to proceed to either to IADSA or straight to surgery
Differential verification avoided? All tests	Yes	IADSA, surgical findings, and pathology were used as reference standards. However, data exclusive to IADSA as a reference standard were obtained from the authors
Incorporation avoided? All tests	Yes	CTA did not form a part of the reference standard
Reference standard results blinded? All tests	No	Those interpreting the IADSA knew the results of the CTA scan
Index test results blinded? All tests	Yes	Those interpreting the CTA were blinded to the IADSA result
Relevant clinical information? All tests	No	Those interpreting the CTA images were blinded to all relevant clinical information including clinical condition and image reports
Uninterpretable results reported? All tests	Unclear	No comment was made about uninterpretable scans
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	Yes	A neuroradiologist evaluated all CTA source and MIP images

Wong 2010.

Clinical features and settings	This study enrolled patients from a tertiary care referral centre in Hong Kong, China, who were admitted for cerebral angiography during the subacute phase (6 to 18 weeks) following spontaneous ICH. All patients had ICH confirmed by CT scan performed within 24 hours of clinical onset. Only patients whose ICH was detected with both MRI and IADSA during the subacute phase were included in the study. Patients with predominant subarachnoid haemorrhage were excluded as were those over 45 years with pre‐existing hypertension and thalamic, putaminal, or posterior fossa haemorrhage
Participants	It is unclear how many patients were screened for inclusion. 169 patients were eligible and all were enrolled. 18 participants had to withdraw from the study (10 were too unwell to undergo MRI and 8 had an MRI from another institution). Females comprised 38% (57/151) of the total population. The mean age was 42 years (SD 15). Over one‐third of patients had pre‐existing hypertension (52/151; 34%). It is unclear how many had a prior intracranial haemorrhage. ICH was lobar in 80 (53%) cases, deep in 49 (32%) cases, and infratentorial in 22 (15%) cases. Mean ICH size at presentation was not documented
Study design	Retrospective case series. All participants underwent both the index and reference standard
Target condition and reference standard(s)	The reference standard was a Phillips V3000 biplanar DSA unit with catheterisation of the relevant cerebral vessels. Technical specifications were not documented. Relevant cervical vessels were catheterised depending on the location of the ICH. Standard views were obtained by hand injection of Omnipaque 300 6 mL to 9 mL and 3D angiography was performed for participants with vascular lesions. A positive result was a vascular abnormality accounting for the haemorrhage
Index and comparator tests	The index test was MRA performed using a 1.5 Tesla whole body scanner with a standard head coil (Magnetom Sonata, Siemens Medical Systems or Intera‐NT, Phillips Medical Systems). MRA without contrast was performed using 3‐dimensional time of flight with a slice thickness of 0.7 mm. An arteriovenous malformation was suspected if the MRA source images showed a conglomerate of parenchymal curvilinear structures. A dural arteriovenous fistula was suspected if MRA showed multiple curvilinear structures over the surface of the brain without parenchymal evidence of a nidus
Follow‐up	Study withdrawals were explained by the authors. The authors did not report whether there were any instances of uninterpretable images. The authors did not report whether any participants suffered an adverse event
Level of expertise of those interpreting the index test	All MRA images were reviewed by 2 neuroradiologists
Notes	Standard maximum intensity projection images of the major intracranial vessels were created following MRA
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	Patients were admitted for IADSA during the subacute period following ICH onset. Patients over the age of 45 years with pre‐existing hypertension and thalamic, putaminal, or posterior fossa haemorrhages were excluded
Acceptable reference standard? All tests	Yes	IADSA was performed by an expert operator
Acceptable delay between tests? All tests	Yes	Time from onset to each test was recorded
Partial verification avoided? All tests	No	This is a retrospective study and, in routine practice, the MRA result would influence the decision to proceed to IADSA
Differential verification avoided? All tests	Yes	IADSA was the only reference standard
Incorporation avoided? All tests	Yes	MRA did not form a part of the reference standard
Reference standard results blinded? All tests	Yes	Those interpreting the IADSA were blinded to the MRA result
Index test results blinded? All tests	Yes	Those interpreting the MRA were blinded to the IADSA result
Relevant clinical information? All tests	Unclear	It is unclear whether those interpreting the MRA and IADSA images had access to all relevant clinical information apart from the results of the competing scan
Uninterpretable results reported? All tests	Unclear	No comment was made about uninterpretable scans
Withdrawals explained? All tests	Yes	Participant withdrawals were explained
Index test interpreted by an expert? All tests	Yes	All MRA images were reviewed by 2 neuroradiologists

Wong 2011.

Clinical features and settings	The study enrolled consecutive patients from a tertiary care referral centre in Hong Kong, China, who presented with spontaneous, non‐hypertensive and/or lobar intracerebral haemorrhage within 96 hours of the clinical ictus. Patients older than 45 years with pre‐existing hypertension and thalamic, putaminal, or posterior fossa haemorrhage, those older than 70 years, those with known renal impairment or an allergy to intravenous contrast, and those with an emergency craniotomy before IADSA were excluded
Participants	966 patients were screened, 135 were eligible, and 109 consented to the study. Females comprised 33% (36/109) of the total population. The mean age was 48 (SD 15). No participants were considered to have had premorbid hypertension. It is unclear how many had a prior intracranial haemorrhage. 12 participants (10%) had pure ICH, 18 (17%) had ICH with subdural haemorrhage, 42 (39%) had ICH with subarachnoid haemorrhage, and 37 (34%) had ICH with intraventricular haemorrhage. ICH was lobar in 80 (73%) cases, deep in 20 (19%) cases, and infratentorial in 9 (8%) cases. Mean ICH size at presentation was 10 mL (SD 22)
Study design	Prospective case series of consecutive patients presenting with spontaneous ICH
Target condition and reference standard(s)	The reference standard was a Phillips V3000 DSA unit with catheterisation of the relevant cerebral vessels. Technical specifications were not documented. Relevant cervical vessels were catheterised depending on the location of the ICH. Standard views were obtained by hand injection of Omnipaque 300 6 mL to 9 mL and 3D angiography was performed for participants with vascular lesions. A positive result was defined as any underlying vascular aetiology for the ICH
Index and comparator tests	The index test was CTA performed using a 64‐slice GE Healthcare helical CT scan with the assistance of SmartPrep bolus tracking software. Arteriography was performed in helical mode 120 kV, 220 mAs with no tilting. Axial images were reconstructed at 0.625 mm intervals and stored as source images for further analysis with 3‐dimensional reconstruction. A positive result was defined as any underlying vascular aetiology for the ICH
Follow‐up	The authors did not state whether there were any study withdrawals. The authors did not report whether there were any instances of uninterpretable images. The authors did not report whether any participants suffered an adverse event
Level of expertise of those interpreting the index test	2 neuroradiologists blind to both the clinical data and the catheter angiography findings interpreted the CTA scans
Notes	Standard multiplanar reformatting and maximum intensity projection images of the major intracranial vessels were created following CTA. We were able to reclassify cases of brain tumours and venous sinus thrombosis as 'negatives' using the published data
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	No	All patients 70 years of age or older and those between 45 to 69 years of age with pre‐existing hypertension and thalamic, putaminal, or posterior fossa haemorrhages were excluded
Acceptable reference standard? All tests	Yes	IADSA was performed by an expert operator
Acceptable delay between tests? All tests	Yes	Time from onset to each test was recorded
Partial verification avoided? All tests	Yes	The CTA result did not determine who underwent IADSA
Differential verification avoided? All tests	Yes	IADSA was the only reference standard
Incorporation avoided? All tests	Yes	CTA did not form a part of the reference standard
Reference standard results blinded? All tests	Yes	Those interpreting the IADSA were blinded to the CTA result
Index test results blinded? All tests	Yes	Those interpreting the CTA were blinded to the IADSA result
Relevant clinical information? All tests	No	Neuroradiologists interpreting the CTA and IADSA scans were blinded to clinical data
Uninterpretable results reported? All tests	Unclear	No comment was made about uninterpretable scans
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	Yes	All CTA scans were interpreted separately by 2 neuroradiologists

Yeung 2009.

Clinical features and settings	Patients admitted to tertiary care centre in Toronto, Canada, with nontraumatic, primarily non‐subarachnoid haemorrhage. Patients had to have undergone CTA and IADSA prior to any surgical intervention
Participants	286 patients were screened and 55 met the eligibility criteria and were enrolled. The study population was 58% (32/55) male and the median age was 49 years (range 16 to 71) for males and 50 years (range 30 to 79) for females. No details were provided about co‐morbidities. Lobar haemorrhages were present in 31 participants, deep haemorrhages in 9, and infratentorial haemorrhages in 7. Pure intraventricular haemorrhage occurred in 5 participants and pure subdural haemorrhage occurred in 3 participants
Study design	Retrospective case series. All participants underwent both the index test and reference standard
Target condition and reference standard(s)	IADSA was performed using a Phillips uniplane neuroangiographic unit. Technical specifications were not provided. Selective injections were performed according to the side of the haematoma and the presence of a CTA detected abnormality. Both external and internal carotid vessels were imaged in cases where no cause was seen on CTA. A positive result was the presence of a 'secondary cause' of haemorrhage
Index and comparator tests	The index test was CTA. All scans were performed using a GE Medical Systems LightSpeed Plus 4‐slice CT or VCT 64‐slice helical scanner. Each covered at least C2 to the vertex. The parameters used included 1.25 or 0.625 mm slice thickness (depending on the scanner). The matrix, pitch, and field of view were not specified. A total of 100 mL to 125 mL Omnipaque 300 or Visipaque 320 were injected at a rate of 4.0 mL/s to 4.5 mL/s with either a 17‐second delay or the use of Smart Prep at the pulmonary artery. A positive result was the presence of a 'secondary cause' of haemorrhage
Follow‐up	The authors did not state whether there were any study withdrawals. The authors did not report whether there were any instances of uninterpretable images. The authors did not report whether any participants suffered an adverse event
Level of expertise of those interpreting the index test	3 staff neuroradiologists blinded to participant information and other imaging studies independently reviewed all CTA studies
Notes	Coronal and sagittal multiplanar reformatting images were recreated at 7 mm thickness spaced by 3 mm. Bilateral 5‐degree rotational multiplanar reformatting images were created at the carotid terminus. The 3D rendered images were created on a GE Advantage Workstation
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	Yes	Participants all had acute intraparenchymal ICH (with or without extension into other intracranial spaces), were well enough to undergo both tests, had no contraindications to either CTA or IADSA, and were recruited with the express purpose of determining the diagnostic accuracy of CTA
Acceptable reference standard? All tests	Yes	The reference standard was IADSA and was described in sufficient detail to replicate
Acceptable delay between tests? All tests	Yes	The delay between tests was reported
Partial verification avoided? All tests	No	The CTA result determined who underwent IADSA
Differential verification avoided? All tests	Yes	IADSA was the only reference standard
Incorporation avoided? All tests	Yes	CTA did not form a part of the reference standard
Reference standard results blinded? All tests	Unclear	2 neuroradiologists reviewed the IADSAs at least 1 month after the CTA to reduce recall bias
Index test results blinded? All tests	Yes	The CTA images were interpreted blinded to the IADSA result
Relevant clinical information? All tests	No	Those interpreting the CTA and IADSA images were blinded to all participant information
Uninterpretable results reported? All tests	Unclear	The authors did not state whether uninterpretable images were encountered
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	Yes	3 staff neuroradiologists independently reviewed all CTA studies

Yoon 2009.

Clinical features and settings	The study included patients referred to a tertiary care referral centre in Seoul, South Korea, for investigation of spontaneous ICH. Patients with predominant subarachnoid haemorrhage, known pre‐existing vascular abnormalities, or those with haemorrhage into a tumour were excluded
Participants	105 patients were screened and 88 were eligible but 10 were not enrolled because they deteriorated rapidly before both tests could be performed. A total of 78 participants were therefore enrolled. The mean age was 48 years (SD 14). Over half (44/78; 56%) were male. No details were provided about co‐morbidities. All 78 had lobar ICH and 44% of patients (34/78) had extension of the bleed into the subarachnoid, subdural, or intraventricular compartments
Study design	Prospective case series. All participants underwent both the index and reference standard
Target condition and reference standard(s)	IADSA was performed using a Phillips Integris system. A matrix of 1024 x 1024 pixels was used. Bilateral selective internal carotid artery injections and either unilateral or bilateral vertebral artery injections were performed for each case. Anteroposterior, lateral, oblique, and additional views of each vessel, if necessary, were obtained using 6 mL to 9 mL of iodixanol (Visipaque 320, GE Healthcare) injected at a rate of 4 mL/s to 6 mL/s. A positive result was an underlying vascular abnormality accounting for the haemorrhage
Index and comparator tests	Multi‐detector CTA was obtained using a 16‐detector row CT scanner (Phillips MX8000 Infinite Detector Technology) The parameters used included 1 mm slice thickness, a 512 x 512 matrix, a pitch of 0.35, and a 20 to 22 cm field of view. Participants were scanned from the foramen magnum through the top of the skull including the entire cerebral vasculature. A total of 100 to 120 mL of iohexol (Omnipaque 300; GE Healthcare) was administered intravenously using a bolus‐tracking technique. A positive result was an underlying vascular abnormality accounting for the haemorrhage
Follow‐up	Study withdrawals were explained. There was no mention of uninterpretable scans. There was no statement regarding adverse effects
Level of expertise of those interpreting the index test	All CTA images were independently evaluated by 2 neuroradiologists
Notes	Multi‐detector CTA was reconstructed into transverse sections with a section width of 0.5 mm from the source images using sagittal and coronal multiplanar reformations, volume‐rendered technique, and volume‐rendered technique after automatic segmentation of a pre‐contrast scan data set. We were able to reclassify cases of moya‐moya as 'negatives' after receiving data from the authors
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	Yes	Participants were all prospectively identified, consecutive cases with acute intraparenchymal ICH (with or without extension into other intracranial spaces) who were well enough to undergo both CTA and IADSA, had no contraindications to CTA or IADSA, and were recruited with the express purpose of determining the diagnostic accuracy of CTA
Acceptable reference standard? All tests	Yes	The reference standard was IADSA for all participants
Acceptable delay between tests? All tests	Yes	The delay between tests was reported
Partial verification avoided? All tests	Yes	CTA results did not influence who underwent IADSA
Differential verification avoided? All tests	Yes	All participants underwent IADSA as the reference standard
Incorporation avoided? All tests	Yes	CTA did not form a part of the reference standard
Reference standard results blinded? All tests	Yes	IADSA was interpreted independent of CTA
Index test results blinded? All tests	Yes	CTA was interpreted independent of the IADSA
Relevant clinical information? All tests	Unclear	There was no statement about whether those interpreting the CTA and IADSA were provided with all relevant clinical information apart from the results of the competing test
Uninterpretable results reported? All tests	Unclear	There was no statement about whether uninterpretable test results were encountered
Withdrawals explained? All tests	Yes	Study withdrawals were explained
Index test interpreted by an expert? All tests	Yes	All CTA images were independently evaluated by 2 neuroradiologists

Zhou 2012.

Clinical features and settings	The study included patients referred to a tertiary care referral centre at Xi'an Jiaotong University, China, for investigation of subacute ICH defined as assessment within 1 week of symptom onset. Diagnosis was made according to the 4th National Vascular Diagnostic Criteria. Patients with blood disorders, space occupying lesions, extracranial pathology, and meningitis were excluded
Participants	183 participants were enrolled. The mean age was 43.7 years (SD 14.9). Overall age ranged from 18 to 81. Slightly over half (97/183; 53%) were male. No details were provided about co‐morbidities apart from pre‐existing hypertension (present in 52/183; 28%). Anatomical haemorrhage location was lobar in 106 (58%), deep in 55 (30%), and infratentorial in 22 (12%)
Study design	It was unclear whether the case series was prospective or retrospective. All participants underwent both the index and reference standard
Target condition and reference standard(s)	IADSA was performed using a Phillips system. Unit specifications were not provided. Bilateral selective internal carotid and vertebral artery injections were performed for each case. Anteroposterior, lateral, and oblique views of each vessel were obtained. A positive test result was not formally defined
Index and comparator tests	The MRA unit was not described. 3‐dimensional time of flight imaging was acquired with a slice thickness of 2.0 mm and a matrix size of 512 x 256. A positive test result was not formally defined
Follow‐up	Study withdrawals were explained. There was no mention of uninterpretable scans. There was no statement regarding adverse effects
Level of expertise of those interpreting the index test	MRA images were interpreted by either a radiologist or a neurosurgeon. There is no statement as to whether the reviewers were blinded to the results of IADSA
Notes	—
Table of Methodological Quality
Item	Authors' judgement	Description
Representative spectrum?	Yes	Patients with CT‐confirmed subacute (assessed within 1 week of ictus) intracerebral haemorrhage were included
Acceptable reference standard? All tests	Yes	The reference standard was IADSA for all participants
Acceptable delay between tests? All tests	No	The authors did not indicate the delay between MRA and IADSA
Partial verification avoided? All tests	No	MRA results did influence who underwent IADSA
Differential verification avoided? All tests	Yes	All participants underwent IADSA as the reference standard
Incorporation avoided? All tests	Yes	MRA did not form a part of the reference standard
Reference standard results blinded? All tests	Unclear	The study does not state whether IADSA was interpreted blinded to the MRA results
Index test results blinded? All tests	Unclear	The study does not state whether MRA was interpreted blinded to the IADSA results
Relevant clinical information? All tests	Unclear	There was no statement about whether those interpreting the MRA and IADSA were provided with all relevant clinical information apart from the results of the competing test
Uninterpretable results reported? All tests	Unclear	There was no statement about whether uninterpretable test results were encountered
Withdrawals explained? All tests	Yes	All participants could be accounted for in the paper
Index test interpreted by an expert? All tests	No	Not all of the results were reviewed by a neuroradiologist (some were only reviewed by neurosurgeons)

CT: computed tomography
 CTA: computed tomography angiography
 IADSA: intra‐arterial digital subtraction angiography
 ICH: intracerebral haemorrhage
 MIP: maximum intensity projection
 MRA: magnetic resonance angiography
 MRI: magnetic resonance imaging
 SD: standard deviation

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Awad 1992	Unable to obtain data exclusive to IADSA
Bekelis 2012	Unable to obtain data exclusive to IADSA
Bowen 2007	Review article
Campeau 2012	Review article
Chandra 2012	Review article
Dammert 2004	Study population was SAH or 'atypical' ICH
Elhammady 2008	IADSA was not the reference standard
Evans 2005	Overall study population of < 20 participants
Fasulakis 2003	Study was performed on patients with known AVMs (the study population was therefore not ICH)
Griffiths 2006	The study population consisted of subarachnoid and intraventricular haemorrhage
Gross 2012	The study population was not ICH
Hünerbein 2003	Unable to obtain data exclusive to IADSA
Kadkhodayan 2012	No CTA or MRA data
Kamel 2013	No comparison between MRA and IADSA
Khosravani 2013	No comparison between MRA and IADSA
Kidwell 2010	Review article
Lee 2007	Unable to obtain data exclusive to IADSA
Leung 2012	Review article
Li 2013	IADSA was not the reference standard
Pott 1992	Index test was not CTA or MRA
Sasiadek 2000	Insufficient information
Sasiadek 2002	Insufficient information
Sha 2008	Unable to obtain data exclusive to IADSA
Truwit 2007	Review article
Wijman 2012	IADSA was not the reference standard
Zheng 2012	No comparison with IADSA

AVM: arteriovenous malformation
 CTA: computed tomography angiography
 IADSA: intra‐arterial digital subtraction angiography
 ICH: intracerebral haemorrhage
 MRA: magnetic resonance imaging
 SAH: subarachnoid haemorrhage

Differences between protocol and review

We calculated sensitivity and specificity for the entire population of studies and evaluated only adults (participants ≥ 16 years of age) in a sensitivity analysis.

We initially planned to conduct a meta‐analyses of study‐specific pairs of sensitivity and specificity to create a summary ROC curve in the SROC space using the random‐effects hierarchical SROC model of Rutter and Gatsonis (Rutter 2001), but instead opted for the bivariate following the review stage (Macaskill 2010).

We used specificity rather than 1‐specificity to define the x‐axis of the SROC plot.

We used the Metandi package within the Stata Statistical Software version 12.1, rather than Statistical Analysis System (SAS) version 9.2 for Windows, to perform the meta‐analysis.

We did not investigate heterogeneity or publication bias because of too few studies (Deeks 2005).

Contributions of authors

CBJ and RA‐SS conceived and designed the review. CBJ, PMW, and RA‐SS designed the data collection sheet and the search strategies. CBJ searched all databases and screened the search results. CBJ organised the retrieval of the papers. Two review authors (CBJ and PMW or RA‐SS) screened papers against the inclusion criteria, appraised the quality of papers, and extracted data from the papers. CBJ wrote to authors of papers for additional information. CBJ obtained and screened data on unpublished studies. CBJ was responsible for data management for the review and entered data into RevMan 5.2. CBJ, PMW, AK, and RA‐SS analysed and interpreted the data. CBJ, PMW, AK, and RA‐SS wrote the manuscript, which all authors reviewed.

Sources of support

Internal sources

• No sources of support supplied

External sources

• UK Medical Research Council Senior Clinical Fellowship, UK.

  RA‐SS is supported by a UK Medical Research Senior Clinical Fellowship

Declarations of interest

CBJ, AK, and RA‐SS have no known disclosures to report. Siemens Medical, who make all modalities of imaging equipment, are a minor sponsor of an educational neurointerventional meeting co‐organised by PMW (all of the grant subsidises the course and no fee goes to the organisers).

New