Computed Tomography Interpretation in a Telestroke Network: Agreement between Spoke Radiologist, Hub Vascular Neurologist, and Hub Neuroradiologist

Abstract

Background and Purpose

The American Stroke Association guidelines emphasized the need for further high-quality studies which assess agreement by radiologists and non-radiologists engaged in emergency telestroke assessments and decision making. Therefore, the objective of this study was to determine the level of agreement of baseline brain CT scan interpretations of acute stroke patients presenting to telestroke spoke hospitals between central reading committee neuroradiologists and each of two groups, spoke hospital radiologists and hub hospital vascular neurologists (telestrokologists).

Methods

The Stroke Team Remote Evaluation Using a Digital Observation Camera Arizona trial was a prospective, urban single-hub, rural two-spoke, randomized, blinded, controlled trial of a two-way, site-independent, audiovisual telemedicine and teleradiology system designed for remote evaluation of adult patients with acute stroke versus telephone consultation to assess eligibility for treatment with intravenous thrombolysis. In the telemedicine arm, the subjects’ CT scans were interpreted by the hub telestrokologist and in the telephone arm, by the spoke radiologist. All subjects’ CT scans were subsequently interpreted centrally, independently, and blindly by two hub neuroradiologists. The primary CT outcome was determination of a CT based contraindication to thrombolytic treatment. Kappa statistics and exact agreement rates were used to analyze interobserver agreement.

Results

Fifty-four subjects underwent random assignment. The overall agreement for the presence of radiological contraindications to thrombolysis was excellent (0.91) and did not differ substantially between hub telestrokologist to neuroradiologist and spoke radiologist to neuroradiologist (0.92 and 0.89, respectively).

Conclusions

In the context of a telestroke network designed to assess patients with acute stroke syndromes, agreement over the presence or absence of radiological contraindications to thrombolysis was excellent whether the comparisons were between telestrokologist and neuroradiologist or between spoke radiologist and neuroradiologist.

Clinical Trial Registration Information

www.Clinicaltrials.Gov Number NCT00623350

Based on a review of the published evidence, the American Heart Association/American Stroke Association (AHA/ASA) granted Class I recommendations to Food and Drug Administration (FDA) approved teleradiology systems for review of brain computed tomography (CT) in patients with suspected acute stroke (Class I, Level of Evidence A), to brain CT scan reviews by stroke specialists or radiologists using FDA approved teleradiology systems for identifying exclusions for thrombolytic therapy (Class I, Level of Evidence A), and to FDA approved teleradiology systems to support rapid imaging interpretation in time for thrombolysis decision making (Class I, Level of Evidence B).¹ However, the AHA/ASA uncovered no published studies which prospectively evaluated CT brain interpretations of patients with acute stroke syndromes by rural spoke community hospital radiologists, urban hub primary stroke center hospital vascular neurologists (telestrokologists), and compared them to that of urban hub primary stroke center hospital neuroradiologists in the context of an active hub and spoke telestroke network. The AHA/ASA guidelines emphasize the need for further high-quality studies which assess and compare accuracy of image interpretation and agreement by radiologists and non-radiologists engaged in emergency telestroke assessments and decision making.

The objective of this study was to determine the level of agreement of baseline brain CT scan interpretations of acute stroke patients presenting to telestroke spoke hospitals between central reading committee neuroradiologists and each of two groups, spoke hospital radiologists and telestrokologists.

Methods

The Stroke Team Remote Evaluation Using a Digital Observation Camera (STRokE DOC) technique, STRokE DOC Arizona trial methodology, and the primary and pooled results were published.^2–7 The STRokE DOC Arizona trial was a prospective, urban single-hub, rural two-spoke, randomized, blinded, controlled trial of a two-way, site-independent, audiovisual telemedicine system designed for remote evaluation of adult patients with acute stroke versus telephone consultation to assess eligibility for treatment with intravenous thrombolysis. Determining the noncontrast head CT interpretation agreement between spoke radiologist, telestrokologist, and central radiology adjudication committee was an established secondary objective of the protocol. To that end, consecutive consented subjects presenting with acute stroke syndromes to the two participating rural spoke hospitals were randomly assigned to telemedicine or telephone consultations. In the telemedicine arm, the subjects’ CT scans were interpreted by one of the hub’s four telestrokologists and in the telephone arm, by one of the spoke’s ten radiologists. All subject’s CT scans were subsequently interpreted centrally, independently, and blindly by two hub neuroradiologists who had no knowledge of prior interpretations, thrombolysis decision making, or subsequent clinical course. Whereas the telestrokologists interpreted the scans in the context of awareness of neurological examination features, both the spoke radiologists and the hub neuroradiologists were presented only with brief one-line statements of CT indication, for example, “left hemiparesis” or “aphasia and right hemiparesis” or “headache, nausea, and diplopia”. The telestrokologists viewed CT scans with a digital imaging and communications in medicine (DICOM) viewer, the spoke radiologists viewed CT scans on Picture Archiving and Communication System (PACS), and the hub neuroradiologists on the central committee viewed the radiologic images from compact discs loaded onto a desktop viewing system (QREADS).⁸ The resolution of the video monitors for all three systems exceeded the standard image size of 256 × 256 or 512 × 512 pixels generated by modern CT scanners. During image interpretation, all of the reviewers were free to adjust window level and window width as felt necessary. The primary CT outcome was determination of a CT based contraindication to thrombolysis. Each of the spoke radiologists or telestrokologists, and central neuroradiologists determined the presence or absence of a radiological contraindication to thrombolysis (i.e. evidence of any intracranial hemorrhage, brain neoplasm, or explanatory etiology other than stroke, or prominent early ischemic changes exceeding one third of the middle cerebral artery territory (i.e. Alberta Stroke Program Early CT (ASPECT) score < 7).⁹ Secondary CT outcomes included localization of the lesion, presence of prior stroke, edema, hemorrhage, neoplasm, and hyperdense artery sign.

Statistical Methods

To calculate agreement of baseline CT scans overall, Fleiss κ-statistics were used to assess the proportion of agreement beyond that expected by chance. Due to the low prevalence of events for each of the variables, the observed agreement was reported as well, overall and within each group. All analysis was performed using the statistical software R 2.11.0 (www.r-project.org).

Results

Fifty-four subjects were randomly assigned to telemedicine (27) and telephone-only (27) consultations. The overall trial flow, subject baseline demographics, and risk factors have been published.² All fifty-four subjects completed baseline noncontrast CT-scans of the head, however one subject’s CT was not transmitted for central interpretation by neuroradiology. Therefore the analyzed dataset was comprised of fifty-three acute stroke subjects’ interpretable baseline CT-scans. For proportions of CT scans harboring a radiological feature, agreement between telestrokology and neuroradiology, agreement between spoke radiology and neuroradiology, and overall agreement, refer to Table 1. There was no statistically significant difference in agreement over the determination of critical radiological features contraindicating thrombolysis administration between the two arms of the trial.

Table 1.

CT Scan – General Information (Overall) and Kappa by Arm and Overall

Radiological Feature	Feature Present N (%)*	Overall N=53		Spoke Radiology & Hub Neuroradiology N=27	Telestrokology & Hub Neuroradiology N=26
		N (%)Agreement	Kappa (95 %CI)	% Agreement	% Agreement
Normal Scan	15 (28)	46 (87)	0.62 (0.38,0.87)	23 (85)	23 (89)
Acute Ischemic Stroke	9 (17)	41 (77)	0.32 (0.02,0.62)	22 (81)	19 (73)
Chronic Stroke	24 (45)	39 (74)	0.49 (0.27,0.71)	17 (63)	22 (85)
Edema	3 (6)	41 (77)	-	21(78)	20 (77)
Tumor	1 (2)	52 (98)	-	26 (96)	26 (100)
Hyperdense Artery	1 (2)	49 (93)	-	25 (93)	24 (92)
Contraindication to Thrombolysis	4 (8)	48 (91)	-	24 (89)	24 (92)

^*Reflects the presence of the feature observed by the Central Read

When disease prevalence is very high or very low, the κ values are decreased relative to the percentage of agreement. The presentation notes when this might affect interpretation and κ values are included only for the overall sample.

Of the fifty-four subjects, sixteen received r-tPA. All sixteen subjects completed baseline CT-scans of head, however one subject’s CT was not transmitted for central interpretation by neuroradiology. Therefore the analyzed dataset was comprised of fifteen thrombolysed subjects’ interpretable baseline CT-scans. Agreement was perfect for absence of intracranial hemorrhage, brain neoplasm, or other explanatory etiologies in the r-tPA subset. In only a single subject, of the r-tPA subset, was there disagreement regarding the presence of radiologic contraindications to r-tPA, with particular reference to the extent of observed early ischemic changes in the middle cerebral artery territory.

Discussion

In a telestroke network, it might be desirable for the spoke emergency practitioner, spoke radiologist, telestrokologist, and even a hub neuroradiologist to view, interpret, and collaboratively come to consensus on every CT head conducted on a telestroke alert patient, but this may not be practical given time, geographic, technological, personnel availability, and connectivity constraints. For instance, in Stroke Telemedicine for Arizona Rural Residents (STARR), in 71.1% of the conducted telestroke alert consultations the CT was interpreted by the telestrokologist prior to decision making, and in 28.9% of the consultations the CT was interpreted by the spoke radiologist alone, prior to decision making. (unpublished; communication with principal investigator). In multiple, single primary stroke center study comparisons of intraobserver agreement on CT head interpretation in patients with acute stroke and other neurological emergencies amongst specialties (emergency physicians, neurologists, radiologists, and neuroradiologists), agreement ranged from 0.39 to 0.69 without knowledge of clinical information to 0.71 to 0.89 with knowledge of clinical information, and interobserver agreement between specialties ranged from 0.61 to 0.83.^9–12 Interobserver variation of CT head interpretation (ASPECT score) by physicians engaged in acute stroke in real time compared to retrospective expert evaluation was still substantial, weighted kappa 0.69.¹³ This is the first report, to our knowledge, of agreement between observers of CT head interpretation in the context of a telestroke network. The principal limitation of the study was the small number of subjects. The scope of the study was tightly focused on key decision-making metrics related to stroke patient eligibility for intravenous thrombolysis, hence the findings may not be applicable to the more complex real-world of acute head CT interpretation in patients who present to emergency departments with neurological emergencies in general. We wish to emphasize that the agreement reached was when telestrokologists possessed complete clinical information, while the radiologists received only short concise descriptions of the patients’ presenting neurological symptoms and signs. This may have created a bias in favor of the interpreting telestrokologist. Nevertheless, it is reassuring that overall agreement on radiological contraindications to thrombolysis was excellent (0.91), and even agreement over the presence of subtle early ischemic changes consistent with acute ischemic stroke was substantial (0.76). Equally reassuring is that agreement over the key radiological features was substantial whether between telestrokologist and neuroradiologist or between spoke radiologist and neuroradiologist. CT head interpretation in acute stroke requires training and expertise.¹⁴ Routine optimization of CT scans to detect hyperdense arteries (e.g. using thin sections and multiplanar reconstructions), and incorporation of CT angiography have become mainstream for most stroke centers. Advanced neurovascular imaging of this sort, which was not the focus of this study, is much more data-intensive and may be more suited to radiology workflow, post-processing, and PACS systems.

Conclusion

In the context of a telestroke network designed to assess patients with acute stroke syndromes, agreement over the presence or absence of radiological contraindications to thrombolysis was excellent whether the comparisons were between telestrokologist and neuroradiologist or between spoke radiologist and neuroradiologist.