Efficacy of site-independent telemedicine in the STRokE DOC trial: a randomised, blinded, prospective study

Abstract

Background

To increase effective use of rt-PA for acute stroke, vascular neurology expertise must be disseminated more widely. We prospectively assessed whether telemedicine (real-time, 2 way audio/video and DICOM interpretation) or telephone was superior for decision-making in acute telemedicine consultations.

Methods

Acute stroke patients were randomized to telemedicine or telephone consultation. Primary outcome measure was whether the thrombolytic treatment decision was correct, as determined by central adjudication. Secondary outcomes included rt-PA use-rate, 90 day functional outcomes, hemorrhages, and technical observations.

Findings

Two hundred thirty-four patients were prospectively evaluated. Mean NIHSS score was 9.5 (11.4±8.7 telemedicine, 7.7±7.0 telephone; p=0.0020). One telemedicine consult (0.9%) was aborted for technical reasons, though was included in intention-to-treat analyses. Correct treatment decision was made more often in telemedicine (98.2% telemedicine, 82% telephone; OR 10.9; 95%CI 2.7-44.6; p=0.0009). IV rt-PA use-rate was 25% (28% telemedicine, 23% telephone; OR 1.3; 95%CI 0.7-2.5; p=0.4248). Ninety day functional outcomes were not different for BI(95–100) (OR 0.6; 95%CI 0.4-1.1; p=0.1268) or for mRS (OR 0.6; 95%CI 0.3-1.1; p=0.0898). There was no mortality difference (OR 1.6; 95%CI 0.8-3.4; p=0.2690). Post-rt-PA ICH rates were not different (7% telemedicine, 8% telephone; OR 0.8; 95%CI 0.1-6.3; p=1.0). There was a difference noted for amount of non-completed data (3.1% telemedicine, 12.0% telephone; OR 0.2; 95%CI 0.1-0.3; p<0.001).

Interpretation

This trial reports that stroke telemedicine consultations result in more accurate decision making, compared to telephone, and can serve as a model for the effective use of telemedicine in other medical fields. The more appropriate decisions, high rt-PA userates, improved data collection, low ICH rates, low technical complications, and favorable time requirements all support telemedicine’s efficacy, most specifically for decision-making, and may enable more practitioners to use telemedicine in daily stroke care.

Introduction

Though approved over 10 years ago, few (2–3%) stroke patients receive rt-PA.¹ Thrombolytic therapy must be used rapidly and appropriately if stroke disability is to be reduced.² Previous approaches to increasing treatment rates have failed partially due to incomplete dissemination of expertise and geographic limitations. Greater specialist availability should increase appropriate treatments and minimize protocol violations.^3,4 Telemedicine, already implemented in numerous fields, could enable dissemination of stroke expertise for consultation, education, and research.^5–8

Telemedicine is reliable for measuring stroke deficit.^9–12 Remote assistance increases rt-PA administrations using telephone¹³ or telemedicine,^14,15,16 Though numerous systems are available, few randomized trials have been performed,¹⁷ and decision-making efficacy remains unknown. We sought to assess the correctness of decision-making in the time-pressured setting of acute stroke. We compared telemedicine (remote audio/video & radiology review) to telephone consultations, to test the hypothesis that telemedicine increases decision-making efficacy. If telemedicine decisions are appropriate, this technology can be immediately implemented into daily practice.

Methods

We planned a 4-year, 400-patient trial, randomizing participants to ‘telemedicine’ or ‘telephone-only’ consultations. Consults were performed at 4 remote sites (spokes) located 30 to 350 miles from an academic hub. The trial was approved by the Human Research Protections Programs, and was registered at ClinicalTrials.gov (NCT00283868). Trial methods are published.¹⁸

Equipment included Internet enabled laptops (used by a pool of 3 fellowship trained vascular neurologists) and the telemedicine systems at remote Emergency Department (ED) facilities. Software enabled site-independent access to 2-way audio/high resolution video, over standard Internet (BF Technologies, Inc, San Diego, CA).

When a patient arrived at a participating spoke ED with acute stroke symptoms, the hub stroke team was contacted by pager system. Participants were to be included if they were at least 18 years, were able to sign consent (or have surrogate), and had symptoms of acute stroke. There were no other specific exclusions. Consent was obtained at the spoke and faxed to the hub consultant, using an Internet fax technique, prior to randomization.

Patients were randomized using permuted blocks, stratified by study site to prevent group imbalances. Randomization to ‘telemedicine’ or ‘telephone- only’ consultation was done in real-time using a web-based randomization system, thus eliminating practitioner preference bias. If the patient randomized to telemedicine, the consult commenced using site-independent access to the telemedicine system. The hub consultant turned on the camera and immediately performed a history and NIHSS exam. Other exam elements were performed by or reported to the consultant as able. Head CT images were viewed using a DICOM viewer.

If the patient randomized to telephone, the hub consultant queried the spoke practitioner about history, physical, laboratory, and local radiologist’s report of the CT, and directed the local practitioner in performing NIHSS elements. Neither the video nor the head CT images were viewed by the consultant.

In both groups, the consultant attempted to complete pre-specified case report forms. The consultant was free to repeat examination items, and could speak with available family members/witnesses. Clinical deficit and functional scales (including the NIHSS and pre and post stroke mRS) were determined by the consultant using information supplied by the bedside practitioner/personnel. After reviewing the history, exam, stroke and outcome scales, and head CT interpretation, the hub consultant rendered a thrombolytic recommendation to the spoke ED practitioner.

The specific objective was to determine the efficacy of telemedicine consultations for decision- making. The primary outcome measure was whether the rt-PA decision was appropriate, as determined using a rigorous, multi-stage, blinded adjudication process, details of which are published.¹⁸ Secondary outcomes included rt-PA userates, 90 day outcomes, ICH rates, data completeness, and technical observations.

The STRokE DOC Adjudicating Committee (SDAC) was composed of specialist physicians with training in acute stroke, and excluded practitioners from the remote spoke facilities. Level 1 adjudication included the hub consultant’s review of the case, with the SDAC blinded as to consultation technique. For Level 2a adjudication, an independent monitor reviewed the spoke’s ED/admission record, and adjudicated the correctness of the rt-PA decision based solely on the NINDS rt-PA inclusion/exclusion.^3,4 Based on detailed discussions, still blinded to the group assignment, the SDAC rendered a separate Level 2b determination as to whether the decision was appropriate based on all information that would have been available at the ED bedside. The Level 2b decision was the primary outcome measurement. Detailed procedures were followed to ensure the SDAC members remained blinded to arm.¹⁸ The consultant and monitor were not present during voting.

To estimate power, we used a Chi-square test (2 sided alpha=0.05) and assumed 80% correct decision rate with telephone, a 10% telemedicine effect size, and sample size of 400; power was 80%. Statistical analysis for the primary outcome used a random-effects logistic regression model.¹⁹ The impact on adjudication decision was modeled as a function of treatment arm. Site was included in the model as a random effect with an exchangeable correlation structure. Due to sparse arrays, the Cochran-Mantel-Haenszel (CMH) chi-squared test, stratified by site, and fixed-effect logistic regression were conducted as a sensitivity analysis. The CMH chi-squared test, stratified by site, and fixed-effect logistic regression were used for all other ‘correct decision’ outcomes. The Fisher’s Exact test was used for rt-PA rate, ICH rate, mortality, 90 day mRS and missing data analysis. The Wilcoxon Rank Sum was used for 90 day Barthel and time-point comparisons.

Three NIHSS items were incompletely performed, most frequently in the telephone group. In an attempt to more accurately compare severity, the NIHSS was adjusted by removing the 3 items (from both groups’ NIHSS) found most often to be incomplete. A random-effect logistic regression model, with a random effect for participant, was used to test the presence of incomplete demographic or NIHSS data fields. All analyses were conducted using the statistical software R 2.1.1(www.R-project.org).

The trial was not restricted to a 3 hour window in order to replicate true acute stroke scenarios where time of onset is initially unknown. Investigators did not want to delay evaluations or exclude potential patients by mandating a conclusive <3 hour time of onset be established before initiating a consultation. However, enrollments did occur where treatment disagreements were impossible (e.g. patients presenting late), thus artificially enhancing agreement in both arms. After approximately 200 patients were enrolled, the Steering Committee proposed that the trial might be underpowered due to this, and recommended a conditional probability analysis for futility/efficacy. The Steering Committee was blinded to data results when making this recommendation. An analysis plan and a priori guidelines for possible trial termination were finalized prior to the statistical core performing analyses. The Steering Committee halted the trial after the blinded conditional power analysis showed a range of probabilities between 0.96 to 0.99 probability that one group was superior to the other on primary outcome at study end, given the data so far across a spectrum of future alternatives.

Role of the Funding Source

This work was supported by the National Institute of Neurological Disorders and Stroke (NINDS) (P50NS044148), the California Institute of Telecommunications Technology (Cal(IT)²), and the Department of Veterans’ Affairs, Research Division. The telemedicine application (AccessVideo^™) was provided by BF Technologies, Inc. Neither the NINDS, Cal(IT)², nor the Department of Veterans’ Affairs had a role in study design; in collection, analysis, or interpretation of data; in writing of the report; or in decision to submit the paper for publication.

Results

Patient Characteristics

(Figure 1) Two hundred thirty-four participants with acute stroke symptoms were evaluated between August 2004 and April 2007. The initial 11 non-randomized participants were evaluated during a “run-in” phase. Of the 223 randomized participants, 1 was excluded for being less than 18 years old, and the remaining 222 were randomized to telemedicine (n=111) or telephone (n=111) consultations. One telemedicine consult was aborted for technical reasons, though was included in analyses. Ninety day outcomes were available for 92.9%. (Table 1) There were no demographic differences between groups. One- hundred- two (46%) were Hispanic. Mean age was 69.7±14.7 and there were 108 (49%) males. Sixty-one (28%) had CAD, 75 (34%) had hyperlipidemia, and 23 (10%) had stroke/TIA family history. Each of these 3 risk factors was greater in telemedicine.

Table 1. Baseline Patient Characteristics.

Table 1 shows STRokE DOC baseline patient demographics and risk factors. Also included next to each percentage are the percentages of unknown/missing data for each trial arm technique. These unknown percentages are included as they may drive the statistical significance of the comparison.

Patient Characteristics	Overall (n = 22)	Telemedicine (n = 111)	Telephone (n = 111)	P value	Estimate 95% CI
Age (years)
Mean ± SD	69.7 ± 14.7	70.4 ± 14.5	69.0 ± 14.9	0.4110	1.4 (−2.5, 5.3)a
Female, n(%)	114 (51)	57 (51)	57 (51)	1.0000	1.00 (0.59, 1.69)b
Race, n(%)				0.5440
White	211 (95)	106 (96)	105 (95)
Black	6 (3)	4 (4)	2 (2)
Pacific Islander	4 (2)	1 (1)	3 (3)
Asian	1 (1)	0 (0)	1 (1)
Not Hispanic, n(%)	120 (54)	60 (54)	60 (54)	1.0000	1.00 (0.59, 1.69)b
Weight (kg)
Mean ± SD	80.0 ± 20.5	79.7 ± 18.8	80.5 ± 36.7	0.8240	−0.8 (−7.1, 5.5)a
Risk Factors, n(%)&(% unknown)
Coronary Disease*	61 (28) (6% unknown)	37 (33) (3% unknown)	24 (22) (10% unknown)	0.0260
MI	17 (8) (14% unknown)	12 (11) (12% unknown)	5 (5) (15% unknown)	0.1840
Prior CVA	81 (37) (5% unknown)	40 (36) (5% unknown)	41 (37) (5% unknown)	1.0000
Atrial Fibrillation	29 (13) (7% unknown)	19 (17) (5% unknown)	10 (9) (8% unknown)	0.1740
Diabetes	78 (35) (4% unknown)	43 (39) (2% unknown)	35 (32) (5% unknown)	0.2520
Hypertension	164 (74) (3% unknown)	83 (75) (5% unknown)	81 (73) (2% unknown)	0.4010
Hyperlipidemia**	75 (34) (15% unknown)	45 (41) (7% unknown)	30 (27) (23% unknown)	0.0020
Fam Hx: Stroke/TIA***	23 (10) (28% unknown)	18 (16) (18% unknown)	5 (5) (39% unknown)	<0.001
Present Alcohol Use***	30 (14) (22% unknown)	15 (14) (9% unknown)	15 (14) (34% unknown)	<0.001
Present Tobacco Use***	21 (10) (20% unknown)	13 (12) (9% unknown)	8 (7) (31% unknown)	<0.001

^*p<0.05

^**p<0.01

^***p<0.001; CI=Confidence Interval

^aOdds Ratios;

^bDifference in means

(Table 2) Mean NIHSS=9.5 (11.4±8.7 telemedicine, 7.7±7.0 telephone; p=0.0020). Mean modified NIHSS=7.3, and was lower in telephone vs. telemedicine (p=0.0040). There was a trend toward less severe strokes at baseline in telephone (p= 0.0772), with 13% mild strokes (mRS=0–1) in telemedicine vs. 23% in telephone. There were no differences in baseline ICH incidence (P= 1.0) or baseline imaging rt-PA contraindications (P=0.4248). More baseline CT scans were normal in telephone (26% in telemedicine, 45% in telephone; p=0.0048). In the rt-PA subgroup, Mean NIHSS=14.5 (16.3±7.4 telemedicine, 12.3±6.5 telephone; p=0.0440).

Table 2. Baseline Stroke Severity.

Table 2 shows pre-stroke function using the investigator’s estimate of the mRS before stroke, and baseline post-stroke severity using mRS, NIHSS and mNIHSS. Baseline NIHSS is significantly higher in telemedicine. Also included are numbers of primary ICHs and baseline CT scan differences for each arm of the trial.

Baseline Stroke Severity	Overall (n=222)	Telemedicine (n=111)	Telephone (n=111)	P value	Estimate (95% CI)
Pre-Stroke mRS (Complete Scale) n (%)
Dichotomized (0–1)	164 (75)	78 (72)	86 (78)	0.3545	0.73 (0.40, 1.35)a
0=No symptoms	141 (64)	65 (60)	76 (69)
1=No significant disability	23 (11)	13 (12)	10 (9)
2=Slight disability	12 (6)	8 (7)	4 (4)
3=Moderate disability	27 (12)	15 (14)	12 (11)
4=Moderate to severe disability	16 (7)	8 (7)	8 (7)
5=Severe disability	1 (1)	0 (0)	1 (1)
Baseline mRS (Complete Scale) n (%)
Dichotomized (0–1)	39 (18)	14 (13)	25 (23)	0.0772	0.51 (0.25, 1.04)a
0=No symptoms	12 (6)	6 (6)	6 (5)
1=No significant disability	27 (12)	8 (7)	19 (17)
2=Slight disability	26 (12)	15 (14)	11 (10)
3=Moderate disability	34 (16)	14 (13)	20 (18)
4=Moderate- severe disability	68 (31)	35 (32)	33 (30)
5=Severe disability	53 (24)	31 (28)	22 (20)
NIHSS Mean ± SD (Median)**	9.5 ± 8.1 (7)	11.4 ± 8.7 (10.5)	7.7 ± 7.0(5)	0.0020	3.70 (1.61, 5.79)b
mNIHSS Mean ± SD (Median)**	7.3 ± 6.8 (5)	8.8 ± 7.4 (8)	5.9 ± 5.9 (4)	0.0040	2.90 (1.13, 4.67)b
Baseline CT
Scan Normal**	78 (36%)	29 (26%)	49 (45%)	0.0048	0.44 (0.25, 0.77)a
Primary ICH	17 (8%)	9 (8%)	8 (7%)	1.0000	1.14 (0.42, 3.06)a
CT Contraindication to rt-PA	29 (14%)	17 (16%)	12 (11%)	0.4248	1.48 (0.67, 3.27)a
rt-PA subset NIHSS (Mean ± SD)*	14.5 ± 7.2	16.3 ± 7.4	12.3 ± 6.5	0.0440	4.00 (0.22, 7.78)b
rt-PA subset mNIHSS (Mean ± SD)	11.4 ± 6.6	12.7 ± 6.7	9.8 ± 6.2	0.1130	2.90 (−0.59, 6.39)b

^*p≤0.05

^**p≤0.005;

^aOdds Ratios;

^bDifference in means; CI=Confidence Interval

Times

(Table 3) “Onset to Door” was 7.7 minutes shorter for telephone, though nonsignificant (163.2min telemedicine, 155.5min telephone; p=0.3520). “Onset to Decision” showed a trend in favor of telephone (258.0min telemedicine, 230.6min telephone; p=0.0670). “Door to Decision” was not different (97.8min). “Call to Consent”, delineating Informed Consent duration, was not different (33.7min). “Consent to Decision”, delineating consult duration, took 9.2 minutes longer for telemedicine (32.0min telemedicine, 22.9min telephone; p<0.001). “Consent to rt-PA” took 6.5 minutes longer for telephone, though was nonsignificant (51.2min telemedicine, 44.8min telephone; p=0.1630). “Decision to rt-PA” took 5.6 minutes longer for telephone (10.0min telemedicine, 15.6min telephone; p=0.0190).

Table 3. Evaluation Times.

Table 3 shows relevant Stroke Code time points: Means ± Standard Deviations, number of participants with each time point (n) are presented. For the p-values, a footnote denotes the statistical significance level. ^* Indicates reason for relevant exclusions.

Stroke Code Times	Overall (min)	Telemedicine (min)	Telephone (min)	P
Onset to Door	159.5 ± 215.72 (n=147)	163.2 ± 195.72 (n=77)	155.5 ± 237.16 (n=70)	0.3520
Onset to Call	185.5 ± 225.9 (n=216)	192.8 ± 234.4 (n=108)	178.1 ± 217.8 (n=107)	0.4380
Onset to EKG	218.9 ± 220.23 (n=117)	220.6 ± 212.52 (n=59)	217.1 ± 229.65 (n=58)	0.4090
Onset to Lab	238.6 ± 240.51 (n=111)	227.0 ± 194.9 (n=57)	250.9 ± 282.18 (n=54)	0.3860
Onset to Decision	244.2 ± 226.03 (n=216)	258.0 + 229.88 (n=107)	230.6 + 222.42 (n=109)	0.0670
Onset to rt-PA*	150.7 ± 35.83 (n=55)	157.2 ± 37.3 (n=30)	143.0 ± 33.05 (n=25)	0.1370
Door to MD Eval	7.61 ± 29.28 (n = 124)	8.75 ± 36.48 (n=68)	6.23 ± 17.11 (n=56)	0.6130
Door to Call	35.55 ± 51.13 (n=146)	31.72 ± 42.66 (n=78)	39.96 ± 59.42 (n=68)	0.3760
Door to Consent	71.81 ± 51.67(n=146)	69.32 ± 43.23 (n=79)	74.75 ± 60.34 (n=67)	0.5280
Door to EKG	61.84 ± 46.67 (n=82)	68.5 ± 47.62 (n=46)	53.33 ± 44.63 (n=36)	0.0590
Door to Lab	70.78 ± 62.35 (n=82)	70.83 ± 48.87 (n=46)	70.72 ± 76.98 (n=36)	0.3400
Door to Neuro Exam	70.09 ± 34.51 (n=142)	75.21 ± 32.80 (n=75)	64.36 ± 35.72 (n=67)	0.0340
Door to CT Reading	84.77 ± 59.8 (n = 119)	84.32 ± 47.38 (n=69)	85.4 ± 74.14 (n=50)	0.6720
Door to Decision	97.77 ± 54.05 (n=146)	99.79 ± 43.47 (n=77)	95.51 ± 64.09 (n=69)	0.1980
Call to Consent	33.7 ± 26.38 (n=214)	33.55 ± 25.45 (n=109)	33.85 ± 27.44 (n=105)	0.8940
Call to Neuro Exam	36.74 ± 32.71 (n=216)	43.4 ± 29.6 (n = 109)	29.95 ± 34.44 (n=107)	<0.001
Call to Decision	59.98 ± 31.84 (n=216)	64.71 ± 29.06 (n=108)	55.24 ± 33.88 (n=108)	0.0250
Consent to Neuro Exam	4.74 ± 19.71 (n=214)	10.92 ± 15.15 (n=108)	−1.56 ± 21.80 (n=106)	<0.001
Consent to Decision	27.45 ± 21.17 (n=214)	32.04 ± 17.34 (n=107)	22.86 ± 23.61 (n=107)	<0.001
Consent to rt-PA**	48.35 ± 19.56 (n=54)	51.23 ± 17.78 (n=30)	44.75 ± 21.42 (n=24)	0.1630
Decision to rt-PA***	12.5 ± 9.55 (n=54)	10.03 ± 9.75 (n=30)	15.58 ± 8.51 (n=24)	0.0190

^*1 patient was excluded (missing bolus time).

^**2 patients were excluded (missing time of consent; missing bolus time).

^***2 Patients were excluded (missing bolus time; negative outlier).

Analyses

(Table 4) Primary Outcome Measure: Level 2b adjudication showed that the correct treatment decision was made much more often using telemedicine (98% telemedicine, 82% telephone; OR 10.9, 95%CI 2.7-44.6; p=0.0009). Other Measures: Level 1 and Level 2a adjudication also favored telemedicine. rt-PA rate was high in both groups (28% telemedicine, 23% telephone; OR 1.3, 95%CI 0.7-2.5; p=0.4248). The percentage of patients reaching 90 day BI(95-100) was not statistically different (43% telemedicine, 54% telephone; OR 0.6, 95%CI 0.4-1.1; p=0.1268), nor was 90 day mRS(dichotomized 0–1) (34% telemedicine, 47% telephone; OR 0.6, 95%CI 0.3-1.1; p=0.0898). There was no difference in mortality (19% telemedicine, 13% telephone; OR 1.6, 95%CI 0.8-3.4; p=0.2690).

Table 4. Analyses- Primary & Secondary.

Table 4 shows primary & secondary analyses for each arm of the trial. Results are presented for both overall trial and r-PA subgroup. Per our methods,¹⁸ multiple levels of adjudication were performed with the primary outcome being Level 2b (SDAC). ‘lr’ = fixed-effect logistic regression. ‘cmh’ = Cochran-Mantel-Haenszel chi-squared test. Secondary adjudication outcomes, rt-PA rates and ICH rates are also reported. In keeping with similar publications, % of all patients reaching a 90 day BI of 95–100, was used as a secondary functional outcome assessment, as was dichotomized 90 day mRS. Adjusted subgroup mortality was included.

Analyses	Telemedicine	Telephone	Odds Ratio (95% CI)	P value
Overall	n=110	n=111
Correct Decision
Level 2b (SDAC) (Primary)	98%	82%	10.9 (2.7 –44.6)	0.0009† (<0.0001 cmh;0.0011 lr)
Level 1 (SDAC)	97%	83%	7.2 (2.1 –24.6)	0.0017 lr; 0.0009 cmh
Level 2a (MM)	96%	93%	2.0 (0.6–6.9)	0.2500 lr; 0.4037 cmh
Level 3a (MM)	97%	93%	2.7 (0.7–10.5)	0.140 lr; 0.2383 cmh
Level 3b (SDAC)	97%	83%	7.2 (2.1 –24.6)	0.0017 lr; 0.0008 cmh
Overall IV rt-PA treatment	28% (n=31)	23% (n=25)	1.3 (0.7 –2.5)	0.3340 lr; 0.4248 cmh
Overall Post Consult ICH	7% (n=2)	8% (n=2)	0.8 (0.1 –6.3)	1.0000*
90d BI (95–100)	43% (n=45/105)	54% (n=56/103)	0.6 (0.4–1.1)	0.1268*
90d mRS (Dichotomized 0–1)	34% (n=36/105)	47% (n=48/103)	0.6 (0.3–1.1)	0.0898*
Overall Mortality	19% (n=21)	13% (n=14)	1.6 (0.8–3.4)	0.2690*
+rt-PA Subgroup	n=31	n=25
Correct Decision
Level 1 (SDAC)	97%	B0%	13.7‡	0.0753 lr; 0.1065 cmh
Level 2a (MM)	94%	76%	46 (0.9–25)	0.0797 lr; 0.0980 cmh
Level 2b (SDAC)	97%	76%	7.4 (1.03–53.2)	0.0445 lr; 0.0466 cmh
Level 3a (MM)	97%	84%	10.2‡	0.1308 lr; 0.1157 cmh
Level 3b (SDAC)	97%	84%	10.2‡	0.1308 lr; 0.2586 cmh
Post rt-PA ICH	7% (n=2)	8% (n=2)	0.8 (0.1 –6.3)	1.0000*
90d BI (95–100)	33% (n=10/30)	48% (n= 12/25)	0.5 (0.2– 1.6)	0.2865*
90d mRS (Dichotomized 0–1)	30% (n=9/30)	32% (n=8/25)	0.9 (0.3–2.9)	1.0000*
Subgroup Mortality	39% (n=12)	12% (n=3)	4.6 (1.1 –19)	0.0340*
Mortality-adjusted for Baseline NIHSS			3.4 (0.6–19)	0.1681 lr

SDAC=STRokE DOC Adjudicating Committee, MM=Medical Monitor, CI=Confidence Interval

^†=random-effect logistic regression (clustered by sites), Ir=fixed-effect logistic regression, cmh=Cochran-Mantel-Haenszel chi-squared test (stratified by sites)

^‡Confidence intervals not presented due to very small cell size.

^*p-values reflect two-tailed Fisher’s Exact test.

In the rt-PA subgroup (n=56), the treatment decision was also correct more often in telemedicine (97% telemedicine, 76% telephone; OR 7.4, 95%CI 1.03-53.2; p=0.0466). Percentage reaching 90 day BI(95-100) was not statistically different (33% telemedicine, 48% telephone; OR 0.5, 95%CI 0.2-1.6; p=0.2865). Ninety day mRS(dichotomized 0–1) was not different (30% telemedicine, 32% telephone; OR 0.9, 95%CI 0.3-2.9; p=1.0). Unadjusted mortality difference (39% telemedicine, 12% telephone; OR 4.6, 95%CI 1.1-19; unadjusted p=0.0340) was noted in this subgroup, but after adjustment for imbalanced baseline NIHSS severity (telemedicine rt-PA mean NIHSS=16), the difference was not significant (p=0.1681). There were no differences in post rt-PA ICH (7% vs. 8%; p=1.0), decision to death time (18.08±25.34 days telemedicine, 5.33±3.06 days telephone; p=0.469), percentage of deaths within 2 days (18% telemedicine, 33% telephone; p=0.5160) or death within 7 days for the overall trial (9% telemedicine, 5% telephone; p=0.2850) or rt-PA subgroup (19% telemedicine, 8% telephone; p=0.2770). There was no difference related to rt-PA treatment of mild (mRS 0–1) strokes: (0% telemedicine, 4% telephone; p=0.45).

Adjudication Decisions

(Table 5) The most common Level 2b protocol violations in the 6 telephone rt-PA treated cases included treating: >3 hour patients, rapidly improving or mild symptoms, in spite of hypertension, or in spite of wound. The 2 violations in the telemedicine rt-PA cases included treating: a patient who may have awoken with vertigo, and treating without excluding a theoretical aortic dissection contraindication. (At 90 days, the 6 telephone patients reached average BI=66, while the 2 telemedicine patients reached average BI=100).

Table 5. Adjudication Disagreements.

Table 5 shows the reasons for adjudication disagreement at the primary (Level 2b) adjudication level. This table includes the total number of disagreements at that level (attributed chronologic numbers instead of actual spoke randomization numbers), whether they were telephone or telemedicine, reason for disagreement in each case, presence of subsequent ICH, and 90 day outcomes.

Patient #	Group	rt-PA	Reason for Disagreement	ICH	90 day BI, mRS
1	Telephone	N	Would have treated mild arm weakness & NIHSS=3.	N	BI=100, mRS=0
2	Telephone	N	Patient didn’t awake with deficit. Deficit noted at bathroom. Likely in window. Would have treated.	N	Death
3	Telephone	N	Would have treated ‘mild’ deficit.	N	BI=100, mRS=0
4	Telephone	Y	Decision made too quickly. Blood pressure > 185. Would have rechecked, as still within 3 hour window.	N	BI=75, mRS=4
5	Telephone	N	Would have treated significant symptoms	N	BI=100, mRS=0
6	Telephone	N	Not a TIA. Would have treated the aphasia.	N	BI=85, mRS=3
7	Telephone	N	Would have treated mild symptoms.	N	Withdrew Consent
8	Telephone	N	Would have treated patient with early hypodensity (likely early ischemic changes only).	N	BI=80, mRS=3
9	Telephone	Y	Likely greater than 3 hrs. Would not have treated.	N	BI=100, mRS=0
10	Telephone	N	Would have treated aphasia (in a teacher).	N	BI=100, mRS=0
11	Telephone	Y	“Open wound” and would have waited for glucose.	N	BI=10, mRS=5
12	Telephone	N	Would have waited longer for fluctuating symptoms.	N	Lost to Follow-up
13	Telephone	Y	Greater than 3 hrs. Would not have treated.	N	BI=100, mRS=2
14	Telephone	N	No CT scan done, would have treated mild deficit.	N	BI=100, mRS=0
15	Telephone	N	Would have treated “improving” symptoms.	N	BI=100, mRS=1
16	Telephone	N	Would have treated aphasia and weakness.	N	BI=100, mRS=1
17	Telephone	N	No attempt to lower blood pressure, & patient had a measurable deficit.	N	BI=100, mRS=0
18	Telephone	Y	Would not have treated mild isolated sensory deficit.	N	BI=55, mRS=4
19	Telemedicine	Y	Possibility of aortic dissection not fully excluded.	N	BI=100, mRS=0
20	Telemedicine	Y	Onset time questionable for posterior circulation stroke (patient awoke with vertigo).	N	BI=100, mRS=2
21	Telephone	Y	Would not have treated due to BP exclusion.	N	BI=55, mRS=4
22	Telephone	N	Would have treated aphasia, slurred speech and sensory loss.	N	BI=100, mRS=1

There were 14 telephone cases where adjudicating body felt rt-PA should likely have been offered. Reasons included: milder symptoms, time <3 hours, isolated aphasia, mild CT changes, fluctuating symptoms, improving symptoms, and failure to attempt BP control. (At 90 days, 12 telephone patients reached average BI=89, 1 patient died, and 2 had missing outcomes).

In this trial, reasons for “appropriate non-treatment within 3 hours” included: mild deficit (n=25), TIA/rapidly improving (n=21), presence of hemorrhage on initial CT (n=19), mimic (n=13), seizure at onset (n=3), marked hypodensity on CT (n=2), and INR exclusion (n=2).

Data Completion

Combined demographics and NIHSS analysis showed a difference in percentage of non-completed elements (3% telemedicine, 12% telephone; OR 0.2, 95%CI 0.1-0.3; p<0.001). There was a difference, favoring telemedicine, for 5 risk factor variables that could influence patient outcome (telemedicine vs. telephone) for: CAD (3% vs. 10%; p=0.050), hyperlipidemia (7% vs. 23%; p=0.002), stroke/TIA family history (18% vs. 39%; p<0.001), alcohol use (9% vs. 34%; p<0.001) and tobacco use (9% vs. 31%; p<0.001).

For the NIHSS, there was a difference in non-completed NIHSS data, worse in telephone, (telemedicine vs. telephone) for: loc-questions (1% vs. 10%; p=0.005), loc-commands (1% vs. 8%; p=0.019), gaze (1% vs. 16%; p<0.001), visual fields (1% vs. 35%; p<0.001), face (1% vs. 8%; p=0.019), left leg (1% vs. 7%; p=0.035), ataxia (1% vs. 35; p<0.001), sensory (1% vs. 15%; p<0.001), dysarthria (1% vs. 9%; p=0.010), and neglect (1% vs. 40%; p<0.001).

Technical Observations & Sites

Site independent evaluations were performed in 110 (99%) telemedicine consults. Fifteen (14%) used wireless technology: 802.11 (14) and EVolution, Data-Optimized (EV-DO) broadband wireless (1). Ninety-nine (87%) used Local Area Network (LAN) wired Internet access.

Consultations were performed relatively evenly across sites, spoke ED practitioners, and consultants. Thirty-one (14%) consultations were performed at site1, 121 (55%) at site2, 19 (9%) at site3, and 51 (23%) at site4. A total of 48 spoke ED practitioners requested consults: Eighteen (38%) initiated only 1 consult, while 34 (71%) initiated 1–5 consults. Only 5 (10%) initiated >10 consults, and only 1 (2%) initiated >12. Remote consultant 1 performed 93 (42%) consults, consultant 2 performed 67 (30%), and consultant 3 performed 62 (28%).

Technical observations were noted in 12 (19%) tele-consultations. Only 1 (0.9%) could not be performed due to technical failure. This case was included in intention-to-treat analyses. Of the 11 remaining observations, 6 were radiology interface problems, 3 were audio difficulties, 1 was a camera control failure and 1 was a delay in obtaining faxed consent

There were no differences in diagnoses. Only 17 (8%) were discharged from the ED with a non-stroke/TIA diagnosis. Though patients could be excluded for multiple reasons, the most common thrombolytic exclusions were time >3 hours (43% telephone, 60% telemedicine; unadjusted p=0.0310), mild/resolving symptoms (52% telephone, 34% telemedicine; unadjusted p=0.0190), no measurable deficit (42% telephone, 33% telemedicine; p=0.2610), and unknown onset (28% telephone, 23% telemedicine; p=0.4770).

Comment

This is the first prospective, blinded, and randomized trial showing that telemedicine is efficacious for acute medical decision-making (Table 4, p=0.0009). Stroke telemedicine is widely implemented and discussed,^{8,14,18,20–23} but in spite of telemedicine’s dissemination, efficacy has not previously been shown. Our results bolster support for using telemedicine (real-time, 2 way audio, 2 way video and radiographic interpretation) in time-pressured situations to make urgent treatment decisions, such as whether to use thrombolytic therapy for acute stroke.

Current rates of rt-PA administrations are low, and could be increased.^1,24,25 Our data show that rt-PA use-rate can be improved by increasing stroke specialists’ availability. Telephone assistance increases treatments,¹³ but we have shown that telemedicine adds a greater degree of correct decision-making, making it preferable (Table 4). ED practitioners, hesitant to give rt-PA in spite of their willingness and ability to determine eligibility, may feel more comfortable with telemedicine back-up.^26–28

Despite the greater degree of correct decision-making in this trial, 3-month functional outcomes (defined as percentage reaching Barthel of 95–100, or dichotomized mRS) were not statistically different between groups. The high rate of telephone rt-PA may have been a primary reason for this lack of difference. Further analysis will show if post stroke care differences may have affected outcomes. Though not powered to show improved functional outcomes, failure to show a functional benefit for telemedicine in this trial may have also been due to our small sample size (since the trial was halted early) and to the significant baseline imbalance of more severe deficit in telemedicine.

Unadjusted baseline NIHSS scores were different in the 2 groups. In telemedicine, there may have been an increased ability to perform the NIHSS, especially for subtle findings. In telephone, incomplete data acquisition may have contributed to a lower NIHSS. The consultant was NIHSS certified and directed the NIHSS exam in both groups, but the spoke practitioners may not have been NIHSS certified. These features support the use of telemedicine to determine a more accurate NIHSS, but made direct NIHSS comparisons in this trial more complex. To more rigorously correct for baseline severity imbalances, we adjusted the NIHSS by excluding 3 frequently incomplete items from both groups’ total NIHSS scores. After adjusting for missing items, there was still a higher total score noted in telemedicine. Based on the NIHSS, the telemedicine group presented with more severe strokes. A similar result was noted for mRS(0–1), which further supports the increased stroke severity in the telemedicine group.

STRokE DOC was designed to compare 2 overall consultation techniques, not to assess rt-PA efficacy. In this trial, the rt-PA telemedicine subgroup had a high baseline NIHSS (mean=16), and a high percentage of coronary disease (26%), diabetes (32%) and hypertension (32%), which may have been integral to patient outcome. The rt-PA telemedicine subgroup 90 day mortality was higher than recent large scale telemedicine reports that showed both good functional outcome and mortality.^15,16

Given the small number of telemedicine rt-PA patients, unadjusted sub-group mortality analyses are suspect, and may have been due to chance. After correcting for baseline NIHSS imbalances, the difference in rt-PA subgroup mortality was not significant (p=0.1681). The result also loses significance if adjusted for multiple comparisons. We were reassured that there was no difference in post rt-PA ICH rates or early death, and 90 day functional outcomes were not different for the BI(95–100) or mRS. The unadjusted subgroup finding is inconsistent with key clinical reports: 1) The +rt-PA telephone subgroup’s unadjusted mortality was lower than that of the rt-PA patients in the NINDS trial (12.6% vs. 17.3%),³ and 2) the +rt-PA telemedicine subgroup’s unadjusted mortality was higher than recent large telemedicine reports^15,16 which have shown lower mortality after telemedicine guided rt-PA therapy.

Direct comparisons of trials with different patient populations, and different post-stroke care protocols, should not be made. Instead, larger trials with more rt-PA telemedicine patients would more appropriately measure long term mortality of stroke patients assessed with telemedicine.

Though we did not specifically measure post rt-PA care, post rt-PA ICH was measured, as the most concerning consequence of insufficient post rt-PA management. The ICH rate (7–8%) was equal between arms and consistent with previous studies^{3, 29} suggesting post rt-PA care was adequate and balanced. There was also no statistical difference in 7 day mortality between rt-PA groups (p=0.2770).

As the telephone group had more incomplete data collection, we did not adjust for the more severe risk factors or increased CT scan findings in telemedicine. The CT scan reports in the telephone group were based on initial local radiology interpretation provided to the ED. Since no images were viewed by the consultant in the telephone arm, this local report may have excluded subtle abnormalities subsequently noted in the radiologists’ final dictation. The difference in rates of normal CT scans may also have been because of a more detailed initial read by the vascular neurologist in the telemedicine group. Therefore, we also did not adjust for this increased CT findings in telemedicine, as they may have been artifactual. Complete central review of all images is underway.

We did not intervene in remote hospital post rt-PA protocols or care-plans. We only assessed the single variable of consult technique. Other studies are assessing the combination of telemedicine and stroke units.³⁰ A higher percentage of telephone rt-PA patients were transferred to the hub (18/31=58% telemedicine, 19/25=76% telephone; OR 0.44, 95%CI 0.11-1.59; p=0.2560). Although not statistically significant, these data suggest the need for further studies.

Consult duration was estimated using recorded time intervals (Table 3). “Time of Consent” marked the time the consultation began. In the telephone arm, times to “neurologic exam” erroneously appeared shorter because the previously performed ED practitioner’s exam was sometimes reported at time of telephone discussion. In telemedicine the consultant personally completed a history before performing the exam.

Obtaining consent is a process requiring discussion, and was essential in this trial.³¹ After consent, telemedicine consults took longer (10 minutes) than telephone since the telemedicine practitioner personally performed a history, examination, and review of the imaging. The improved decision-making in telemedicine may justify this time difference, though long term patient outcome trials would be needed for verification. The favorable time requirement for telemedicine consults (32 minutes) is still likely less than that of bedside consults, and has the advantage of eliminating driving time for both patients and practitioners.

Our results emphasize the need for efficient stroke-code policies, and rapid treatment strategies. Stroke-code times were generally not different between groups (overall Door to MD 7.6min, overall Door to Decision 97.8min). If the consultant were contacted immediately, without need for consent, these times would be more consistent with US guidelines.³² The “Decision to rt-PA” time emphasizes that telemedicine can quantitatively lessen neuronal loss,³³ as the telephone group required approximately 6 minutes longer simply to administer rt-PA (p=0.0190). This lessened time requirement may have resulted from the telemedicine practitioner’s consistent presence and encouragement during this period.

We found that telemedicine improves the consultant’s ability to obtain information prior to making treatment decisions. Data completion was 9% greater in telemedicine than telephone for key risk factors and NIHSS items. During telephone consults, despite encouraging the spoke practitioner to return to the bedside to reassess history and exam elements, the NIHSS was still often incomplete. The NIHSS questions showing > 15% missing data (gaze, visual fields, ataxia, sensory and neglect) in telephone are also items previously documented to show poor inter-observer reliability.^{10, 34, 35} How incompleteness of data collection directly impacts decision-making is not known, though this could lead to errors.

Despite the telemedicine system’s complexity, technical problems did not impact the successful completion of the trial. The site independence and Quality of Service¹⁸ technology are evidenced by the fact that only 1 consult was not possible due to technical failure. The 11% technical observations were generally due to either audio or video quality problems, and did not prevent the consultations. Of these technical observations, six were DICOM issues, and 1 was actually due to a delay in fax transmission. There was only 1 adjudication disagreement in the cases that reported technical problems, related to a potential aortic dissection and not due to noted audio difficulties.

Trial limitations must be noted. We cannot measure the actual increase in rt-PA use, since we did not collect data prior to the trial. However, during the year prior to trial initiation, only 1 facility had ED neurology support and rt-PA treatments were rare. Another limitation is that “Code Strokes” may not have been activated on all stroke patients presenting to the sites, so the true denominator is unknown.

We compared the telemedicine arm to a telephone- only arm because many ED practitioners attempt telephone discussions with specialists when no neurology consultant is physically available. Comparing telemedicine to no consultation at all would have been impractical and potentially unethical. The chosen design, however, underestimates the true benefit of telemedicine, since telemedicine was not compared to “placebo”.

Similarly, the telephone arm does not truly replicate standard “curbside” telephone practice, as our consultants were meticulous in determining onset time (e.g. personally calling witnesses), filling out detailed case report forms, and dictating recommendations into the spoke’s documentation record. These features, inherent to a clinical trial but unlikely in clinical practice, may have resulted in more complete consultations and fewer telephone disagreements. Real-world telephone practice would be less efficacious than our telephone arm.

We instituted trial procedures to reduce chances of adjudication committee unblinding. Rules restricted voting if there was concern for any member of the SDAC inferring randomization. The consultant was the only SDAC member who knew the randomization arm, and was excluded from the adjudication room during voting to minimize the potential for unblinding. Data was locked, and team members remained blinded until all adjudications were complete. Though personally viewing CT images may partially explain improved decision-making, we feel that teleradiology is an integral component of telemedicine and have therefore not separated the two.

We chose an intra-hospital randomization design to limit learning effect or Hawthorne effect to only 1 trial arm or to only 1 facility (that may not have been well matched to another). This trial assessed the 1 variable of adding telemedicine to stroke evaluations, and changed no care protocols, to avoid contentions that benefit was due to other elements of improved care. A total of 48 spoke ED practitioners initiated consults from 4 different spokes, with the majority of practitioners being involved in only 1–5 consults. The multiple sites and limited involvement of any 1 ED practitioner lessened any substantial learning effect.

In summary, the STRokE DOC study is the first trial to establish the benefit of telemedicine over telephone specifically for acute medical decision-making. Since rt-PA reduces stroke-related disability when administered correctly,^3,33,36 increasing the rapid and appropriate use of rt-PA will significantly benefit public health. Telemedicine is a viable solution that can now be added to the stroke armamentarium, enabling more practitioners to treat strokes rapidly and effectively, irrespective of location. Replication of these results, and long term patient- outcome trials, are still needed.