Implementation of Computerized Physician Order Entry is Associated With Increased Thrombolytic Administration for Emergency Department Patients With Acute Ischemic Stroke

Abstract

Objective

Electronic Health Record (EHR) systems with computerized physician order entry (CPOE) and condition-specific order sets are intended to standardize patient management and minimize errors of omission. However, the impact of these systems on disease-specific process measures and patient outcomes is not well established. We sought to evaluate the impact of CPOE-EHR implementation on process measures and short-term health outcomes for patients hospitalized with acute ischemic stroke.

Methods

We conducted a quasi-experimental cohort study of patients hospitalized for acute ischemic stroke with concurrent controls that took advantage of the staggered implementation of a comprehensive CPOE-EHR across 16 medical centers within an integrated healthcare delivery system from 2007 to 2012.

The study population included all patients admitted to the hospital from the Emergency Department (ED) for acute ischemic stroke with an initial neuroimaging study within 2.5 hours of ED arrival. We evaluated the association between the availability of a CPOE-EHR and the rates of 1) ED intravenous tissue plasminogen activator (IV tPA) administration, 2) hospital-acquired pneumonia (PNA) and 3) inhospital and 90-day mortality using doubly robust estimation models to adjust for demographics, comorbidities, secular trends, and concurrent primary stroke center certification status at each center.

Results

Of 10,081 eligible patients, 6,686 (66.3%) were treated in centers after CPOE-EHR had been implemented. CPOE was associated with significantly higher rates of IV tPA administration (rate difference 3.4%, 95% CI; 0.8%, 6.0%) but not with rate differences in PNA or mortality.

Conclusions

For patient hospitalized for acute ischemic stroke, CPOE use was associated with increased use of IV tPA.

Introduction

Background

Stroke is a major cause of death and a leading cause of serious long-term disability in the United States.[1,2] A systematic approach to the acute management of patients with ischemic stroke–including the timely administration of intravenous tissue plasminogen activator (IV tPA) for eligible patients, routine dysphagia screening and evaluation to prevent aspiration, and nursing protocols for frequent neurological assessments–can help avoid complications and improve outcomes.[3] As such, Primary Stroke Center (PSC) certification has been associated with improvements in the quality of care for patients with acute stroke.[4–6] However, these interventions have not been uniformly or consistently applied.[7–9]

Importance

The Emergency Department (ED) evaluation of patients with suspected stroke is focused on rapidly assessing eligibility for time-sensitive interventions such as IV tPA, a treatment that has been shown in multiple trials to improve neurological outcomes for acute ischemic stroke.[10–13] Although many hospitals utilize standardized care pathways and protocols to facilitate this process, there is still considerable variability in approach and compliance, even among PSCs.[9]

As such, stroke-specific order sets offer a way to facilitate the delivery of time-sensitive interventions, minimize errors of omission and initiate stroke care pathways in a standardized way. Electronic health records (EHRs) with computerized physician order entry (CPOE) are capable of efficiently capturing such clinical protocols into electronic orders and order sets by providing an immediately actionable menu of disease-specific orders that incorporate best practices and guideline concordant care. Thus, the recent rapid proliferation of CPOE-EHRs has resulted in a concomitant proliferation of electronic order sets. Preliminary data suggest that EHRs with condition-specific CPOE may improve adherence to clinical guidelines, as well as patient outcomes.[14–16] Despite its promise however, uptake of CPOE has been variable, and significant barriers exist.[17] Other realized or potential benefits of comprehensive EHRs include improved documentation, cross-checks on medication administration, easily accessible and legible clinical data, and the potential for early identification of subtle signs of clinical deterioration (e.g., vital sign trends.)[18] However, some have suggested CPOE may result in delays in care and over-utilization of medical resources and the benefits of CPOE on health outcomes are not firmly established.[18–22]

Goals of This Investigation

We took advantage of a fortuitous natural experiment enabled by the staggered implementation of CPOE-EHR across 16 centers within a large integrated delivery system in order to evaluate the impact of CPOE-EHR on key process measures and outcomes for patients with acute ischemic stroke.

Methods

Setting

Kaiser Permanente Northern California is a large integrated health care delivery system that provides comprehensive medical care for more than 3.7 million members at medical centers across the region. Medical center EDs are staffed by more than 500 salaried (board-certified or eligible) emergency physicians and have an annual adult ED visit census of approximately one million visits a year. Between 2006 and 2010, in staggered fashion across 16 medical centers, Kaiser Permanente Northern California rolled out a commercially-available comprehensive inpatient EHR (Epic, Verona, WI) including CPOE with internally-developed condition-specific order sets. The implementation schedule included two phases: the first phase incorporated a limited EHR that allowed for physician documentation only (Documentation Only-EHR) and the second phase included a full implementation of a comprehensive EHR with CPOE (CPOE-EHR.)

Independently, the hospital system also pursued staggered implementation of PSC certification across all of its medical centers between 2004 and 2012. All system hospitals pursuing PSC status enrolled in the American Heart Association’s “Get With The Guidelines” reporting system. The rollout of the certified EHR across the system was determined a priori at the medical center level and was not systematically influenced by any individual service line such as ED or stroke care.[23] The staggered implementation of these concomitant yet independent processes allowed for a natural experiment to evaluate the impact of CPOE-EHR implementation across 16 medical centers while controlling for secular trends. The timetable for implementation of the EHR and PSC certification is shown in Figure 1.

Figure 1.

During the study period, an ED stroke order set became available, in electronic format only, as soon as the CPOE-EHR became available at each medical center. The order set was designed to be used for patients presenting with signs or symptoms of acute stroke. There were no system- or medical center-wide mandates to utilize the order set; the decision to use the order set use for each patient was at the discretion of the treating physician. Prior to CPOE-EHR implementation, ordering practices varied from facility to facility, but virtually all ED orders were written on generic paper order forms. Orders for dosing and administering IV tPA were usually completed on a separate paper form.

The core components of the ED stroke order set are shown in Table 1. In contrast to other ED order sets (e.g. “Abdominal Pain” and “Altered Level of Consciousness”), the stroke order set incorporates clinical decision support tools to facilitate a rapid assessment of eligibility for IV tPA and more uniform and timely nursing assessments (including dysphagia screening and stroke severity scoring.) The key assistive features relating to IV tPA, stroke severity assessment, and dysphagia screening were maintained throughout the study period.

Table 1.

Components of the Stroke Order Set

IV tPA

Indications

Contraindications

Exclusion criteria

Criteria for consideration of intra-arterial thrombolysis

Web links

ABCD2

NIHSS calculator

Lab order menu

Neuroimaging

CT head

CT angiogram

MRI

Carotid ultrasound

Nursing vital signs/monitoring

Nurse completes mNIHSS every four hours

Neuro checks per unit standard

Cardiac monitoring

Blood pressure parameters

Swallowing screen

Bedside

Diet order

Consultation order menu

IV tPA stands for intravenous tissue plasminogen activator;

NIHSS, National Institutes of Health stroke scale; CT, computed tomography;

MRI, magnetic resonance imaging; and mNIHSS, modified National Institutes of Health stroke scale.

The Kaiser Foundation Research Institute Institutional Review Board reviewed and approved the study protocol under a waiver of informed consent.

Selection of Participants

We identified adults (age > 18) with an ED visit between 2007–2012 that resulted in a hospitalization where the primary hospital discharge diagnosis was ischemic stroke as defined by the Get With The Guidelines program (ICD-9-CM codes 433.00, 433.01, 433.10, 433.11, 433.20, 433.21, 433.30, 433.31, 433.80, 433.81, 433.90, 433.91, 434.00, 434.01, 434.10, 434.11, 434.90, 434.91.) We included only the initial (index) hospitalization for ischemic stroke during the study period; we excluded patients who were not evaluated in system EDs and patients who were transferred from other acute-care hospitals. We also excluded encounters with an index ED visit that occurred during the three month run-in phase of CPOE-EHR implementation at each center, based on prior data suggesting that there are temporary practice changes that occur during the initial EHR implementation.[22] Finally, we excluded patients with an initial neuroimaging study that was completed within 12 hours prior to ED arrival or more than 2.5 hours (90^th percentile) after ED arrival in order to focus on acute stroke management.

Interventions

Our primary comparison was between stroke patients treated at a medical center when a CPOE-EHR and ED stroke order set was available versus those treated at a medical center when a Documentation Only-EHR without CPOE was available.

We hypothesized that acute ischemic stroke patients treated at centers with access to a CPOE-EHR would be: 1) more likely to receive care included in the stroke order set, e.g. IV tPA in the ED, 2) have fewer inpatient complications, e.g. fewer inpatient diagnoses of pneumonia (including aspiration pneumonia) and 3) have lower inpatient and short-term mortality rates.

Outcomes

The three primary outcomes were: 1) IV tPA administration in the ED. Acute ischemic stroke patients treated with IV tPA were identified using EHR databases and validated with chart review. Patients for whom IV tPA infusion was initiated but stopped prior to completion (due to bleeding complications or resolution of symptoms) were included. 2) Hospital-acquired pneumonia. We used hospital discharge ICD-9 codes for pneumonia (481, 482, 483, 484, 485, 486) and aspiration pneumonia (507 and 507.1.) Because of potential misclassification of pneumonia subtypes, we chose all inpatient pneumonias as our primary outcome measure. 3) Short-term mortality: in-hospital and at increments out to 90 days. In-hospital deaths were identified using hospital discharge data and all subsequent deaths were identified using record linkage to the Social Security Death Master File and the California State Department of Vital Statistics. Matching of mortality data across sources was performed with IBM QualityStage and mortality increments were calculated from the date of index admission.[24]

Outcome Validity Checks

Three investigators (DWB, DP, MK) reviewed the EHRs for all eligible patients who were identified by system databases as having received IV tPA. After confirmation that IV tPA had been administered, additional variables were abstracted from medical records in a convenience sample of approximately 75% of patients. These variables included: physician-documented last known well time, time of IV tPA administration, first physician-documented National Institutes of Health Stroke Scale (NIHSS) score prior to IV tPA, and symptomatic intracerebral hemorrhage (sICH) after tPA.[10] Chart validation followed published best practices except that abstractors were not blinded to the larger study hypotheses (which were not based on chart review data.) [25]

Methods and Measurements

Patient-level clinical data were abstracted from existing health system databases and using discharge diagnosis (ICD-9) codes as described previously.[26,27] Electronic data sources are consistent in their definitions across medical centers. We abstracted descriptive counts, patient encounter-level demographics, comorbidities (based on a modified summary Elixhauser comorbidity score, a composite score with 30 comorbidity categories), mode of arrival (ambulance/emergency medical services versus any other), and use of the ED stroke order set.[28] Additionally, we abstracted nursing flowsheet documentation of dysphagia screening in the ED (completed and documented in ED or not.)

The PSC model mandates rapid identification and close tracking of stroke patients, an emphasis that has been shown to affect ED diagnosis and treatment patterns.[27,29] PSC status was categorized as pre-certification, intra-certification and post-certification, as in our prior work.[27] We chose the Elixhauser comorbidity score because it has been proven to be a good predictor of inpatient comorbidity risk and included statin use status because this has been associated with improved outcomes in stroke patients.[30,31] Statin use was defined as follows: a patient was defined as a current user if the most recent prescription filled would cover the day of admission and as being as some user if they had had a statin prescription filled within 365 days prior to admission but the most recent prescription filled did not cover the day of admission. An off-hours ED arrival was defined as any ED visit on a weekend, holiday or between the hours of 11PM and 7AM.

For a supplemental analysis after implementation of the CPOE-EHR, we also abstracted a variable not routinely retrievable prior to CPOE-EHR implementation: the modified National Institutes of Health Stroke Scale (mNIHSS) as recorded by the ED nurse. The mNIHSS is a measure of stroke severity measure that may be more reliable than the full NIHSS.[32] We abstracted the first complete score recorded in the ED and categorized this score into ranges (0–2, 3–6, 7–15, >15) as validated in prior studies.[33] Initial stroke severity scores have been shown to reliably predict the morbidity and mortality associated with ischemic stroke from discharge to 3 months.[34,35]

Analysis

To estimate the effect of CPOE-EHR, we used a doubly robust estimation that uses a multivariate regression to examine the likelihood of patient outcomes (IV tPA administration, inpatient pneumonia, and mortality) weighted by inverse-probability of treatment (CPOE-EHR use.) Double robust estimation is a promising analytic approach to elucidating causal effects from observational datasets that contain multiple possible confounders and we chose this approach as a safeguard for potential model misspecification. [36] We chose the covariates in in our model based after review of the relevant literature and selecting variables based on availability, clinical face validity and bivariate differences across comparison groups. The covariates in our outcome model include the treating medical center’s PSC status, patient age, sex, race/ethnicity, neighborhood socioeconomic status (based on census block groups,) smoking status, statin use within the last year (current, some use, none), mode of arrival to ED (ambulance or other), calendar year, modified Elixhauser score, and indicators for each medical center to account for facility-specific differences. Any Intensive Care Unit (ICU) admission at any point during the hospitalization was included as covariate in the outcome model for mortality only (due to the expected co-linearity of ICU admissions with IV tPA administration.)

We included the same set of covariates in the treatment model with the exception of indicators for medical center, because CPOE-EHR was rolled out by medical center. We report adjusted rates to represent the average rate as if everyone in the population was treated using the Documentation Only-EHR or CPOE-EHR, and the differences between these rates.

For the outcome validity comparisons, differences in baseline characteristics as well as secondary outcomes from the chart review, including duration from last known well time to IV tPA administration, were assessed with bivariate analysis. Chi-square tests were used for categorical variables; continuous values were assessed with the t-test when normally distributed and with the Wilcoxon-Mann-Whitney non-parametric test when not normally distributed. Five percent of charts were independently reviewed by a second abstractor to assess inter-rater reliability, measured as percent agreement. For all analyses, statistical significance was defined as a p-value <0.05.

Supplemental Analysis

An important component of the CPOE-EHR evaluated in this study was the presence of a condition-specific order set with assistive attributes as described above and in Table 1. Since stroke order set use was not universal after CPOE-EHR implementation, we compared the outcomes of patients for whom the stroke order set was used to those where the stroke order set was available in the CPOE-EHR but not used. We hypothesized that use of the stroke order set, rather than simply the availability of the stroke order set, would improve short-term outcomes. For example, an emergency physician treating a patient with suspected stroke would not forget to order neuroimaging–order set or not–but she/he might, without the order set, forget to order dysphagia screening or frequent neuro checks. Thus, as a supplementary analysis, only among the subset of patients who were treated after CPOE-EHR implementation, we compared outcomes between patients whose ED physician did and did not use the ED stroke order set.

Our supplementary analysis included (as above) the initial mNIHSS value, but was otherwise the same as for the impact of CPOE-EHR availability. Our supplemental modeling included, as a sensitivity analysis for missing data, logistic regression models for our primary outcomes and with the same covariates including multiple imputation for missing mNIHSS scores (multiple imputation was not possible in our doubly robust modeling.)

We used Stata SE 13.1 (StataCorp LP, College Station, TX) for all statistical analyses.

Results

Characteristics of study subjects

We identified 13,755 patient encounters with an ED visit and primary hospital discharge diagnosis of ischemic stroke during the study period. During this same time period, there were 604,040 hospital admissions stemming from an ED visit across the system. Of the 13,755 ischemic stroke encounters, 529 were excluded because the ED visit occurred during the three-month EHR run-in phase, 1195 were excluded for having neuroimaging in the 12 hours prior to ED arrival, and 1212 were excluded for having neuroimaging > 2.5 hours after ED arrival. We also excluded 738 repeat encounters for the same patient after an index hospitalization. Of the 10,081 study eligible patient encounters, 6,686 (66.3%) occurred after the implementation of the CPOE-EHR. The ED stroke order set was used in 3,677 (55%) of these post-CPOE-EHR patients. Descriptive patient characteristics and unadjusted outcomes are shown in Table 2.

Table 2.

Patient Encounter Characteristics CPOE-EHR vs. Documentation Only-EHR (N=10,081)

	Total	Doc-EHR in Use	CPOE-EHR in Use
	N=10081	N=3395 (33.7)	N=6686 (66.3)
Encounters Post-PSC, No. (%)	7319 (72.6)	1738 (51.2)	5581 (83.5)
Median Age (IQR)	76 (19)	75 (19)	77 (19)
Male Sex, No. (%)	5265 (52.2)	1748 (51.5)	3517 (52.6)
Ethnicity, No. (%)*
White	6723 (66.7)	2199 (64.8)	4524 (67.7)
Black	1104 (11.0)	461(13.6)	643 (9.6)
Hispanic	669 (6.6)	147 (4.3)	522 (7.8)
Asian	1260 (12.5)	429 (12.6)	831 (12.4)
Other	325 (3.2)	159 (4.7)	166 (2.5)
Non-Low SES, No. (%)	7453 (73.9)	2516 (74.1)	4937 (73.8)
Non-Smoker, No. (%)	7887 (78.2)	2283 (67.3)	5604 (83.8)
Current Statin Use, No. (%)§	3857 (38.3)	1307 (38.5)	2550 (38.1)
Some Statin Use, No. (%)†	1451 (14.4)	501 (14.8)	950 (14.2)
Elixhauser, Mean	9.69	8.9	10.1
ED Arrival by EMS, No. (%)	4822 (47.8)	1485 (43.7)	3337 (49.9)
After-Hours Visit, No. (%)**	3650 (36.2)	1195 (35.2)	2455 (36.7)
IV tPA Use, No. (%)	708 (7.0%)	111 (3.3)	597 (8.9)
Hospital-Acquired PNA, No. (%)	613 (6.1)	215 (6.3)	398 (6.0)
In-Hospital Mortality, No. (%)	527 (5.2)	222 (6.5)	305 (4.6)
90-Day Mortality, No. (%)	1585 (15.7)	557 (16.4)	1028 (15.4)

CPOE stands for computerized provider order entry; EHR, electronic health record; Doc; Documentation only; PSC, primary stroke center; SES, socioeconomic status; ED, emergency department; and EMS, emergency medical services, PNA, pneumonia

^*Small percentage of unknown ethnicity

^§Current statin user: if the most recent prescription filled would cover the day of admission

^†Some statin use: those who had statin filled within 365 days prior to admission but the most recent prescription filled didn’t cover the day of admission

Main results

The availability of CPOE-EHR was significantly associated with ED IV tPA administration (rate difference 3.4%, 95% CI; 0.8%, 6.0%). Model results are shown in Table 3.

Table 3.

Adjusted rate of outcomes in patients with stroke: CPOE-EHR vs. Documentation Only-EHR (N=10,081)

	Adjusted Rate Doc-EHR	Adjusted Rate CPOE-EHR	Rate difference w/CPOE-EHR
	% of Pts (95% CI)	% of Pts (95% CI)	% of Pts (95% CI)
IV tPA in ED	3.8% (1.4%, 6.2%)	7.2% (6.6%, 7.9%)	3.4% (0.8%, 6.0%)
Pneumonia	7.3% (3.5%, 11.0%)	5.7% (5.0%, 6.4%)	−1.6% (−5.4%, 2.3%)
Mortality
In-hospital	7.8% (4.9%, 10.7%)	4.8% (4.1%, 5.6%)	−3.0% (−6.0%, 0.03%)
7-day	7.5% (3.5%, 11.4%)	4.9% (4.2%, 5.6%)	−2.5% (−6.6%, 1.5%)
30-day	15.2% (11.3%, 19.1%)	11.2% (10.2%, 12.2%)	−4.0% (−8.2%, 0.1%)
60-day	15.8% (11.0%, 20.7%)	14.1% (13.0%, 15.2%)	−1.7% (−6.7%, 3.3%)
90-day	16.3% (12.4%, 20.1%)	15.8% (14.6%, 16.9%)	−0.5% (−4.3%, 3.3%)

CPOE stands for computerized provider order entry; EHR, electronic health record; and IV tPA, intravenous tissue plasminogen activator.

Model: logistic regression with inverse-probability weighting (STATA command teffects aipw) for doubly robust estimates. Adjusted rate Doc-EHR represents the average rate as if everyone in the population was treated using the Doc-EHR; Adjusted rate CPOE-EHR represents the average rate as if everyone in the population was treated using the CPOE-EHR.

We reviewed a convenience sample of medical records from 552 of 708 (78%) patient encounters with confirmed administration of IV tPA. Inter-rater reliability for chart review data was excellent. We observed 100% agreement for reported data. Among charts of patients treated with IV tPA, there were no statistically significant (p≥.0.05) differences in sICH rates (overall rate 24/552 [4.4%]) across CPOE-EHR and Order Set cohorts. Nor were there significant differences in physician initial documented NIHSS mean value (overall mean score of 8.3) or time of symptom onset to IV tPA administration in the ED (overall mean of 148.5 minutes.)

Supplemental Analysis

Descriptive results of the supplementary analysis among only the subset of study patients who were treated using the CPOE-EHR are shown in Table 4. Comparing patients who were treated with the stroke order set with those who were not, a documented mNIHSS was more likely to be missing (12.5% vs. 3.5%) in the no order set group as was documentation of dysphagia screening (24.4% vs. 3.1%.) When completed, the mNIHSS had a very low rate of missing items (<1%.)

Table 4.

Supplementary Analysis among patients with Stroke Treated using CPOE-EHR: Patient Characteristics by Stroke Order Set use (N=6,686)

	Total	Stroke Order Set Not Used	Stroke Order Set Used
	N=6686	N=3009 (45.0)	N=3677 (55.0)
Encounters Post-PSC, No. (%)	5581 (83.5)	2413 (80.2)	3168 (86.2)
Median Age (IQR)	77 (19)	77 (18)	76 (19)
Male Sex, No. (%)	3517 (52.6)	1615 (53.7)	1902 (51.7)
Ethnicity, No. (%)
White	4524 (67.7)	2085 (69.3)	2439 (66.3)
Black	643 (9.6)	295 (9.8)	348 (9.5)
Hispanic	522 (7.8)	235 (7.8)	287 (7.8)
Asian	831 (12.4)	328 (10.9)	503 (13.7)
Other	166 (2.5)	66 (2.2)	100 (2.7)
Non-Low SES, No. (%)	4937 (73.8)	2157 (71.7)	2780 (75.6)
Non-Smoker, No. (%)	5604 (83.8)	2522 (83.8)	3082 (83.8)
Current Statin Use, No. (%)	2550(38.1)	1170 (38.9)	1380 (37.5)
History of Statin Use, No. (%)	950 (14.2)	412 (13.7)	538 (14.6)
Elixhauser, Mean ED Arrival by EMS, No. (%)	10.10	9.8 1440 (47.9)	10.4 1897 (51.6)
	3337 (49.9)
Dysphagia Documentation	5837 (87.3)	2274 (75.6)	3563 (96.9)
Initial mNIHSS, No. (%)
0–2	3261 (48.8)	1517 (50.4)	1744 (47.4)
3–6	1394 (20.9)	535 (17.8)	859 (23.4)
7–15	946 (14.2)	346 (11.5)	600 (16.3)
15+	580 (8.7)	234 (7.8)	346 (9.4)
Unknown	505 (7.6)	377 (12.5)	128 (3.5)
mNIHSS Non-Missing, Mean	4.9	4.5	5.2
IV tPA Use, No. (%)	597 (8.9)	129 (4.3)	468 (12.7)
Hospital-Acquired PNA, No. (%)	398 (6.0)	187 (6.2)	211 (5.7)
In-Hospital Mortality, No. (%)	305 (4.6)	139 (4.6)	166 (4.5)
90-Day Mortality, No. (%)	1028 (15.4)	495 (16.5)	533 (14.5)

PSC stands for primary stroke center; SES, socioeconomic status; mNIHSS, modified National Institutes of Health stroke scale; ICU, intensive care unit; ED, emergency department; EMS, emergency medical services.

Table 5 shows doubly robust estimation results for the supplementary analysis. After adjustment, stroke order set use was associated with increased rates of ED IV tPA administration, lower rates of hospital-acquired pneumonia and improved mortality rates at 30, 60 and 90 days after hospital admission. In our sensitivity analysis, multiple imputations to account for missing mNIHSS (results not shown) did not change the statistical significance (p <0.05) of these associations.

Table 5.

Adjusted rates of outcomes among patients with stroke treated using CPOE-EHR: Stroke order set use vs. no stroke order set use ( N=6,686)

	Adjusted Rate No Order Set	Adjusted Rate Order Set	Rate difference w/Order Set
	% of Pts (95% CI)	% of Pts (95% CI)	% of Pts (95% CI)
IV tPA in ED	4.7% (3.7%, 5.6%)	12.6% (11.4%, 13.8%)	8.0% (6.4%, 9.5%)
Pneumonia	7.5% (6.3%, 8.8%)	4.9% (4.2%, 5.6%)	−2.6% (−4.0%, −1.2%)
Mortality
In-hospital	4.6% (3.8%, 5.4%)	4.1% (3.4%, 4.8%)	−0.5% (−1.5%, 0.5%)
7-day	5.1% (4.3%, 6.0%)	4.3% (3.6%, 5.0%)	−0.8% (−1.9%, 0.3%)
30-day	11.7% (10.5%, 12.9%)	10.0% (9.0%, 11.0%)	−1.7% (−3.2%, −0.2%)
60-day	15.3% (13.9%, 16.7%)	12.7% (11.5%, 13.8%)	−2.6% (−4.3%, −0.9%)
90−day	16.9% (15.5%, 18.4%)	14.1% (12.8%, 15.3%)	−2.9% (−4.7%, −1.1%)

CPOE stands for computerized provider order entry; EHR, electronic health record; IV tPA, intravenous tissue plasminogen activator.

Model: logistic regression with inverse-probability weighting (STATA command teffects aipw) for doubly robust estimates. Adjusted rate doc-EHR represents the average rate as if everyone in the population was treated using the doc-EHR; Adjusted rate CPOE-EHR represents the average rate as if everyone in

Limitations

Our study should be interpreted in the context of certain limitations. First, this was a retrospective study that relied primarily on automated electronic data for important covariates and outcomes. However, we chose variables with high fidelity as validated by chart review and as assessed in prior research conducted within our system.[15,31,37] Despite the presence of secular trends in stroke management during the study period, we mitigated these temporal effects by taking advantage of the staggered implementation of the CPOE-EHR as well as through the staggered implementation of a key potential confounder (PSC certification) and by adjusting our model for trends over time. Our modeling was designed to adjust for other disparities between study groups (e.g., more patients in the CPOE cohort were treated after PSC certification and more arrived by EMS than in the Documentation-Only Cohort.) While our analysis plan also addressed confounding on the hospital level, we cannot exclude the possibility that on the provider-level, more conscientious physicians were more likely to utilize the stroke order set.

An additional limitation of our supplemental analysis is possible confounding by indication. In the supplementary order set comparison, patients for whom the order set was used may have had more obvious presentations of stroke and thus were more likely to be candidates for IV tPA and have other confounding characteristics that make them less likely to have adverse short-term outcomes. To mitigate this concern we found that 1) our CPOE-EHR Cohort (intention-to-treat) analysis demonstrates similar results as those in the stroke order set use comparison, albeit of lesser magnitude, and 2) stroke severity and comorbidity burden were either not significantly different between comparison groups or were skewed towards greater severity or burden in the treatment groups. Nonetheless, residual confounding may still be present since this was an observational study.

Patients with “wake up stroke” or who present off-hour are more likely to have worse short-term outcomes.[38,39] However, because the stroke order set includes the only compilation of ED stroke-specific orders (such as dysphagia evaluation), there is no compelling a priori reason why it would be used more or less frequently in patients with delayed or off-hour presentation of stroke than those presenting within the IV tPA window. A final limitation is that two relevant data elements were outside the scope of this study: 1) acute ischemic stroke rates in patients who did not receive this as their final discharge diagnosis and 2) rates of patients eligible for IV tPA who did not receive it.

Discussion

We found that implementation of an integrated CPOE-EHR and the use of a disease-specific ED stroke order set were each associated with more frequent use of IV tPA in acute ischemic stroke patients. These absolute thrombolytic rates are above those that have typically been reported in the literature.[40,41] In our supplemental analysis, we observed a lower risk of inpatient pneumonia and a mortality benefit at 30 to 90 days post-admission amongst patients in which the CPOE ED stroke order set was used.

A CPOE-EHR is a complex intervention that can have numerous consequences (intended and otherwise) on both care processes and clinical outcomes.[18] In this study, we focused on the impact of the CPOE-EHR in a specific patient population for which clinicians had access to a number of assistive tools–including an ED stroke order set with embedded decision support. Prior work has demonstrated the beneficial effects of paper-based and electronic order sets for certain diseases and conditions such as diabetes, chronic obstructive pulmonary disease, renal colic, and acute myocardial infarction.[15,16,37,42–44] This evidence suggests that use of evidence-based order sets can lead to more guideline-concordant care with a lower risk of errors of omission and administration. A unique attribute of this study was the concurrent implementation of a CPOE-EHR at each center and a system-wide effort to implement a Joint Commission-based quality initiative. These two (independent) processes intersected at the stroke order set, which was informed and enhanced by activities required to prepare for and to maintain PSC certification, especially the requirements for urgent evaluation and timely consideration for IV tPA in the ED and dysphagia screening followed by, when indicated, NPO orders and formal evaluations by speech therapists.

Despite some continued skepticism about the benefit of IV tPA on patient outcomes, the literature continues to progress towards expanded indications.[11,12,45] And, we are increasingly realizing the importance of symptom onset to treatment time as an important indicator of ultimate outcomes for acute ischemic stroke.[13,46] Recent literature has associated improvement in stroke severity scores at 2 and 24 hours with improved functional outcomes at three months.[47] We also recognize that improvements in preventive, acute, and follow-up care make a difference in outcomes.[48,49] Despite these multi-dimensional processes, we found an improvement in IV tPA administration rates with availability of a CPOE-EHR as well as even greater improvement when an ED stroke order set was used.

We recognize that prior studies have not demonstrated a consistent benefit for dysphagia screening.[50] However, there is evidence that early initiation of inpatient speech therapy can improve patient outcomes and thus we submit that it is plausible that such screening, perhaps prompted by ED stroke order set use, could ultimately lead to decreased rates of aspiration pneumonia among acute stroke patients.[48]

Our supplemental modeling results suggested a mortality benefit accrued with the use of the ED stroke order set. While this EHR tool is not likely to be solely responsible for the improved outcomes observed in this study, it may represent a proxy measure for optimum care for certain patients, in particular those for whom the speed of initiating therapy, the completeness of information available to the clinician, and the intensity of inpatient care make a real difference in short term outcomes. For example, CPOE, and the stroke order set in particular, may, as shown in other recent investigations, have helped attenuate the effect of off-hour arrival on outcomes.[51] Furthermore, repeated use of the order set on the provider level may have had an iterative educational benefit (e.g., familiarity with indications/contraindications to IV tPA therapy) that resulted in improved care–whether the order set was used in a particular instance or not. Ultimately, we see the order set itself as optimizing the confluence of two separate processes: a robust CPOE-EHR that integrates care across many providers and locations while limiting errors of ignorance and omission, combined with a quality initiative that has identified disease-specific best practices and guidelines.

We believe that our findings represent a dawning era of the EHR: one that blends convenience and best practices. Decision support can be integrated into the EHR in a way that supports best practices but does not overload and distract the provider with excess clicks or busy space. Not all emergency conditions will be amenable to the development of such interventions, but for specific, time-sensitive conditions, perhaps this study might help further justify a move towards developing informed and non-intrusive physician order entry systems that are guided by best practice guidelines. This study, to us, provides tangible evidence that the true promise of the CPOE-EHR is beginning to be realized.

In summary, during a staggered implementation of a CPOE-EHR across medical centers within a large integrated health system, the availability of a CPOE-EHR with an ED stroke order set and specific use of this order set was associated with increased use of IV tPA.