Implementation of Real-Time Electronic Clinical Decision Support for Emergency Department Patients with Pneumonia Across a Healthcare System

Abstract

A real-time electronic CDS for pneumonia (ePNa) identifies possible pneumonia patients, measures severity and antimicrobial resistance risk, and then recommends disposition, antibiotics, and microbiology studies. Use is voluntary, and clinicians may modify treatment recommendations. ePNa was associated with lower mortality in emergency department (ED) patients versus usual care (Annals EM 66:511). We adapted ePNa for the Cerner EHR, and implemented it across Intermountain Healthcare EDs (Utah, USA) throughout 2018. We introduced ePNa through didactic, interactive presentations to ED clinicians; follow-up visits identified barriers and facilitators to use. Email reminded clinicians and answered questions. Hospital admitting clinicians encouraged ePNa use to smooth care transitions. Audit-and-feedback measured utilization, showing variations from best practice when ePNa and associated electronic order sets were not used. Use was initially low, but gradually increased especially at larger hospitals. A user-friendly interface, frequent reminders, audit-and- feedback, a user survey, a nurse educator, and local physician champions are additive towards implementation success.

Introduction/Background

Pneumonia is an acute infection of the lungs that fills alveoli with inflammatory cells and fluid, making it difficult to breathe, take in oxygen, and expel carbon dioxide. Pneumonia is the eighth leading cause of death in the United States, with more than six million cases annually, one million hospitalizations, and costs over 7 billion dollars for inpatient treatment alone. 1 , 2 However, pneumonia care varies dramatically across healthcare providers and institutions.

Pneumonia treatment is neither straightforward nor simple. When a patient is suspected of having pneumonia, clinicians must (1) assess symptoms and clinical findings and determine whether pneumonia is likely compared to another diagnosis, (2) identify the most appropriate site for treatment, 3) determine whether the bacteria causing the infection is likely to be resistant to commonly prescribed antibiotics, and then select the appropriate antibiotic. Correctly performing these tasks is critical for patient safety but given the complexity of these questions and the fundamental limitations of human decision-making, too often care varies from best practice. For example, without pneumonia clinical decision support (CDS), a provider might discharge a patient home on oral antibiotics instead of arranging for hospital admission to provide needed 24/7 nursing care, electronic monitoring and intravenous antibiotics. Studies have demonstrated variability in rates of hospital admission between different institutions,^3,4 and two-fold variability in rate of hospitalization between physicians in a single emergency department (ED).⁵

Well-established scientific guidelines exist for pneumonia treatment, but they are underused. Decades of research, quality improvement, and policy work led to the development, deployment, and validation of pneumonia treatment guidelines; most recently, the Infectious Disease Society of America (IDSA) and the American Thoracic Society (ATS) 2007 pneumonia treatment guideline.⁶ Guidelines are intended to help clinicians accurately diagnose pneumonia, determine the most appropriate treatment setting, and deliver the most effective treatment. The 2007 guideline was the primary knowledge base behind our real-time, electronic pneumonia CDS tool (ePNa).

Effectively deployed pneumonia CDS should standardize treatment and improve patient safety. Paper-based clinical decision support improves pneumonia treatment, but not enough to fully optimize it. Informed by the 1993 ATS pneumonia treatment guideline, our team deployed a pneumonia treatment CDS tool across Intermountain Healthcare (Utah, USA) starting in 1995 using paper-based flow charts and standardized written order sets. This intervention was associated with reduced mortality for pneumonia patients.^7,8 Although the results showed effectiveness, the paper-based CDS pneumonia tool did not integrate well into clinical workflow, making widespread adoption difficult. Building on this early work, our team integrated guideline recommendations and best available evidence to develop ePNa for the Intermountain Healthcare (Intermountain) electronic health record (EHR). ePNa extracts real-time data from the EHR to guide diagnosis, risk stratification, and treatment.

Favorable Clinical Outcome Data

ePNa operating in Intermountain’s legacy EHR was deployed in four Salt Lake Valley EDs beginning in May 2011, and during 2012 was used by ED physicians in 63% of pneumonia patients. We reported results of a quasi-experimental implementation study of ePNa in the four EDs compared with three similar Intermountain hospitals along Utah’s urban corridor that served as usual care, concurrent controls.⁹ All-cause, 30-day mortality among patients with community-acquired pneumonia where ePNa was available was significantly lower compared to three usual care hospitals (OR: 0.53; 95% CI: 0.28 to 0.99). Unadjusted mortality was 7.8% in usual care hospitals versus 5.2% in the ePNa hospitals. During a baseline year prior to ePNa implementation, mortality of pneumonia patients was similar between the 7 hospitals. Patients with higher severity illness were more likely to be admitted to ePNa hospitals for treatment versus control hospitals where more of those patients were sent home. Patients in the ePNa hospitals were more likely to receive guideline recommended antibiotics.

Goals

Based on the value of ePNa demonstrated in the four Salt Lake Valley EDs, the Intermountain Office of Research funded a large-scale effort to develop and deploy ePNa for Intermountain’s version of the newly installed Cerner EHR (iCentra) across all 22 Intermountain hospitals. The deployment of ePNa was designed and conducted in step-wise fashion according to a prespecified implementation trial (ClinicalTrials.gov Identifier: NCT03358342). Our goals were:

1. Adapt ePNa from its legacy version to function well in iCentra. To accomplish this, we wanted to learn from clinician user observations, conduct a formal survey, and incorporate their suggestions to continuously improve ePNa function.

2. Achieve 80% ED physician utilization of ePNa in patients diagnosed with pneumonia at Intermountain hospitals. We targeted 80% rather than 100% use because of diagnostic uncertainty during ED patient care, and to be realistic considering ePNa’s “opt in” design..

3. Identify effective methods of implementing CDS for EDs, including barriers and facilitators.

4. Measure processes of care and clinical outcomes associated with implementation of ePNa after trial completion.

Description of ePNa

ePNa is an innovative, real-time electronic CDS tool for pneumonia treatment in the ED. No comparable tools have been described in published literature.¹⁰ It is unique in having been deployed and utilized by ED clinicians with published favorable clinical outcome data.

ePNa logic incorporates the Five Rights Framework. (Table 1)¹¹ ePNa integrates a pneumonia detection system with a management tool that delivers the right information to ED clinicians caring for patients with suspected pneumonia. Its detection system helps identify ED patients who might have pneumonia based on their presenting symptoms, physical exam, laboratory and radiographic findings. The clinical management system of ePNa evaluates each patient’s severity of illness to recommend (1) which patient disposition (outpatient care, inpatient ward care, or the intensive care unit) is the safest setting for pneumonia treatment, (2) which antibiotics should be prescribed, and (3) which microbiology studies (e.g., blood culture) are warranted. Patients with lower severity diseases treated at home have been shown to be more satisfied with their care and to return to work and/or usual activities more rapidly compared with similar severity patients admitted to the hospital.12 ^,13 Direct cost of care is 20 times higher for comparable patients admitted to the hospital versus those treated at home. However, more severely ill patients are more safely treated by direct admission to a hospital ward or intensive care unit.¹⁴

Table 1:

ePNa and the Five Rights Framework of Decision Support

Right	ePNa Attribute
1. Deliver the right information	Chief complaint, vital signs, chest imaging, age, laboratory findings, microbiology results and antibiotic resistance information
2. To the right person	Clinicians at the point of pneumonia care
3. Using the rightformat	Simple, single recommendations for diagnosis, treatment, and disposition per screen with accompanying supportive data.
4. In the right channel	Embedded electronically in the health record
5. At the right time	From the beginning, throughout, and the end of the pneumonia episode of care.

Hospital admission from the ED requires transition of care to a new setting and different providers, putting patients at risk if patient information and initial treatment are not effectively communicated. Ideally, initial ED treatment comprises best practice that is smoothly transitioned and continued in the hospital. However, if the inpatient team makes different treatment decisions based on individual clinician variability rather than patient-centered illness features, there is discontinuity of care and potential for patient harm. Similarly, patients returning for follow-up a day or two following their initial ED encounter are often seen by a different clinician. Standardizing clinical practice facilitates continuity of care by different providers.

Technical description of ePNa

ePNa consists of a pneumonia detection system combined with a management tool seamlessly integrated into usual ED physician workflow through a single user interface.

Detection system

The ePNa detection system is embedded in the iCentra decision support environment. It leverages tools designed to run prediction and classification algorithms in a clinical setting. It operates by consuming a stream of 40 clinical data elements captured from the EHR during ED patient care (Figure 1). The ePNa screening framework uses a natural language processing (NLP) subsystem to identify information in free-text reports to determine the presence of pneumonia. The detection system relies on a probabilistic Bayesian network to calculate likelihood of pneumonia.

Figure 1:

Screening tool probabilistic logic to determine likelihood of pneumonia. Size of the box approximates the relative importance of the different data elements.

When the combination of data results in a probability of pneumonia ≥ 40%, ePNa alerts the clinician using asynchronous messaging. At this point, the clinician can launch the tool, but may opt out if the patient is not believed to have pneumonia. Physicians may identify patients with a calculated risk of < 40% estimated pneumonia likelihood and still activate the management tool. The 40% threshold yields a 50% positive predictive value of the pneumonia alert, thereby decreasing “alert fatigue” and limiting negative impact on the ED clinical environment.¹⁵ After confirmation of pneumonia, ePNa takes the clinician through risk-stratification and recommends patient disposition and treatment. Physicians may also directly launch the management portion of ePNa without an alert, utilized when calculated pneumonia probability is < 40%.

ePNa objectively calculates severity of illness for each patient with pneumonia.

1. Estimates 30-day mortality risk using an electronic version of a validated severity score (eCURB).¹⁶ The 5 elements are patient age, initial systolic blood pressure, initial respiratory rate, altered mental status, and blood urea nitrogen level.

2. Calculates the patient’s ability to oxygenate arterial blood compared to the fraction of inspired oxygen (PaO2/FiO2) from measured percutaneous oxygen saturation (SpO2).¹⁷

3. Gathers electronically the 9 minor criteria for severe pneumonia (IDSA/ATS 2007), summed into an ordinal score.¹⁸

4. Identifies patients with an associated pleural effusion by NLP of the chest imaging report

ePNa then recommends (1) patient disposition, (2) antibiotic therapy, and (3) suggested microbiology tests appropriate to the patient’s condition. It displays the data in a form suitable for physician review. The physician can alter tool recommendations based on individualized clinical judgment.

Site of care - outpatient, hospital ward, or intensive care unit

Patients with eCURB-predicted mortality ≥ 5%, PaO2/FiO2 < 280 mm Hg adjusted for altitude, or significant pleural effusion are recommended for hospital admission in accordance with previously validated criteria.¹⁹ Patients with < 3 severe Community Acquired Pneumonia (CAP) minor criteria, or PaO2/FiO2 < 120 mm Hg are recommended for direct admission to the intensive care unit. (figure 2) Severity information is displayed to the ED physician with the site of care recommendation. The open loop design of ePNa allows the physician to accept or reject this recommendation; if rejected, the physician is asked to provide a reason from a pre-populated pick list or to add free text. This functionality allows real-time learning from the use of the tool.²⁰

Figure 2:

Site of care recommendation.

Identification of patients with risk factors for antibiotic resistant bacterial pathogens

Drug Resistance In Pneumonia (DRIP) is a 10 factor ordinal score that measures risk for antibiotic resistant bacterial pathogens developed at Intermountain and externally validated.²¹ In 2014, ePNa was updated to electronically calculate DRIP and incorporate its results into antibiotic treatment recommendations. This operation involves the following:

1. Searches the longitudinal EHR for hospitalization ≥48 hours within the previous 60 days and matches facility name and addresses for skilled nursing homes, rehabilitation facilities, and long-term acute care facilities. It also searches International Classification of Diseases codes version 10 (ICD-10) indicative of chronic obstructive pulmonary disease, diabetes, chronic heart failure, and active cancer.

2. Extracts previous microbiology results for Methicillin-resistant Staphylococcus aureus and Pseudomonas aeruginosa.

3. Searches electronic prescribing records for antibiotic use within the prior three months or use of gastric acid reducing medications.

4. Applies NLP to ED chest imaging reports to identify presence of a feeding tube, a measure of functional status.

5. Asks the ED physician to confirm that the patient is ambulatory (versus chair/bed bound).

Figure 3 shows the user interface for determining the DRIP score. Physicians click on the right-sided box to add or delete risk factors if there is information not present in the electronic health record (e.g. a visitor to Utah without prior electronic health data) or if the information identified by ePNa is not correct. Antibiotic selection is based on site of care, and whether the DRIP score is >3.

Figure 3:

The user interface for determining the DRIP score.

Diagnostic testing recommended by ePNa

Patients admitted to the Intensive Care Unit are recommended for two blood cultures, urine antigen tests for Streptococcus pneumoniae and Legionella pneumoniae, nasal swab for Methicillin resistant Staphylococcus aureus and viral respiratory pathogens by Polymerase Chain Reaction, and a tracheal aspirate for semi-quantitative culture if newly intubated. The same battery of studies is recommended for patients with a DRIP score >3 being admitted to a hospital ward.

Adaptation of ePNa for iCentra

Beginning in 2016, Intermountain changed over from its legacy EHR to iCentra. Beginning in January 2017, ePNa was reprogrammed and adapted to operate within iCentra. Beta testing began in the 4 Salt Lake Valley EDs in July 2017. After implementation in a 5^th ED in November 2017 identified additional issues with functionality requiring reprogramming, a successful re-launch began in January 2018 across the remaining Intermountain Hospitals.

The iCentra version of ePNa retained a “P” alert on the ED electronic tracking board (LaunchPoint), but the one-click launch functionality of the legacy version could not be replicated. Three “clicks” from the physician user are needed to launch ePNa in the iCentra version, but then ePNa functions easily once initiated. The iCentra version of ePNa features a direct link to Computerized Physician Order Entry that was not available in the legacy EHR. As the last step of ePNa, the clinician launches one of 6 pneumonia order sets (2 outpatient, 2 hospital ward, 2 intensive care unit, each with a low risk/high risk for antibiotic resistant pathogens version). ePNa iCentra also contains drug allergy information linked to the order entry set, functionality not available in the legacy EHR.

Automatic loading of ePNa output into clinician ED and admission notes. ePNa’s calculated percent likelihood of pneumonia, the severity calculations (eCURB, PaO2/FiO2, and severe CAP criteria), and the DRIP score are automatically loaded into clinician documentation under a sectional heading of “Clinical Decision Support.” In addition, ePNa’s recommendation for site of care, whether the ED physician agreed with its recommendations, plus any additional ED physician comments, are automatically loaded in ED and admission notes.

Implementation of ePNa across Intermountain

Intermountain operates hospitals in small, rural locations (often staffed by family practice physicians) as well as large, tertiary care medical centers in urban regions that use a board-certified emergency medicine staffing model. After achieving acceptable tool functionality in January 2018, we began implementing ePNa into 6 geographic clusters of Intermountain Hospital ED’s at approximately 2-month intervals. We began with larger hospitals, with the later implementation sites being smaller rural hospitals.

Methods of CDS implementation

1. Front line provider education: ED physicians, Advanced Practice Clinicians, ED nurses, admitting hospital and critical care physicians first received didactic, customized and interactive presentations prior to ePNa launch. Emergency departments were the focus for implementation since if ePNa isn’t run in the ED, it is less likely to be started after hospital admission. Emergency department ‘buy-in’ is key and promotes continuity of care between providers throughout the hospital stay.

2. Nurse educator engagement: Multiple follow up visits by the ED nurse educator (JR) began after ePNa launch at each site.

3. Physician champions: We identified physician champions in every ED to teach and encourage their peers to adopt ePNa into routine care of pneumonia patients. The largest ED physician group identified thirteen members as super users available to answer questions and promote utilization while working in the ED.

4. Feedback to improve the CDS tool: Physician champions provided a voice for feedback and insights to functionality of their location. Individual providers also gave feedback to the nurse educator during her visits. Follow up group meetings were conducted with physicians from the ePNa team, answering questions and concerns from providers.

5. Email reminders: Reminders and answers to frequently asked questions were sent to providers.

6. Audit and feedback: Repeated audits of ePNa use were fed back to providers along with issues involving pneumonia patient care where ePNa had not been used. Audits that included the physician provider and patient encounter identifiers were made available to all providers at each ED, engendering discussion and provider accountability. We observed the largest increases in ePNa utilization after audit results were openly shared by local champions/ED leadership, leading to discussion among providers.

7. Admitting hospital physicians “encourage” the routine use of ePNa by ED physicians and increase patient safety by facilitating transitions of care from the ED to the hospital. ED physicians frequently comment that ePNa structures and supports their conversations with admitting physicians.

Rate of ePNa uptake

Despite published favorable outcome data, ED physicians were initially slow to adopt the iCentra ePNa, judging its value by the user interface and subjective assessment of its impact on their individual clinical care. This adoption pattern was more pronounced outside of Salt Lake Valley (where clinicians had used the legacy version of ePNa). Audits were conducted over time to measure uptake and use of the tool. These audits showed variations from best practice in about half of pneumonia patients when ePNa was not used, or when the associated computerized order set was not also used. Six audits were performed during 2018 (Figure 4). Hospital clusters where ePNa was launched earlier in 2018 have more measurements than later clusters. Each audit spanned 2 to 4 weeks (longer at smaller hospitals to increase the number of patients), ranging between 4 and 30 cases per facility depending on the interval and facility ED case volume. Patients were initially identified by ICD-10 codes for pneumonia, or for sepsis and acute respiratory failure with pneumonia present as a secondary diagnosis. Physicians (ND, CV, NJ, MW) individually reviewed each case to verify ED physician diagnosis of pneumonia, excluding those where the physician diagnosis was not pneumonia, e.g. aspiration pneumonitis. Figure 4 illustrates the percent utilization of ePNa by hospital over the 6 audits.

Figure 4:

Utilization rates by date in 14 Intermountain Hospitals during 2018.

Results of emergency department physician survey.

A REDCAP confidential survey of ED physicians was conducted in December 2018. Survey details have been previously published.²⁰ The response rate of completed surveys was 53%, or 90 of 169 invited physicians. We used Principal Component Analysis followed by Reliability testing to group like questions together. Figure 5 illustrates respondents’ opinions about how well ePNa helps with clinical decision-making. The mean score on component 1 is 3.6 (SD=0.79). Response options ranged from strongly disagree (1) to strongly agree (5) with a neutral rating corresponding to (3), a mean of 3.6 indicates that the majority of respondents hold favorable opinions about how well ePNa helps their clinical decision-making. However, responses regarding decision support with diagnosing pneumonia Revealed that most respondents felt ePNa does not help. Respondents’ opinions of the “P” alert system (component 2) resulted in a mean score of 3.1 (SD=1.3). A mean of 3.1 is approximately in the center of the scale. However, scores are spread out across the response scale, indicating that respondents have diverse opinions on how well the alert system works.

Figure 5:

ePNa helps with clinical decision making.

The median response to questions about how much time ePNa requires is 3.0 (IQR=2) (Figure 6). This median is in the center of the scale, suggesting that respondents are split over how much time it takes to use ePNa.

Figure 6:

It takes too much time to use ePNa.

The median response for “ePNa is free from technical problems” is 2.0 (IQR=1.0), Figure 7. A median of 2.0 is on the disagree side of the scale, suggesting that most respondents feel that ePNa does have technical problems.

Figure 7:

ePNa is free from technical problems.

With the mean responses to components 1 and 2 being below 4.0 (agree), progress can be made in improving respondents’ perceptions toward ePNa. The comments on improving ePNa support this conclusion. Removing bugs and glitches and adding new functionality will likely enhance perceptions of ePNa.

There were a few strong, negative comments about ePNa. However, these comments were balanced by an equal number of strong, positive comments. People with strong, negative opinions may never be persuaded that ePNa is effective and usable no matter how well it works, or that CDS is needed. However, in this survey the majority of respondents believe that ePNa is slightly above average in quality, and those opinions may improve with enhancements to the interface and functionality of ePNa. Efforts on improving perception and utilization should focus on clinicians who are amenable to adopting CDS into their clinical care.

Two survey comments from clinicians represent several similar comments and illustrate current limitations of ePNa. “The tool often gives recommendations that are inappropriate and potentially dangerous. I’ve had it recommend outpatient therapy for patients who need the ICU and vice versa.” ePNa requires accurate, real time data to calculate severity of illness which drive disposition recommendations. The nurse educator has emphasized to ED nurses the importance of charting SpO2 on room air (for calculating PaO2/FiO2), and confusion (oriented to person, place, and time or not; key components of both eCURB and the severe CAP criteria). For example, a patient brought in by ambulance may have had a room air SpO2 of 60% measured by prehospital medics and placed on high flow oxygen by mask, resulting in a saturation of 99% on arrival to the ED. Only the latter might be available to ePNa, but PaO2/FiO2 is not accurately calculated from high SpO2.

Uncharted data are considered normal by all severity systems, yet if confusion is present and not charted, ePNa might give a “dangerous” recommendation. ePNa requires timely, accurate entry of patient data, yet the frequently busy and occasionally chaotic ED environment can make accurate CDS function difficult. ePNa is open loop CDS which cannot anticipate every clinical scenario, thus the treating physician must do what he/she feels is safe and appropriate for the individual pneumonia patient.

Another illustrative comment: “Better alert system. A dedicated care pathway column would make the user interface and inter-operability much improved”. The screening algorithm was not updated for iCentra ePNa, and due to technical limitations, the single click for launching ePNa featured in the legacy version could not be carried over. This presented an early barrier to use and adoption of ePNa. We are now re-deriving the screening tool for iCentra and plan a pop-up dialog box to launch the program with a single click once ePNa identifies a patient with ≥ 40% likelihood of pneumonia.

Summary and Conclusions

ePNa is an innovative electronic CDS tool for ED patients with pneumonia. Published outcome data demonstrated reduced mortality and appropriate hospital admission of high severity patients with pneumonia. A long-term goal of the current study is to demonstrate similar results across the large and small, urban and rural Intermountain Hospitals. However, the huge challenge of broad adoption by ED physicians must occur first.

Adaptation of ePNa into iCentra allowed implementation across all Intermountain hospitals. In this paper, we have described our implementation process, reported ePNa usage during 2018, and results of our ED physician survey. Physician utilization has been higher in larger, urban medical centers with longer exposure to ePNa. Recurring personal and electronic feedback/reminders from the ePNa team is increasing its use across Intermountain. Improvements in the pneumonia detection component and improving the user interface that launches ePNa are underway, as identified by ED physician users and described above. More work to improve charting of critical data elements by ED nurses will increase tool accuracy and use of ePNa. We hope to eventually export ePNa outside of Intermountain to determine its generalizability among different pneumonia populations and providers with different diagnosis and treatment patterns. These efforts will benefit from the lessons learned during the implementation of ePNa at Intermountain.