Development, Implementation and Use of Electronic Surveillance for Ventilator-Associated Events (VAE) in Adults

Abstract

Mechanical ventilation provides an important, life-saving therapy for severely ill patients, but ventilated patients are at an increased risk for complications, poor outcomes, and death during hospitalization.¹ The timely measurement of negative outcomes is important in order to identify potential issues and to minimize the risk to patients. The Centers for Disease Control and Prevention (CDC) created an algorithm for identifying Ventilator-Associated Events (VAE) in adult patients for reporting to the National Healthcare Safety Network (NHSN). Currently, the primarily manual surveillance tools require a significant amount of time from hospital infection prevention (IP) staff to apply and interpret. This paper describes the implementation of an electronic VAE tool using an internal clinical data repository and an internally developed electronic surveillance system that resulted in a reduction of labor efforts involved in identifying VAE at Barnes Jewish Hospital (BJH).

Introduction

The use of mechanical ventilation can be an important, life-saving treatment for severely ill patients. The Centers for Disease Control and Prevention (CDC) estimate as many as 300,000 patients receive mechanical ventilation each year, but also notes that this can lead to additional complications, poor outcomes, and death during hospitalization.¹ Because of the increased risk to patients, it is important that hospitals monitor for adverse events associated with the use of mechanical ventilation.

Prior to 2013, the BJC HealthCare (BJC) Infection Prevention (IP) staff performed surveillance for ventilator-associated pneumonia (VAP). VAP is among the most common hospital-acquired infections, but accurate surveillance for VAP was difficult because of the lack of universally accepted objective definitions. VAP surveillance was time consuming, potentially less accurate than clinical/microbiologic criteria, and the use of quantitative lower respiratory tract cultures for the establishment of VAP is not universally performed.^2–3

The CDC’s National Healthcare Safety Network (NHSN) Working Group developed a new and more objective approach for surveillance that focuses on Ventilator-Associated Events (VAE). These include Ventilator-Associated Conditions (VAC), Infection-related Ventilator-Associated Complications (IVAC), possible VAP and probable VAP.¹ The new methodology was developed with the goals of: limiting the VAP definition’s subjectivity and inaccuracy; using readily available and objective clinical data; making inter-facility comparisons more meaningful; and encouraging broader prevention strategies. It was also designed to allow electronic data collection, given the availability of a hospital-wide electronic health record (EHR) system with exportable data. The new definition still requires daily monitoring of patients, and would be time intensive to complete manually in large institutions. In order to decrease the burden of manual surveillance for VAE, BJC developed an electronic surveillance process for gathering relevant data elements. The approach used a combination of new interfaces and the incorporation of a new electronic algorithm with an existing, intranet-based surveillance system called Surveillance Assistant (SA) that was internally developed and deployed at BJC in 2009.

Hospital Setting

BJC is a large, nonprofit healthcare organization affiliated with Washington University School of Medicine (WUSM) that delivers services to the greater Saint Louis metropolitan region. The 12 BJC hospitals provide adult and pediatric care at locations ranging from rural and community hospitals to large, academic institutions.

Barnes Jewish Hospital (BJH) is a 1250 bed, tertiary care academic facility associated with WUSM. There are six intensive care units (ICUs) at BJH with 121 total beds, including medical, surgical, cardiac, cardiothoracic and neurological specialties. Ventilated patients can also be cared for on a bone marrow transplant unit, which is staffed to have up to 8 ventilated patients at a time, a step-down long term vent unit with 10 beds and an advanced heart failure unit with 6 beds. On average, there are a little over 1,500 ventilator days each month at BJH.

Methods

The first step in developing the electronic algorithm for VAE surveillance was to analyze the NHSN specification. Based on CDC’s NHSN VAE protocol, a VAC represents an episode of sustained respiratory deterioration, caused by both infectious and non-infectious conditions and complications occurring in mechanically-ventilated patients. A VAC is defined by a sustained period of worsening oxygenation that immediately follows a baseline period of stability or improvement on the ventilator. To meet the VAC definition, a mechanically-ventilated patient must have at least two calendar days of stable or decreasing daily minimum positive end-expiratory pressure (PEEP) or fraction of inspired oxygen (FiO2) followed by at least 2 days of increased daily minimum PEEP or FiO2. The increase in the daily minimum PEEP must be ≥ 3cm H2O or an increase in the daily minimum FiO2 of ≥ 0.20 (20 percentage points in oxygen concentration) than the daily minimums during the baseline period.

An IVAC is defined as a VAC in which the patient has either a temperature or white blood cell count outside the expected ranges provided by NHSN and a new, eligible antimicrobial agent started and continued for four or more calendar days. Both of these triggers have to happen within the window period defined as on or after the third day of mechanical ventilation and within 2 days before or after the onset of worsening oxygenation.

Possible VAP is defined by the presence of purulent secretions or a positive lower respiratory tract culture. Probable VAP is defined by the presence of purulent secretions in addition to a positive lower respiratory tract culture meeting certain quantitative or semi-quantitative thresholds of pathogen growth (endotracheal aspirate [ETA], >105 CFU/ml; BAL, >104 CFU/ml; tissue, >104 CFU/g; protected specimen brush [PSB], >103 CFU/ml). The probable VAP definition could also be met based upon the presence of a positive pleural fluid culture, lung tissue with histopathological evidence of infection, or positive diagnostic tests for Legionella or selected respiratory tract viruses, without the concomitant requirement for purulent secretions.

This implementation project was focused on getting the needed data for BJH due to the surveillance burden at that facility. Analysis of VAE rates over the study period were evaluated using linear regression (SPSS V 21.0, IBM SPSS Inc, Armonk, NY).

Development and Implementation Processes

BJC previously developed an enterprise clinical decision support system and repository (CDS) for surveillance and real-time alerting.⁴ The Pharmacy Expert Systems database (PES) has been in use since 1994 and receives registration, lab, vital sign, pharmacy, microbiology, and select nursing assessment data through a combination of some real-time and nightly batch interfaces. This data allows for the generation of batched alerts, some real-time alerts, and both prospective and retrospective surveillance monitoring for a wide range of hospital initiatives.

Prior to starting the VAE project, the implementation team did a gap analysis between the current database and the requirements of the NHSN algorithm. The analysis showed that the current CDS contained all of the necessary data for calculating the VAC and IVAC candidates, with the exception of some of the ventilator settings. In late 2012, BJC initiated a new data acquisition project to capture the ventilator settings data documented within the hospital EHR.⁵

In an effort to bring the ventilator settings data into CDS, we expanded a nightly data extract from the source clinical documentation system at BJH to include of all data elements needed for accurate determination of VAC and IVAC cases according to the NHSN specifications. Using the ventilator setting data along with existing data available in CDS (including antibiotics, vital signs, and microbiology results), we were able to implement electronic surveillance for the respiratory status (VAC) and the infection and inflammation components (IVAC) of VAE.

Due to data limitations, the electronic algorithm can only identify patients as having a VAC or an IVAC. For example, while microbiology data is available within the clinical data repository, the specific quantitative results required for the purulent respiratory secretions are not reliably available in a discrete format. It would be possible to detect a positive endotracheal aspirate culture, but it would not be possible to apply the requirement of ≥10⁵ CFU/ml with the existing interface. As a result, the final determination for possible or probable VAP requires clinical review of IVAC patients by IP staff.

The manual and electronic surveillance were performed in parallel for 6 intensive care units, 1 step-down long term ventilation unit, 1 oncology unit and 1 advanced heart failure unit at BJH for 8 months, between January and August 2013. The performance of the electronic algorithm for VAC and IVAC surveillance was manually verified by IP staff. In an automated fashion, the electronic algorithm gathered and summarized daily data for every ventilated patient.

During the assessment period, an Excel report with patient identifiers, mechanical ventilator (MV) data, PEEP (minimum), FiO2 (minimum), temperature (minimum and maximum), WBC (minimum and maximum), and antibiotics was provided to IP for review on an ad-hoc basis. IP staff was blinded to the electronic algorithm’s decision of VAC or IVAC status. Every patient was reviewed independently by the two sources for 8 months. The electronic algorithm went through several iterations of refinements to ensure the NHSN definitions were correctly applied. After the initial 8 months of validation were completed, the electronic algorithm was incorporated into the daily work of SA for BJH.

Results

In 2013, there were 3,691 ventilated patient visits, accounting for 19,105 ventilator days at BJH. Of the 3,691 patient visits, 121 were identified with 1 or more VAE events: 62 were VAC (3.2/1,000 ventilator days), 35 were IVAC (1.8/1,000 ventilator days), 14 were possible VAP (0.7/1,000 ventilator days) and 10 were probable VAP (0.5/1,000 ventilator days). VAE rates showed no increasing or decreasing trends over the study period (p = 0.32, Figure 1).

Figure 1.

The electronic algorithm required about 600 hours from an informatics intern to develop, and about 80 hours from a senior informatics analyst to complete. The senior informatics analyst was primarily responsible for a final code review and the logic to incorporate the data into the existing electronic surveillance application. About 175 hours of IP staff time was spent working with developers and validating results. The initial develop time could have been reduced by using a more experienced resource, but the overall cost would have been equivalent given the higher resource cost of a senior informatics analyst compared to an intern.

In the 8 months of duplicative review, many tweaks were made to the electronic algorithm to account for intricacies of the process. In the end, the electronic algorithm was able to correctly identify all cases of VAC and IVAC allowing IP staff to then identify possible or probable VAP. After the initial period of reviewing the VAE candidates using a daily report, work was done to incorporate the results of the query into SA that provides the IP staff with a line list that they can review on daily basis (Figure 2).

Figure 2.

Initially all new events are loaded into the SA application as either VAC or IVAC. VAC cases are not displayed on the line list since they are not candidates for further classification. IVAC cases must be opened to a detail screen where a workflow module will help the user assess the IVAC case against the possible and probable VAP definitions (Figure 3). The detail page provides the IP staff with as much data as is available in the clinical repository, including microbiology data, room assignments, antibiotics, etc. It also determines the window period, which can change depending on the dates of mechanical ventilation and symptom development.

Figure 3.

If the patient meets possible or probable VAP criteria, IP makes the appropriate selections and fills in details marking the case as possible or probable VAP. New VAC and IVAC cases are posted once a day on the SA web interface and IP staff can pull all previously identified cases as well as the new cases. At the time of identification, case information is given to the unit staff where the patient was located. On a monthly basis, rates of each type of VAE are fed back to all areas in a standardized format. All areas are monitored for trends in VAE.

The automated line list report reduced the IP labor efforts for surveillance. The former VAP definition surveillance took approximately 10 hours per ICU per month, equating to about 60 IP hours per month to complete ICU surveillance at BJH. The remaining 3 non-ICU units with ventilated patients generally have low census and surveillance time committed to those unit was not considered in this calculation. It is estimated that completely manual surveillance of the new VAE definition would require the same amount of time (60 hours/month). By manually using the Excel report of components of the VAE definition, that time was reduced to about 2 hours per ICU per month (12 hours/months). The complete automation of the VAE definitions in SA decreased IP time to about 30 minutes per unit per month (3 hours/month), an overall time savings of 57 hours/month.

Discussion

While the electronic VAE surveillance module was successfully deployed at BJH in November 2013, the overall effort represented a significant investment of resources from both the clinical and technical sides of the organization.

The greatest implementation cost was the development and testing of the electronic algorithm. This was in large part due to the complexity of the specifications and the lack of prior experience with the definition. At the time the electronic algorithm was being developed, there was no easy way to validate results outside of a manual review. The development process was very iterative, and questions had to be addressed either by reviewing the specifications or working with our NHSN technical contacts. While the initial investment spent on developing the electronic VAE algorithm and building an infrastructure to accommodate data collection from different hospital systems was significant, BJC is planning to leverage the investment and increase cost savings by implementing the algorithm enterprise wide and rolling it out to the other 9 adult hospitals.

It is important to note that NHSN now offers a web-based interface that takes relevant patient level data (mechanical ventilation dates, FiO2, PEEP values, etc.) and applies the VAC or IVAC criteria. This VAE Calculator provides a quick way to validate the algorithm logic and test out different rule scenarios against a standard. Having this available a year ago would have dramatically reduced the development time, and it has already provided an easier way to perform regression testing when bugs or issues have been uncovered. Furthermore, NHSN has announced that a new web service will be developed that receives de-identified patient data via an XML message. The web service returns an XML result set with the VAC/IVAC classification. While this does replace some of the functionality we have developed, the advantage to hospitals and vendors with the capacity to interact with the web service is clear in terms of development time and future maintenance as definitions change. At the time this paper was written, a prototype of the NHSN’s web service was available, but no clear timeline was provided for a production version. Hospitals and vendors still have to do the work to build the required XML message and there has to be some infrastructure for storing the data that is returned. In the case of BJC HealthCare, the results of the web service would be used to populate the same infection tables that populate the detail screens from Figures 2 and 3.

Another significant challenge with the implementation involved getting access to the necessary ventilator data. The pilot hospital involved in the algorithm development already had a daily interface of ventilator data in place prior to the start of the project. This reduced the effort required to capture the additional fields required for the algorithm. In addition, the hospital had a clear champion who worked with the application team to modify the extract with the additional data fields. The interfaces for the other system hospitals require an entirely new interface to be developed. The project team had to meet with subject matter experts at the hospitals to make sure the necessary fields were included in the extract request, and Meaningful Use and other regulatory initiatives have limited access to key technical resources.

The remaining project resource investment was associated with creating the new detail web page in the SA application. The developers worked closely with the IP staff to design the screens with the goal of making it easy to apply the correct definitions. The workflow was considered, and dynamic content was added to reduce the chance of data entry errors. For example, an IVAC cannot be marked as complete unless the specific checkbox is selected that indicates no evidence of a possible or probable VAP.

There have been some concerns raised by those involved in developing and testing these new definitions. One is that healthcare workers may be able to learn to “game the system” by making small, short changes in ventilator settings daily to skew the true picture of a patient’s progress. As the team has access to all ventilator setting changes for the patient population, this can be analyzed. One next step for the group is to trend patient-specific ventilator settings, looking for daily outliers.

This development project was strengthened by the fact that the validation included many patients from various medical backgrounds, including several types of intensive care and other critical care units and the oncology population. Due to this fact, and the extensive 8 month period of manual validation of the electronic algorithm, the group hypothesizes that the electronic algorithm is robustly built to correctly identify VAE in a variety of patient populations. However, as this study was completed in a large academic institution, it may not be generalizable to all populations. The application will be applied to the community hospitals that are members of BJC Healthcare, with additional manual validation of the electronic algorithm to ensure generalizability.

To date, the numbers of VAC, IVAC, possible VAP, and probable VAP have been small, and within anticipated ranges. No interventions have been specifically developed in response to this data, however there are ongoing efforts in these units to decrease adverse events associated with ventilator use. As we continue to perform surveillance and gather data, we will be able to better analyze and interpret them for use in intervention development.

Conclusion

BJC HealthCare was able to successfully implement an electronic VAE surveillance algorithm based on the CDC VAE definition and incorporate it into an existing electronic surveillance system at BJH. This saves an estimated 57 hours of IP staff work per month at just one hospital by providing decision support for identification of VAC and IVAC cases and reducing the burden of data collection. The VAE module also helps provide the additional data required to classify an IVAC case as possible or probable VAP cases. The data is stored in a local repository where it can be queried or eventually uploaded to NHSN.

The costs involved in the development and implementation must be carefully weighed against the savings obtained. Even with a robust data infrastructure, there was significant development cost in getting access to the necessary data, creating the electronic algorithm, and modifying the existing web application. In the case of a large multi-hospital system the cost can be easier to justify due to the volume of patients receiving mechanical ventilation and the ability to leverage the initial investment by implementing the electronic algorithm enterprise wide as BJC is planning on doing.

For a smaller hospital without a strong IT and data infrastructure, the costs might well outweigh the benefit. This is especially true when taking into account the new tools that NHSN is providing for hospitals and vendors to use in the classification of the VAC and IVAC cases.