Automated Data Acquisition and Scoring for JCAHO ICU Core Measures

Abstract

The Joint Commission for the Accreditation of Healthcare Organizations (JCAHO) has recently established six Core Measures as indicators of quality in the delivery of care to adult Intensive Care Unit (ICU) patients. These Core Measures are demanding in terms of the amount of physiologic, medication, treatment and observational data required for measurement. At Cedars-Sinai Medical Center (CSMC), a Clinical Information System (CIS) gathers data for all ICU patients. For 72 beds in eight adult ICUs, the CIS was configured to ensure automated and easy charting of Core Measure observations. Severity of illness is automatically calculated from physiologic and admission data. All data is automatically transferred to a relational database daily. A Core Measure scoring program was devised to produce weekly and monthly run charts of JCAHO ICU Core Measure performance for each ICU. Internal publication of results and refinement of care processes led to progressive improvement in compliance with Core Measure treatments.

Introduction

The ICU Core Measure data set has been defined as “A unique grouping of performance indicators carefully selected to provide, when viewed together, a robust picture of care provided in intensive care” [1]. Core Measures are evidence-based and can be considered surrogate markers for a wide variety of quality-related activities in clinical care. JCAHO introduced draft ICU Core Measures for public comment in 2003 and recently published the first set of six final ICU Core Measures [2]: These include:

• ICU-1 Ventilator-Associated Pneumonia (VAP) Prevention – Patient Positioning

• ICU-2 Stress Ulcer Disease (SUD) Prophylaxis

• ICU-3 Deep Venous Thrombosis (DVT) Prophylaxis

• ICU-4 Central Line Associated Blood Stream Infection

• ICU-5 ICU Length of Stay (LOS) (Risk Adjusted)

• ICU-6 Hospital Mortality for ICU Patients (Risk Adjusted)

ICU-1, -2 and -3 apply only to patients on a ventilator. Because the draft ICU Core Measures were in flux for a prolonged period of time, the CSMC medical staff, Critical Care Committee and Quality Improvement department decided to devise automated measurements and scoring for the first four Core Measures while awaiting JCAHO’s direction on risk adjustment for the final two measures.

Methods

All CSMC ICUs utilize the CareVue Clinical Information system (Philips Medical Systems, Andover, MA). The CIS is networked to bedside devices including physiologic monitors, ventilators, urimeters, intravenous pumps, glucometers and, when utilized, intra-aortic balloon pumps. It is also networked to clinical laboratories, the hospital registration system and transcription system. All physiologic, medication, laboratory and observational data are stored in the CIS, which serves as a paperless electronic medical record [3]. The multidisciplinary plan of care, nursing notes and physician procedure notes are stored in the system, along with the physician admission history and physical and daily progress notes in many units. The CIS includes an electronic Medication Administration Record (MAR) that provides the time of administration for all medications and intravenous fluids, including additives.

Three measures of severity of illness are automatically calculated for each adult ICU patient on admission and again daily. Physiologic, demographic, laboratory, chronic history and other data are extracted and scored according to the guidelines for the Simplified Acute Physiology Score (SAPS), Acute Physiologic and Chronic Health Evaluation Score (APACHE II) and the Quantitative Therapeutic Intervention Scoring System (QTISS) [4–6]. Scoring is performed on a separate server and results are transferred back to CIS flowsheets over a HL7 data link.

When nurse charting was required to document a component of a Core Measure, the ICU nurse managers decided the format of data entry. A special Core Measures flowsheet was configured to provide care-givers with a quick way to monitor compliance with Core Measures ICU-1, ICU-2 and ICU-3 (Figure 1).

Figure 1.

CIS Core Measure Review Flowsheet

Process measures to reduce the incidence of central line associated infections (ICU-4) are charted in the CIS by physicians as part of a structured text central line insertion note. Drop-down choices for wide sterile draping, use of ultrasound for vein location and use of an antibiotic-coated catheter are provided as part of routine documentation (Figure 2).

Figure 2.

CIS Physician Central Line Insertion Procedure Note

Although CIS data is held in a relational database (HP Allbase, Hewlett-Packard Co., Palo Alto, CA), it is stored in a format that is not conducive to multi-patient analysis. For such analyses, all CIS data is transferred daily to a Clinical Data Archive (CareVue CDA, Philips Medical Systems) which uses a different relational database (Oracle 8, Oracle Inc., Redwood City, CA). Granular CDA data is transformed into patient length of stay records in a separate Oracle database (CareVue ISM, Philips Medical Systems). Core Measure scoring is performed on data in the CDA and ISM.

Most data required to score ICU Core Measures were already in the CIS when JCAHO announced the program. However, it proved challenging to develop the scoring algorithms and to agree on valid exceptions to Core Measure treatments, especially in the absence of specific JCAHO guidelines. Certain exceptions were derived from common sense, such as not providing additional anticoagulation for DVT prophylaxis to patients with an underlying coagulopathy. However, for the scoring program to automatically provide an exception for such patients, “coagulopathy” had to be objectively defined in terms of laboratory measurements that could be taken into account. Ultimately the Critical Care Committee provided a venue for expert guidance to reach agreement on appropriate exception criteria. Although the Core Measures are simple to state, the complete algorithm to score them and recognize appropriate exceptions is quite complex. The algorithm to score Stress Ulcer Disease Prophylaxis is shown in Figure 3.

Figure 3.

Stress Ulcer Disease Prophylaxis (SUD) ICU-2 Scoring Algorithm

SQL queries were constructed to score each Core Measure on each ICU patient daily. The ISM was used whenever possible and detailed data was extracted from the CDA when required. Each patient is scored every day (100% sampling) and results are tabulated for each ICU on a weekly and monthly basis. Results are provided promptly to physician directors and nurse managers of each ICU as well as the hospital’s Quality Improvement department.

Results

Approximately six months of experience collecting Core Measure data, scoring results and auditing of electronic charts was required to finalize the scoring algorithms. ICU physician directors and nurse managers carefully audited every patient for to validate the scoring algorithm. If an anomaly was identified, the algorithm was corrected and all prior data was re-analyzed to obtain corrected monthly scores. Eventually, the automated results agreed with manual audits. Little guidance on methods for scoring and exclusion of inappropriate patients was available in the literature or from JCAHO. Issues were taken to the Critical Care Committee for adjudication as they arose.

Early results showed that changes in certain ICU practices were required to ensure compliance. For example, the scoring program counted medications toward compliance with SUD and DVT prophylaxis only when they were charted as given, not when they were ordered. For new admissions, this required ICU nurses and pharmacists to ensure that these otherwise non-urgent medications were given promptly, especially for patients admitted to the ICU on ventilators in the late afternoon or early evening. In addition, medications ordered for once a day administration are usually given at the standard time of 9 am the following day. However, when this was done for once a day SUD prophylaxis medications, patients were counted as non-compliant on the day of admission. These meds now had to be given “stat” to avoid a non-compliant score.

Prior to initiation of Core Measure scoring, most ICU directors and nurse managers believed Core Measure treatments were routinely provided in their units. However, the initial results showed that not to be the case. A significant team effort involving ICU faculty, residents, fellows, nurses and pharmacists was required to ensure excellent performance. Compliance with the “Ventilator Bundle” of all three ventilator Core Measures achieved daily improved over time and is currently at or near 100% (Figure 4).

Figure 4.

Ventilator Bundle Compliance (VAP Prevention, DVT and SUD prophylaxis daily)

Discussion and Summary

The use of an ICU CIS to measure JCAHO Core Measure compliance is novel. Extraction of quality-related data is not always considered when CIS functionality is discussed [7]. However, in the pre-Core Measures era, we have reported on the extraction of quality, severity and outcome related data as a useful byproduct of a comprehensive CIS [8-9]. It is natural that such systems be used to gather raw data and generate compliance scores for evidence based quality indicators.

JCAHO’s ICU Core Measures endured a long period of evaluation and comment before becoming final. Significant changes and refinements were made during the evaluation process and future changes are all but inevitable as ICUs gain experience with them. The present study fully addresses measurement of the first three Core Measures, for prevention of VAP, DVT and SUD in ventilator patients. In addition, it measures compliance with three elements of the central line insertion process known to provide evidence based reductions in blood stream infections, i.e., wide sterile draping, use of ultrasound for vein location and use of an antibiotic-coated central catheter. However, JCAHO Core Measure ICU-4 requires that the line-associated blood stream infection (BSI) rate be measured continuously, without statistical sampling. This information could potentially validate whether improved central line process measures lead to a lower rate of BSI.

JCAHO Core Measures ICU-5 and ICU-6 provide severity adjusted measurements of ICU length of stay and hospital mortality for ICU patients, respectively. The severity adjustment method chosen is APACHE IV, which is a further refinement of the APACHE III score with additional subcategories for cardiac surgical patients. Detailed information about the APACHE IV scoring system is not known at this point, so its calculation remains a future endeavor. However, basic severity scoring methods are well known and our group and others have published multiple studies of ICU and hospital outcomes based on prior severity scores including SAPS, APACHE II and APACHE III [10–14].

Although the therapies reflected in Core Measures ICU-1, -2 and -3 are evidence based, it is not known whether the zeal to perform well on these measures will translate into better patient outcomes. In particular, it is unknown whether “stat” administration of SUD prophylaxis actually prevents the disease; however it interrupts normal workflow and costs valuable nursing and pharmacist time. The necessity for this practice should be considered an unintended consequence of Core Measure compliance. There may be others and their impact may not necessarily be benign.

Core Measures ICU-4, -5 and -6 measure different types of outcomes but not underlying ICU therapies or practices. Especially when outcomes are adjusted for severity of illness, comparisons between institutions are fraught with difficulty and complexity [10, 15]. Nonetheless, we have shown that an ICU CIS can automatically gather most of the data required for quality and outcome measurements and make it available for analysis and reporting. Reporting the JCAHO ICU Core Measures was associated with a progressive improvement in performance at our institution.