Assessing Compliance with Established Pneumonia Core Measures at a Comprehensive Cancer Center

Abstract

Introduction

Healthcare organizations use Pneumonia Core Measures (PCM) to ensure delivery of high-quality care. A multidisciplinary team was organized to optimize care and enhance compliance in a comprehensive cancer emergency center.

Methods

We performed a 4-phase study, three of which were interventional: intense education regarding PCM; microbiologic analysis of the pathogens responsible for the pneumonias; development and implementation of an institutional pneumonia algorithm and order-set. In phase-4, we analyzed five PCMs.

Results

The percentage of pneumonia patients from whom blood cultures were obtained increased from 73% to 91% after intervention (P<0.001); sputum cultures increased from baseline 24.6% to 51% (P=0.004) post order set implementation, and order set utilization increased from 40% to 77%. We achieved the benchmark for only one PCM, PN 3a. More than 80% of patients met clinical and microbiological criteria for healthcare-associated pneumonia.

Conclusions

We identified a gap between our patient population and some PCMs which relates to antibiotics selection. The treatment of cancer patients and pneumonia falls outside established guidelines for treating community-acquired pneumonia. Although the algorithm and order set implemented optimized care and minimized variation, national benchmarks for four of the PCMs were not met. Our findings provide information for policymakers considering pneumonia measurements for antibiotic selection in a cancer care setting.

Introduction

Pneumonia is a major cause of death due to infectious diseases in the United States and the overall leading infectious cause of death among cancer patients (Murphy, Xu, & Kochanek, 2012). Clinical pathways with standardized order sets are frequently used to ensure the delivery of high-quality care to patients with pneumonia (Fleming, Ogola, & Ballard, 2009).

To measure the performance of healthcare organizations in delivering high-quality care to patients with pneumonia, The Joint Commission developed Pneumonia Core Measures (PCM), which are reportable evidence-based standards of practice. The Joint Commission is working with the Centers for Medicare & Medicaid Services (CMS) to align these measures to enable hospital-specific performance rates to be compared with state and national rates. The measurements themselves can be used in other areas of accreditation, expanded to public reporting and pay-for-performance initiatives (Bielanski, 2009).

The PCMs are based on the National Hospital Quality Measures, which integrates standardized common measures from The Joint Commission and CMS. The following are the PCMs for October 2011 through March 2012 (Q4 2011 and Q1 2012):

• PN 3a: Blood cultures performed within 24 hours prior to or 24 hours after hospital arrival for patients who were transferred or admitted to the intensive care unit (ICU) within 24 hours of hospital arrival

• PN 3b: Blood cultures performed in the emergency department prior to initial antibiotic received in hospital

• PN 6: Initial antibiotic selection for community-acquired pneumonia (CAP) in immunocompetent-ICU and NON-ICU patients

• PN 6a: Initial antibiotic selection for CAP in immunocompetent ICU patient

• PN 6b: Initial antibiotic selection for CAP in immunocompetent non-ICU patient.

In October 2012, CMS began rewarding hospitals on the quality of care provided through the new Hospital Value-Based Purchasing Program (Centers for Medicare & Medicaid Services (CMS), 2011).

The Hospital Value-Based Purchasing Program, established by the Affordable Care Act (“Patient Protection and Affordable Care Act,” 2010), will implement a pay-for-performance approach to the payment system that accounts for the largest share of Medicare spending. While specialty hospitals are currently exempt from this initiative, they will begin the process of public reporting for the first time in 2013. To prepare for future reporting we have elected to begin reporting the PCM.

In 2006, a multidisciplinary Pneumonia Team was organized at The University of Texas MD Anderson Cancer Center to evaluate current practices, optimize care, and enhance compliance with PCM. These efforts were initiated in the MD Anderson Emergency Center (EC). Three major areas of clinical gaps and improvement were identified. (1) The EC staff’s lack of experience and compliance with pneumonia quality metrics. (2) Lack of standardized pneumonia order sets for pathogen-directed treatment. (3) Variation in the type of pneumonia patients presenting to the cancer center. Most of our patients have healthcare-associated pneumonia (HCAP), vs. CAP, requiring specific antibiotics which may or may not be in alignment with CMS guidelines. These areas were tackled in three different phases by implementing quality tools; the quality model for improvement used was the Plan, Do, Study, Act (PDSA) cycle, and phase-4 was conducted to evaluate our performance and compare it to national benchmarks. We present our experience here.

Methods

We performed a 4-phase study of PCM compliance in treating cancer patients who presented to MD Anderson’s EC with pneumonia in one of 19 months between May 2006 and June 2012: MD Anderson’s Institutional Review Board approved the study. A waiver of patients’ written consent was obtained.

MD Anderson’s EC is a 43-bed unit that has about 20,000 patient visits per year. A multidisciplinary team was organized into a Pneumonia Team. The team includes physicians from MD Anderson’s Infectious Diseases, Pulmonary and Emergency Medicine departments who were recognized in their respective departments as experts in the topic of pneumonia; EC clinical pharmacists, nurses and respiratory therapists. This team was charged with evaluating, analyzing and disseminating data; developing, implementing and monitoring the pneumonia algorithm and order set; and educating all clinical staff throughout the institution.

Pertinent data, including microbiology from blood and sputum cultures, type of malignancy, and timing of antibiotic were abstracted from the patients’ electronic medical record (EMR) and entered into a database. The present study included patients 18 years and older who were identified with a diagnosis of pneumonia based on the International Classification of Diseases Ninth Revision (ICD-9). A record review of those cases was done confirming the documentation of symptoms of an acute lower respiratory tract illness, and new infiltrate as detected using chest radiography or computed tomography at the time of admission to the EC. The study also included patients with acute respiratory measure and septicemia with a secondary diagnosis of pneumonia. Patients who were younger than 18 years old, pregnant, had aspiration pneumonia, and/or had been diagnosed with pneumonia within 7 days prior to EC visit were excluded from the study.

Eligible patients were further classified as having either CAP or HCAP. HCAP in our patients is defined as a patient who, in addition to the pneumonia inclusion criteria, met any of the following criteria: hospitalization for two or more days within 90 days, residence in a nursing home or extended care facility, home infusion therapy (including antibiotic infusion therapy), chronic dialysis within 30 days, home wound care, a family member with a multidrug-resistant pathogen (L. A. Mandell et al., 2007) or any cancer therapy in the four weeks prior to their presentation to the EC. Patients who did not meet the criteria for having HCAP were designated as having CAP or pneumonia that developed in the community setting (Centers for Disease Control, 2012).

Four independent trained reviewers (an EC physician, a medical fellow, a clinical pharmacist and a nurse), evaluated patients’ EMR to confirm that the patients met the criteria for having pneumonia. Reviewers were in agreement regarding questionable elements, and business rules for standardization. All records of patients who met criteria for pneumonia were eligible for analysis of the CMS PCM. A detail description of the core measures can be found at The Joint Commission website (Joint_Commission, 2013).

Phase-1

The objective of this phase was to evaluate baseline practices, identify barriers impeding compliance with indicators, and improve compliance with PCM to 80%. The quality model for improvement used was PDSA cycle. We retrospectively reviewed the charts of each of the 187 patients who presented to MD Anderson’s EC in May, June, and July of 2006 with an ICD-9 diagnosis of pneumonia and who met the inclusion criteria.

To analyze the potential causes of poor compliance with PCM, a cause and effect analysis was developed that highlighted the major categories such as blood culture and timing of antibiotics. The analysis revealed that the EC staff were not aware of the importance of achieving the quality indicators for pneumonia. This was because our institution was Prospective Payment System exempt and we were not required to report core measures. We also made a process map to track patients from their initial arrival to the EC through their disposition. We identified areas needing improvement and implemented the following interventions:

1. A cup for sputum collection is given during triage or in the room, to facilitate collection.

2. Timing of, and personnel involved in, obtaining cultures were clarified.

3. The laboratory sheet was modified to add two sets of blood cultures and sputum culture.

4. The automated medication dispensing system was programmed to remind nurses to draw cultures before administering antibiotics.

The strategy for implementing these interventions consisted of an intensive education campaign. Education was broad and targeted administrators and all clinical staff, including phlebotomists, ER consultants, and unit clerks. The educational methods used included one to one contact, presentations during scheduled ER staff meetings, and e-mails; posters were placed in different areas of the ER, brochures were created and distributed to all. The intervention phase lasted one month. We collected post-intervention data on each of the 85 pneumonia patients who presented in November and December of 2006.

Phase-2

The objective of this phase was to identify the microbiology and empiric antibiotics for EC cancer patients with pneumonia.

We developed a data collection tool to retrospectively review the microbiology, malignancy and pneumonia types. Data from 85 pneumonia patients in November and December of 2006 and from 187 pneumonia patients in May, June, and July of 2006 were collected.

Phase-3

The objective of this phase was to deploy an institutional pneumonia algorithm and order set. The pneumonia algorithm and order set were developed following recommended treatment guidelines using the data obtained in phases-1 and 2 (“Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare-associated pneumonia,” 2005). The institution adopted the algorithm and order set as best practice of care in December 2007 (Online Figure 1). In February 2008, the pneumonia order set became available in the EMR for institution-wide use. For patients with pneumonia who were admitted to the ICU, the ICU order set was applied. The strategy to initiate change consisted of intensive education initially and then yearly before the pneumonia season. After the pneumonia order set was implemented, we collected the PCM data of 163 patients with pneumonia who presented to the EC in February, October, and December of 2008. To determine the pneumonia order set was used correctly, we analyzed the data of 113 patients with pneumonia who presented to the EC during two randomly selected weeks in each month in October 2010 and February 2011. We used Minitab as the statistical program for control chart analysis.

Phase-4

The objective of this phase was to determine whether the institution was in compliance with PCM as recommended by the National Hospital Quality Measures. The microbiology data was also collected and analyzed during this phase.

We identified 427 pneumonia patients who were admitted through the EC in Q4 2011 (October–December) or Q1 2012 (January–March). On the basis of a 95% confidence interval as recommended by CMS, we randomly sampled 174 patients. Those patients’ data were then abstracted by four independent reviewers directly into University HealthSystem Consortium (UHC); and then validated for accuracy based on the Rate Based Measure Category Report and the Failed Case Report provided by UHC.

We used the UHC vendor benchmarking group with all UHC members participating in the Core Measure program. Top performance and benchmarks are calculated quarterly based on the UHC members rates.

Data analysis and Results

The data were collected for each phase contemporaneously, utilizing the hospital data of patients seen in the EC as mentioned previously. The denominator used to calculate percentage was the number of all patients who qualified as having a diagnosis of pneumonia and who met all inclusion criteria. Main outcome measures included the percentage of patients who met the PCM evaluated in phases-1 and 4. A standard control chart (p-chart) was used for phase-3 analysis looking at the correct use of recommended antibiotics as per pneumonia order set. The phase-4 data were analyzed by quarter-year result. PCM data were entered into the University HealthSystem Consortium (UHC) Core Measures Database. The UHC programs the specifications into the database automatically and provides the data exchange directly to Quality Net. We also evaluated the reasons for failing to meet benchmarks for the data collected in phase-4. More than 80% of the patients who presented with pneumonia to the MD Anderson EC had HCAP (Online Figure 2).

Phase-1

We identified 187 EC patients who had pneumonia. Ninety-three patients (49.7%) had hematologic malignancies (HM), 93 patients (49.7%) had an underlying solid tumor and 1 patient (0.5%) had no cancer. At that time, 24.6% sputum producers had cultures obtained; 73% had blood cultures drawn before initiating antibiotics; and 78% received antibiotics within 4 hours of arriving to the EC.

Educational Content

After the intervention, the percentage frequency of blood cultures increased from 73% to 90% (99.9% CI, P<0.001). The percentage frequency of sputum cultures increased from 24.6% to 34% (P=0.336). After the pneumonia algorithm and order set was implemented, blood cultures were obtained in 86% and sputum cultures were obtained in 51% (P=0.04) of the patients. Ninety-four percent of EC patients received antibiotics (Table 1).

Table 1.

Pneumonia metrics (on line)

Metric *	Baseline Data May, June, July 2006	After Initial Intervention November, December 2006	After Pneumonia Algorithms and Orders Set Intervention (February, October, December 2008)
	Total 187	Total 85	Total 163
Blood cultures obtained before antibiotics	73%	91%	86%
Sputum culture obtained from producers	24.6% (17/69)	34% (12/29)	51% (23/45)
Antibiotic administration while in the emergency center (mean antibiotic time in minutes)	96% 190.52 min	93% 186.96 min	94% 181.33 min
Antibiotic administration with the first 2 hours	51%	45%	40%
Antibiotic administration within the first 4 hours	78%	79%	77%
Use of order set; correct use of recommended antibiotics	N/A	N/A	21% 40% (used general order sets, with recommended antibiotics by guidelines)

Abbreviations: min-minutes;

^*Target post intervention ≥ to 80%

Phase-2: Microbiology

Of the 272 patients who were analyzed in phases-1 and 2, 98 (36%) had positive cultures from respiratory specimens and/or blood. Of those 98 patients, 57 (58%) had HM, and 41 (42%) had solid tumor. Respiratory specimen cultures were positive in 65 patients (66%), blood cultures were positive in 21 patients (21%), and blood cultures and respiratory cultures were both positive in 12 patients (12%). Sixty-one patients (61%) had monomicrobial infections, and 37 patients (38%) had polymicrobial infections. The organisms isolated most often from the respiratory specimens of patients with solid tumors were Pseudomonas species, Stenotrophomonas maltophilia, Streptococcus species, and Staphylococcus aureus. A similar pattern was seen in patients with HM including resistant organisms. In patients who had positive blood and respiratory cultures, the blood culture isolates were often different from the respiratory specimen isolates. Candida species, but no molds, were isolated from patients with solid tumors, whereas molds (mainly Aspergillus species) were isolated from patients with HMs. Two patients had mycobacterial infections, and one patient had Pneumocystis pneumonia. The phase 2 contributed to identifying local microbiology affecting our patient population and to direct antimicrobial treatment.

In phase-4, of the 174 patients evaluated, respiratory specimens from 64 of the 164 HCAP patients (39%) and 8 of the 10 CAP patients (80%) were collected. The distribution of the pathogens isolated from sputum cultures, bronchoscopy specimens, and/or blood from these patients is shown in Table 2.

Table 2.

Distribution of Bacterial Isolated Pathogens Between 174 Patients with community-acquired pneumonia (CAP) and healthcare-associated pneumonia (HCAP) in patients with cultures from Phase 4-From October 2011 to March 2012

Bacterial Pathogens From Sputum or Bronchoscopy (N = 35)	Total (MDR*)	CAP	HCAP (MDR*)
Sputum cultures and bronchoscopies	73	9	64
Positive	30	4	26
Gram-positive pathogens	9	3	6
Staphylococcus aureus	8	3	5
MSSA**	2	2
MRSA**	6	1	5
Enterococcus sp	1		1
Mycobacterium arupensis	1		1
Gram-negative pathogens	25	2	23
Pseudomonas aeruginosa	9 (2)		9 (2)
Haemophilus sp	5		5
Klebsiella pneumoniae	3		3
Serratia marcescens	1		1
Escherichia coli	2 (1)		2 (1)
Enterobacter cloacae	1	1
Stenotrophomonas maltophilia	1		1
Legionella sp	1		1
Citrobacter koseri	1	1
Moraxella catarrhalis	1		1
Polymicrobial infection	2	1	1

^*MDR, multidrug resistant pathogen

^**MSSA, methicillin-sensitive Staphylococcus aureus; MRSA, methicillin-resistant Staphylococcus aureus

Phase-3: Order set

In 2008, after it became available in the EMR, the pneumonia order set was used for 21% of the 163 pneumonia patients who presented to the EC. However, for 40% of the patients with pneumonia where different admission orders were used, antibiotics were administered appropriately. After an intense education effort, the proportion of patients for whom the correct pneumonia order set was used was 77% (CI, 0.68–84) demonstrating an improvement over time, from the population studied during February, October and December 20, and the sample studied from October 2010–February 2011 (Online Figure 3).

Phase-4: Compliance with PCM

During phase-4, more than 89% of the patients we evaluated were treated in compliance with PN 3b, and all patients admitted to the ICU were treated in compliance with PN 3a. However, poor compliance scores were obtained for PN 6, PN6a, and PN 6b. We had 0% compliance for measures PN 6, PN6a, PN6b for initial antibiotic selection for CAP in immunocompetent ICU and non-ICU patients. The reasons for patients’ exclusion from the analysis are given in Table 3.

Table 3.

Phase 4 Reasons for Exclusion in CY2011 Q4 and CY2012 Q1

Reason for Exclusion	Q4 CY2011			Q1 CY2012
Total Patients Sampled	87			81
	PN-3a	PN-3b	PN-6	PN-3a	PN-3b	PN-6
Total Number of Excluded Patients	80	31	85	73	23	82
Normal or absence of new findings in chest x ray, or no chest-x-ray performed	7 (8.8%)	7 (22.6%)	7 (8.2%)	3 (4.1%)	3 (13.0%)	3 (3.7%)
Blood cultures collected after antibiotics administration/antibiotics not received within 24 hours	0	3 (9.7%)	NA	0	6 (26.1%)	0
Involved in clinical trial for same condition	1 (1.1%)	1 (3.2%)	1 (1.2%)	1 (1.4%)	1 (4.3%)	1(1.2%)
Comfort measures ordered on Day 0 or Day 1 of admission/discharge disposition-acute care or expired or left against medical advice	7 (8.8%)	8(25.8%)	7 (8.2%)	2 (2.7%)	3 (13%)	2 (2.4%)
Compromising condition or prior hospitalization in 14 days or both	0	0	8 (9.4%)	0	0	8 (9.8%)
Patient has HCAP per CMS criteria	0	0	49 (57.6%)	0	0	54 (65.9%)
Absence of documentation of pneumonia in ED or direct admit	9 (11.3%)	12 (38.7%)	9 (10.6%)	10 (13.7%)	10 (43.4%)	10 (12.2%)
Patient was transferred from another hospital or ED	1 (1.1%)	0	1 (1.2%)	2 (2.7%)	0	2 (2.4%)
Duration of stay less than 1 day admission	1 (1.1%)	0	3 (3.6%)	1 (1.4%)	0	2 (2.4%)
Patients were not admitted to the ICU within the first 24 hours	54 (67.5%)	0	0	54 (74.0%)		0

Abbreviations: HCAP-healthcare-associated pneumonia; CMS-Center for Medicare & Medicaid Services; ED-emergency department; ICU-intensive care unit

^*Exclusion criteria are not mutually exclusive. Meaning one measure may or may not be applicable to other measures resulting in duplication of exclusions. The same patient may be excluded from PN 3a, PN 3b as well as PN 6b. Total number of excluded patients for that measure is reflected in the total minus the denominator of the results.

^**PN6a and PN6b exclusion criteria is not listed but reflected in the total number of patients stratified by ICU (6a) and non ICU admission (6b).

6a is the inverse exclusion of 6b.

Discussion

We have achieved the main goal of optimizing care in patients with pneumonia; however it did not translate into compliance with all PCM. Some of the antibiotics administered did not fall under recommended antibiotics by CMS measurements. We describe a multiyear, multidisciplinary process of quality improvement initiatives aimed at improving our compliance with PCM and care of cancer patients who present to a comprehensive cancer center with pneumonia.

In phase-1 of the study, our compliance with established pneumonia performance indicators improved significantly after quality improvement initiatives were implemented. We thus achieved our goal of obtaining blood cultures before antibiotic administration for more than 80% of patients. We significantly improved on the rate for obtaining sputum cultures in producers. The PCMs for antibiotic timing have changed since phase-1 was completed. However, more than 93% of the patients received antibiotics while in the EC. These improvements were sustained after the algorithm and order set was implemented.

In phase-2, we found that the organisms that cause CAP in other patient populations (e.g., Streptococcus pneumoniae, Haemophilus influenzae, Legionella species, Mycoplasma pneumonia, Chlamydia pneumoniae) are seldom seen in our cancer patients. Both multidrug-resistant and extended-spectrum beta-lactamase pathogens were identified in HCAP patients; consequently, empiric antibiotic therapy options for cancer patients who present with pneumonia may vary considerably from those mentioned in various guidelines (Lionel A. Mandell et al., 2007). Ideally, empiric antibiotic therapy for such patients should be based on local microbiologic and susceptibility data.

The Pneumonia Team found that meeting the current PCM does not necessarily translate into optimal care for cancer patients. Patients with pneumonia at MD Anderson’s EC are divided into two distinct groups: patients with solid tumors and patients with HM. The microbiology in both groups was consistent with HCAP, not CAP.

Most of the patients in the present study met the clinical and microbiologic criteria for HCAP; therefore, CAP patients comprised a small portion of the study population. The pneumonia algorithm and order set was implemented to standardize, minimize variation and to match the care needs of our patient population and the framework to improve outcome. A previous study has demonstrated that the use of a standardized pneumonia order set can improve PCM compliance and reduce in-hospital mortality (Mandell et al, 2007). In phase-3, we observed an improvement in the utilization of the pneumonia order set. We believe intense and yearly education was the main factor driving this gain.

In phase-4, we discovered that MD Anderson’s compliance with some of the pneumonia indicators was poor and worse than that of non-specialty hospitals. In those instances, the denominator used to calculate the compliance rates represented a considerably small sample of the study population. This difference may be due to the fact that MD Anderson is a comprehensive cancer center whose patient population is markedly different from that of non-specialty hospitals. Also, antibiotic treatment was not in accordance with the CMS guidelines for antibiotic use in ICU or non-ICU patients who have CAP. However, antibiotic treatment was given in accordance with institutional and local guidelines. Similar findings regarding non-adherence to recommended antibiotics by CMS was observed in patients with pneumonia admitted to ICU and in those with previous methicillin-resistant Staphylococcus aureus infection in other study (Shahian et al., 2011).

When looking at the specific PCM, analysis of the poor compliance with PN-3b revealed that in some cases blood cultures were collected a few minutes after antibiotics were administered. Patients who arrive late in the evening to the EC and are treated and then admitted as inpatients the next morning, had their admission date entered in the UHC as the day after EC treatment. Thus, it appears that all blood tests, imaging studies, and antibiotic administrations were performed before the patient arrived at the hospital. We found this to be a system issue with timing of procedures.

Analysis of the poor compliance with PN 6a and PN 6b demonstrated that the three most often-given antibiotics, piperacillin, tazobactam and cefepime in combination with vancomycin, were not concordant with the antibiotics recommended by the PCM for CAP as recommended by the CMS guideline (Joint_Commission, 2013).

Our study was not without potential limitations. One potential limitation was the variability of the chart reviewers; four different reviewers abstracted pneumonia data from EMR, which may have led to inconsistencies in the way the data were abstracted. However, we were confident that, owing to prior discussions and collaboration among the reviewers, their abstracted data would be of similar standards and accuracy, and we performed a validation study that confirmed this finding. Another potential limitation was the lack of comprehensive EMR, which made data abstraction a time-consuming endeavor. The reviewers had to retrieve various documents to find the most accurate data. The exact timing of antibiotics administration could be inaccurate owing to a lack of medication scanning process prior to medication administration. Because the study was conducted at a highly specialized EC, its results may not be generalizable to other emergency departments; however, they may apply to hospitals or centers that treat cancer patients.

On the basis of on our findings, we offer the following recommendations: To improve performance indicators compliance and adhere to quality improvement projects, institutions should establish and monitor education programs to reflect the changes related to PCM. The development of a clinical pathway and an order set helps ensure the delivery of the high-quality care necessary to achieve the best outcomes in cancer patients with pneumonia.

The operational definition of a measure can become extremely complex and important when making national comparisons. Successfully and meaningfully measuring the performance of PCM in cancer patients requires special consideration of the type of pneumonia that is prevalent and metrics regarding recommendation of antibiotics for this population should be incorporated into the PCM. Our findings provide important information about pneumonia measurements in a cancer setting that policy makers should consider.