Rapid Implementation of Evidence-Based Guidelines for Imaging After First Urinary Tract Infection

Abstract

BACKGROUND AND OBJECTIVES:

The American Academy of Pediatrics published a new guideline for management of first urinary tract infection (UTI) in children aged 2 to 24 months in September 2011. The imaging evaluation changed from the previous guideline to recommend voiding cystourethrogram (VCUG) only for patients with an abnormal renal and bladder ultrasound (RBUS). The objective was to decrease the proportion of guideline-eligible children with a normal RBUS who underwent VCUG from median of 92% for patients treated as inpatients and 100% for patients treated in the emergency department to 5% in both settings.

METHODS:

This was a quality improvement implementation study in a large academic medical center. Key drivers included: appropriate guideline knowledge, timely identification of guideline eligible patients, and effective communication with the community-based primary care provider. A multidisciplinary team developed and tested interventions. Impact was assessed with annotated run charts. Statistical comparisons were made with χ² analysis and Fisher’s exact test.

RESULTS:

The proportion of children with first UTI and normal RBUS who underwent VCUG decreased from a median of 92% to 0% within 1 month of initiating the project among those hospitalized and from 100% to 40% within 4 months among those diagnosed in the emergency department. Rates have been sustained for 12 months and 8 months, respectively. Interventions using the electronic medical record and ordering system were most impactful.

CONCLUSIONS:

Rapid adoption of evidence-based UTI care across multiple settings is achievable. Practice change occurred faster and to a greater magnitude in the inpatient setting compared with the outpatient setting.

Substantial variation exists in the delivery of care recommended in evidence-based guidelines for children with urinary tract infection (UTI) in both inpatient and outpatient settings.^1–3 Inpatient use of UTI clinical practice guidelines with an admission order set is associated with shorter length of stay and lower hospital costs,² but variation in imaging practices persists. Additionally, the majority of children with UTI are never hospitalized and variation in UTI care is more pronounced in the outpatient setting.^3,4

In September 2011, the American Academy of Pediatrics (AAP) released an updated Clinical Practice Parameter for treatment of febrile infants aged 2 to 24 months with first-time UTI.⁵ This guideline included a change in recommended imaging: voiding cystourethrogram (VCUG) was no longer recommended for patients without abnormalities on renal and bladder ultrasound (RBUS).^5,6 At our institution, the baseline median rate of VCUG completion after normal RBUS for children treated in the inpatient setting was 92% and the rate for those diagnosed in emergency department (ED) or urgent care settings and discharged from the hospital was 100%. We sought to implement the new evidence-based imaging recommendations with the goal of decreasing the proportion of guideline-eligible patients with a normal RBUS who underwent VCUG in the 60 days after diagnosis during hospitalization or an ED visit to a median rate of 5%. Although a median rate of 0% would reflect 100% compliance with the AAP guideline, we selected a goal of 5%, a priori, to allow for the possibility of an “atypical or complex clinical circumstance,”⁵ as indicated in the guideline, which prompted a provider to obtain a VCUG.

Methods

We used quality improvement (QI) methods to plan and implement strategies for rapid evidence adoption of the AAP UTI guidelines for 2 groups of patients initially diagnosed and treated at the Cincinnati Children’s Hospital Medical Center (CCHMC). The 2 groups were categorized on the basis of care settings: inpatient and outpatient (diagnosed in the ED and discharged from the hospital). inpatient and outpatient, diagnosed in the ED and discharged from the hospital.

Setting

The CCHMC is a large, urban pediatric academic medical center with an electronic health record (EHR) system at all hospital sites and a history of developing and implementing evidence-based guidelines.^7–9 This work was supported by the REACH (Rapid Evidence Adoption to improve Child Health) core at CCHMC. REACH aims to decrease the time to reliable adoption of evidence by using QI methods.¹⁰

Patients of the general pediatric service are admitted at either the main campus or the satellite community campus. At the main campus, care is provided by teams of residents supervised by Hospital Medicine (HM) faculty for 85% of the patients and by community-based pediatricians for the remaining 15%. At the satellite community campus, HM faculty provide direct patient care. CCHMC provides urgent care at 3 community locations and ED care at 2 locations. There are 277 community-based primary care practices with 909 physicians within the primary referral area for CCHMC. Three practice locations are staffed by CCHMC physician employees.

Planning the Intervention

Our specific aim was to decrease the proportion of guideline-eligible patients with a normal RBUS who underwent VCUG in the 60 days after diagnosis during hospitalization or an ED visit from a median of 92% and 100%, respectively, to 5% within 120 days. Our team identified 6 key drivers for rapid adoption of evidence-based UTI care (see Fig 1): (1) appropriate knowledge, (2) timely and accurate identification of guideline-eligible patients, (3) adherence to evidence-based guideline, (4) timely and accurate ultrasound result interpretation, (5) timely investigation of lack of adherence, and (6) effective and standardized communication of patient plan of care. We targeted most of these key drivers with multiple improvement activities and interventions, as follows: (1) resident lecture, (2) Grand Rounds, (3) faculty lectures, (4) community physician education materials, (5) order set and EHR documentation changes, (6) family education materials, and (7) patient identification and notification of inpatient teams.

FIGURE 1.

Key driver diagram. PCP, primary care physician.

Improvement Activities

Physician Education on UTI Guideline

We presented an overview of the AAP UTI guideline at a resident morning report and HM faculty meeting in September 2011. The ED faculty attended a learning session in October 2011. Hospital Grand Rounds on the guideline occurred November 1, 2011; this weekly lecture is attended by CCHMC faculty and staff, residents, and community physicians. Additional educational interventions included an article in a newsletter posted online and mailed to all hospital and community practitioners in March 2012. We distributed an informational flyer to all community practices in March 2012.

Order Set and EHR Documentation

We reengineered existing UTI EHR-based order sets to align closely with the new guidelines. For example, we removed VCUG from standard admission orders. Additionally, we revised an electronic history and physical (H&P) template to add AAP recommendations to the template’s treatment plan. These EHR changes occurred in September 2011. In January 2012, we added an automated prompt to encourage the use of the order set for guideline-eligible patients at admission.

Family Education Materials

In the ED, we developed and refined discharge educational materials and information sheets for families. We added these new materials to the EHR, which were automatically available when a guideline-eligible patient had an ED discharge order placed with diagnosis of UTI. We updated additional online educational materials available to families, patients, and community providers on the hospital Web site. These interventions occurred in March and April 2012.

Identify and Mitigate

We completed daily review of newly admitted patients to the HM teams to identify potential guideline-eligible patients. After we identified these patients, the resident team was sent a text-page to inform them of the new guideline. After the first 6 months, we developed an electronic identification strategy that resulted in daily electronic review of admissions to identify those who potentially met guideline eligibility with e-mail notification of study staff. After an additional month of daily e-mail review, we decreased review to monthly surveillance.

Identification of Eligible Subjects

Eligibility criteria for this study mirror those of the new AAP UTI guideline. We included children if they were aged 2 to 24 months with first UTI. The guideline defined UTI as pyuria and ˃50 000 colony-forming units per mL of a single organism from an appropriately obtained urine culture, without obvious neurologic or anatomic abnormalities known to be associated with recurrent UTI or renal damage.⁵ Whereas the guideline specifically covered febrile infants, we included all infants regardless of fever documentation because we could not accurately determine presence of fever before admission and did not want to exclude patients with true first-time febrile UTI. We performed retrospective medical record reviews to obtain preintervention baseline data (September 1, 2010, to August 31, 2011) and intervention period data (September 1, 2011, to October 31, 2012). We identified children hospitalized with first UTI through a query of hospital-billing data by using International Classification of Diseases, Ninth Revision, diagnosis codes for UTI or pyelonephritis (590.1, 590.10, 590.11, 590.2, 590.8, 590.80, 590.81, 599.0, or 771.82). We identified children initially treated for UTI in the ED and discharged without admission via a query of patients with either (1) positive urine culture or (2) VCUG or RBUS obtained within 60 days of ED visit. We reviewed all identified medical records to verify eligibility, including first UTI as defined by the guideline, and recorded the dates and results of imaging studies obtained. To be classified as first UTI, patients had to have no previous positive urine cultures, RBUS studies, or VCUG studies in the CCHMC EHR and laboratory systems as well as no reference to previous UTI in the EHR. One of three study personnel reviewed all charts. To ensure reliable chart review, a second blinded reviewer reviewed 50% of the charts (n = 75). Interrater agreement was high (κ = 0.97), with 2 disagreements subsequently resolved through group discussion. For 4 complex ultrasound reports, we circulated reports among the research personnel and reached consensus.

Outcome Assessment

A survey of HM faculty completed before and after the educational session assessed practice behavior intentions for the management of a patient with first febrile UTI described in a clinical vignette. This assessment asked respondents to report on a 5-point Likert scale how likely they typically (preeducation) and how likely they were in the future (posteducation) to (1) order RBUS and (2) order VCUG.

We categorized imaging results as either normal or abnormal. For the RBUS, we recorded any final interpretation other than “normal ultrasound” or presence of “bladder debris” as abnormal. For the VCUG, we recorded any final interpretation other than “normal VCUG” as abnormal. For children who underwent both RBUS and VCUG, we categorized the timing of order entry and image completion as occurring on the same day or different days.

Analysis

For the educational survey data, we compared paired pre- and postsurvey results by using the Wilcoxon signed-rank test. We used run charts with 1-month time intervals to assess change in the proportion of patients with normal RBUS who underwent VCUG within 60 days of diagnosis of UTI. We used median proportions to limit the influence of outliers. Research staff recorded data and created annotated run charts for inpatient and outpatient settings using Microsoft Excel (Microsoft Corporation, Redmond, WA) to assess the impact of interventions on the outcome measure over time. We used established rules for identifying special cause.^11–13 We compared differences in imaging results and timing of image ordering between the preintervention and intervention periods by using χ² or Fisher’s exact test as appropriate.

Human Subjects Protection

The CCHMC Institutional Review Board reviewed the project and considered it to not be human subjects research. Informed consent beyond standard treatment consent was not required.

Results

Educational Intervention

Seventeen HM faculty members completed the pre- and posteducation assessment. In this group, 7 had <5 years of experience, 5 had 5 to 10 years of experience, and 5 had >10 years of experience. After the education session, faculty intentions to obtain an RBUS after first-time UTI were unchanged with 100% reporting intent to “usually” or “almost always” order an RBUS. Faculty reported intent to “usually” or “almost always” obtain a VCUG decreased significantly from 35% to 0% (P < .0001) after the education session.

Inpatient and Outpatient Imaging

In the inpatient setting, 42 patients met inclusion criteria during the preintervention period and 39 eligible patients met inclusion criteria during the intervention period. The median proportion of children undergoing VCUG after normal RBUS decreased from 92% to 0% (Fig 2). This change became apparent within 1 month of project implementation and has now been sustained for >1 year. The percentage with a completed RBUS remained high (100%) during the intervention period (Table 1).

FIGURE 2.

Run chart of the proportion of eligible patients with a VCUG after admission for first UTI and a normal RBUS from July 2010 through October 2012.

TABLE 1.

Ultrasound and VCUG Completions and Findings by Care Setting

	Preintervention, n (%)	Intervention, n (%)	P
Inpatienta
RBUS obtained	41 (97.6)	39 (100)	>.99
RBUS normal	28 (68.3)	22 (56.4)	.36
RBUS normal and VCUG obtained	20 (71.4)	0 (0)	<.0001
RBUS abnormal and VCUG obtained	11 (84.6)	11 (64.7)	.41
VUR identified
VUR Grade 4–5	2 (4.7)	1 (2.6)	>.99
VUR Grade 1–3	6 (14.3)	3 (7.7)	.48
Outpatientb
RBUS obtained	31 (54.4)	39 (51.3)	.86
RBUS normal	25 (80.6)	33 (84.6)	>.99
RBUS normal and VCUG obtained	25 (100)	14 (42.4)	<.0001
RBUS abnormal and VCUG obtained	6 (100)	4 (66.7)	.45
VUR identified
VUR 4–5	1 (1.8)	1 (1.3)	>.99
VUR 1–3	7 (12.3)	7 (9.2)	.58

VUR, vesicoureteral reflux.

^an = 42 and 39 for Preintervention and Intervention groups, respectively.

^bn = 57 and 76 for Preintervention and Intervention groups, respectively.

In the outpatient setting, 57 eligible patients were treated in and discharged from the ED during the preintervention period and 76 patients were treated in and discharged from the ED during the intervention period. The median proportion of children undergoing VCUG after normal RBUS decreased from 100% to 40% (Fig 3). This change occurred within 4 months of project implementation (Fig 3). The percentage with a completed RBUS was unchanged preintervention (54.4%) to postintervention (51.3%) (Table 1).

FIGURE 3.

Run chart of the proportion of eligible patients in the ED for first UTI with a VCUG and a normal RBUS from July 2010 through October 2012.

Timing of Image Ordering and Completion

For admitted patients, the rates of RBUS completion before discharge were similar for the preintervention and intervention periods (pre: 41 of 42 patients = 98%; post: 39 of 39 patients = 100%; P > .99). The rates of VCUG completion before discharge were also similar (pre: 5 of 31 patients = 16%; post: 3 of 11 patients = 27%; P = .41). For patients diagnosed in the outpatient setting, no RBUSs or VCUGs were completed before ED discharge in either period. For patients undergoing both RBUS and VCUG, the majority (15 of 17 = 88%) had same-day ordering and same-day completion of both imaging studies. A minority (2 of 17 patients = 12%) had orders placed on different days and studies completed on different days, with VCUGs obtained after an abnormal RBUS in both instances. There were no patients who had orders placed simultaneously and then had imaging completed on different days. All orders were placed by the primary care provider.

Imaging Abnormalities

During the intervention period, abnormal RBUS findings for those patients undergoing subsequent VCUG (n = 15) and those not undergoing VCUG (n = 8), respectively, included urothelial thickening (n = 10, n = 7), hydronephrosis (n = 7, n = 1), collecting system duplication (n = 3, n = 0), pelvicaliectasis (n = 2, n = 1), and renal agenesis (n = 1, n = 0). Among patients with an abnormal RBUS during the intervention period, 64.7% of inpatients and 66.7% of outpatients completed a VCUG. Rates of vesicoureteral reflux identification were similar in the pre- and postintervention periods (Table 1).

Discussion

Our bundled intervention significantly decreased the median monthly percentage of children aged 2 to 24 months with first UTI and normal RBUS who underwent VCUG from 92% to 0% in the inpatient setting and from 100% to 40% in the outpatient setting. This study reveals that evidence-based guidelines can be rapidly adopted, although adoption rates can vary by clinical setting. Our educational intervention decreased the percentage of HM faculty respondents who intended to obtain a VCUG without affecting the percentage that would obtain an RBUS. The percentage of patients with a completed RBUS remained similar in both settings. Nearly two-thirds of patients with an abnormal RBUS completed a VCUG regardless of setting. Among outpatients who completed a VCUG, 88% (15 of 17) had simultaneous ordering and same-day completion of both imaging studies, whereas only 1 of these 17 children had an abnormal RBUS.

The magnitude and rapid nature of this practice change are noteworthy. Wide variation in VCUG completion rates has been described for patients with UTI who were admitted at 1 of 25 children’s hospitals.² A recent study found a decrease in VCUG completion after first-time UTI by implementing the United Kingdom’s National Institute for Health and Clinical Excellence 2007 guidelines, with 99% VCUG completion in the year before and 13% VCUG completion in the year after implementation.⁴ However, the rates of practice change are difficult to compare because the previous study did not report data in time series and had a 4-month “run-in” period in which outcomes were not reported.⁴ At our institution, there is a culture of QI and evidence use that likely facilitated rapid adoption of the evidence-based guidelines for UTI management.^9,14

We found adoption of the evidence was slower for those patients diagnosed in the ED and then cared for by their primary care physician. This finding is supported by other studies that have shown that guideline compliance has been lower in the outpatient setting when compared with the inpatient setting.³ Although we successfully decreased the proportion of guideline-eligible children with normal RBUS who underwent VCUG, we have not yet reached our goal in the outpatient setting. Although education regarding the evidence may be necessary to initiate change, these interventions appear to be insufficient to reliably change practice.¹⁵ We believe that changes made within the EHR, specifically to the admission order set and the H&P template, drove success in the inpatient setting. EHR-based physician prompts, such as the one to encourage use of the appropriate order set, have been shown to increase use of evidence-based care practices.^2,3,16 Developing such interventions for patients in the outpatient setting has been more challenging because practices in our region lack a shared EHR. Although we were able to change ED discharge materials and online educational materials for providers in the community, we have not been able to develop interventions that can ensure consistent use. After examination of our failures in the outpatient setting, it appears that community physicians are ordering both RBUSs and VCUGs at the same time, perhaps for ease of ordering and family scheduling. Interventions are needed that would allow the result of the RBUS to inform the decision to complete a VCUG.

This study has limitations. First, the design did not allow us to determine the strength of individual intervention components or discern the impact of AAP efforts to disseminate the UTI guideline. We purposefully timed the rollout of our interventions to be synergistic with AAP efforts. Given the long-standing difficulty changing practice with evidence-based guidelines, it appears likely that our interventions had a strong effect. Attributing the decrease in VCUG completion to our intervention efforts is also supported by our own difficulty in changing practices among patients who transitioned from the ED to community settings where intervention efforts were limited to physician and parent education. Second, it is possible that we misclassified patients with recurrent UTI as a first-time UTI. We attempted to minimize this risk by reviewing laboratory data and documentation including H&P, discharge summary, and radiology reports for any evidence of past UTI. Third, we recorded VCUG completion within 60 days of diagnosis for all patients. Within our primary service area, CCHMC is the only institution completing VCUGs; however, it is possible that the VCUG was completed after this 60-day time frame. It is possible, although unlikely, that the RBUS was completed at another institution for outpatients. Additionally, whereas we did include a local primary care pediatrician on our team, eliciting further insights on both the image-ordering and decision-making processes directly from community providers is needed.

We tracked additional patient outcome measures to assess for unintended consequences of our intervention. All admitted patients completed an RBUS before discharge. Only half of all outpatients completed an RBUS, but this rate was similar to the preintervention period. We can only speculate that some parents and/or physicians decided to forego imaging, whereas others may have experienced barriers to completing an RBUS. We measured the VCUG completion rate for patients with abnormal ultrasound results. Across both settings, nearly two-thirds of patients with an abnormal RBUS completed a VCUG during the intervention period. Among the 8 patients who did not complete a VCUG after an abnormal RBUS, 7 had “mild urothelial thickening” as the only RBUS abnormality. The AAP guideline notes that RBUS findings in the acute infectious period may be misinterpreted as obstruction. We speculate that the ordering physician may have attributed urothelial thickening to acute infection and decided not to obtain a VCUG. These results suggest that the current system of care offers a trade-off. Although the completion of an RBUS is more reliable when obtained in the inpatient setting (100% vs 51%), results are more likely to be normal when obtained in the outpatient setting (85% vs 56%), perhaps due to resolution of acute inflammation. Additionally, despite lower rates of VCUG completion, rates of vesicoureteral reflux detection in all patients with first UTI were similar during the 2 time periods.

Conclusions

This study reveals that rapid evidence adoption can be achieved by using QI methods. The faster rate and greater magnitude of adoption in the inpatient setting suggests that additional interventions may be needed to reliably deliver evidence-based UTI care for patients treated in the outpatient setting.

Glossary

AAP

American Academy of Pediatrics

CCHMC

Cincinnati Children’s Hospital Medical Center

ED

emergency department

EHR

electronic health record

H&P

history and physical

HM

Hospital Medicine

QI

quality improvement

RBUS

renal and bladder ultrasound

UTI

urinary tract infection

VCUG

voiding cystourethrogram