Quality Improvement Interventions to Improve Critical Congenital Heart Disease Screening

Introduction:

Newborn screening for critical congenital heart disease (CCHD) using pulse oximetry improves detection and is associated with decreased related infant mortality. In 2015, the Healthy Hearts of Babies Act required hospitals to screen all newborns in the District of Columbia for CCHD using pulse oximetry and to provide documentation of individual screening results to the Department of Health. A regulatory report from the electronic health record revealed an opportunity to improve both documentation and protocol adherence within our hospital. We aimed to reduce documentation errors and protocol violations by 75% and sustain this improvement for 6 months.

Methods:

In February of 2014, our center, a large free-standing children’s hospital, implemented CCHD screening in the neonatal intensive care unit on all infants without known congenital heart disease or receiving supplemental oxygen. During the intervention period (January 2016 to December 2018), an interdisciplinary team engaged in regular review and analysis of reports, monthly closed-loop feedback, and iterative refinements to the electronic health record. Statistical process control charts were used to compare a baseline period to the intervention period and track monthly progress.

Results:

Between February 2014 and December 2018, we screened 2,214 infants for CCHD. The average percentage of documentation errors decreased from 23.5% during the baseline period to 1.2% during the intervention period, a sustained reduction for over 2 years. Protocol violations occurred at an average of 2.1% in the baseline period, with a sustained decrease to 0.6% during the intervention period.

Conclusions:

This multimodal quality improvement project demonstrated a sustained reduction of CCHD screening documentation errors and protocol violations.

INTRODUCTION

Critical congenital heart disease (CCHD) is the leading cause of death in infants due to congenital malformations.¹ Implementation for CCHD screening using pulse oximetry in asymptomatic newborns before discharge has improved our ability to detect CCHD and is associated with a reduction in CCHD infant deaths in states that require screening.²

Beginning in 2014, Children’s National Health System (Children’s National) required all transferred newborns to be screened for CCHD by hospital policy. In 2015, the District of Columbia passed The Healthy Hearts of Babies Act, a CCHD screening mandate.³ This law also required individual screening results to be reported to the Department of Health according to recommended best practices.⁴ As of July 2018, universal screening of all newborns for CCHD is recommended or required in all 50 states in the United States. The creation of a regulatory CCHD screening electronic report allowed for both internal and external reviews. An internal review revealed an opportunity to improve documentation, reporting, and protocol adherence and prompted the initiation of this quality improvement (QI) project.

The team investigated different modalities to improve the effectiveness and accuracy of screening and found other programs struggled with issues related to protocol adherence and incorrect results documentation, which can contribute to ineffective screening. One study found that misinterpreted results may occur as often as 0.2% of the time.⁵ In another evaluation of a newborn CCHD screening program, incorrect interpretation of screening results occurred in 0.1% of cases and incomplete documentation in 6.5% of cases.⁶ Other hospitals have identified the use and modification of the electronic health record (EHR) as a potential way to reduce screening protocol deviations.⁷ The purpose of our QI project was to decrease errors associated with the CCHD screening process at our institution. Specifically, the QI team sought to improve adherence to the CCHD screening protocol and ensure the correct documentation of the results within the EHR.

We aimed to reduce CCHD screening documentation errors and protocol violations by 75% and sustain this improvement for 6 months.

METHODS

Setting and CCHD Screening Implementation

Children’s National is a free-standing pediatric hospital in Washington, DC, with a level 4 neonatal intensive care unit (NICU) nursery. While not a birthing hospital, the NICU patient population includes patients transferred from outside birth hospitals. Although a missed diagnosis of CCHD is far less common in the NICU setting, many states do not include exemptions from screening for special populations (such as patients in intensive care units).

In February 2014 per policy, Children’s National implemented CCHD screening of all neonates who were transferred to our hospital and had not already been screened at their birth institution. A “train the trainer” approach⁸ and use of the Children’s National Toolkit⁹ were used to educate clinical staff. Pediatric residents and nurse practitioners entered orders for screening, and bedside nursing staff performed the CCHD screening.

Children’s National implemented the CCHD screening protocol recommended by the American Academy of Pediatrics (AAP) for use in well-baby nurseries (Fig. 1).¹⁰ Eligibility criteria for NICU screening are often developed locally and hospital specific; currently, no national recommendations for NICU screening exist, and best practices around NICU screening are still emerging.¹¹ Through clinical team consensus, modifications to the eligibility criteria were tailored specifically for the NICU infant population and included a requirement that the infant is off supplemental oxygen for at least 24 hours before screening. We excluded infants who were diagnosed with CCHD prenatally, who had received an echocardiogram, and who were receiving supplemental oxygen at discharge.

Fig. 1.

AAP recommended CCHD screening protocol. AAP, American Academy of Pediatrics; Pulse Ox, pulse oximetry; RH, right hand. Image reproduced with permission from the Children’s National CCHD Screening Toolkit.

Documentation of CCHD screening results took place within the EHR (Cerner Corporation, North Kansas City, Missouri), allowing the team to develop automated electronic reports from a query that was run every month and submitted externally every quarter. The QI opportunity described in this article arose as the implementing team reviewed the first monthly report and found an unsatisfactorily high rate of error. The team categorized error types into those that violated the screening protocol and those that staff documented incorrectly.

Baseline data, pulled retrospectively using the report query function, showed that screeners were documenting incorrectly 23.5% of the time for screens performed in the 6-month baseline preintervention period (July to December 2015). The most frequently occurring mistake was documenting the results in the wrong field. To better understand workflow and the reasons for common errors, the team surveyed pediatric residents, attending physicians, and nurses regarding obstacles to correct ordering and documentation, revealing that the order name and wording of the documentation forms were contributing factors.

This project was undertaken as a Quality Improvement Initiative at Children’s National, and it does not constitute human subjects research. As such, the Institutional Review Board did not require review and approval of the project.

QI Interventions

QI Team

QI is, ideally, a continuous process. The QI team formed several years before the study period and had assisted in the development and implementation of CCHD screening locally. The team consisted of a CCHD screening program/QI nurse, a nurse within the informatics department at the hospital, a neonatologist, a pediatric cardiologist, and unit nurse educators.

Definitions

Protocol violations included (1) failure to rescreen infants who should have been rescreened, (2) rescreening an infant who failed the initial screen, and (3) rescreening infants who passed. We defined documentation errors as (1) documenting a second or third screen without having documented an initial screen and (2) documenting incorrectly whether an infant was 24 hours of age at the time of the screen.

Key Interventions

The issue of protocol interpretation errors was considered during the design of the original documentation page within the EHR. Before implementation, we embedded decision support into the results documentation page and implemented just in time education. After entering the screening values into the results page in the EHR, a result of a pass, rescreen, or fail (along with the follow-up procedure for each) guided the screener (Fig. 2).

Fig. 2.

EHR Screenshots and enhancements.

In January of 2016, the CCHD screening data reports were created and made available to the implementing team. The first report was reviewed that month, and we sent the first quarterly report to the DC Department of Health in April of 2016. The QI intervention period lasted from January 2016 to December 2018. A key driver diagram was created to illustrate the interventions and goals of the QI project (Fig. 3).

Fig. 3.

Key driver diagram.

Utilizing the Model for Improvement¹² and Standards for Quality Improvement Reporting Excellence (SQUIRE) framework,¹³ the team developed aims and measures to track whether changes were improvements and to report/share results. Plan, Do, Study, Act cycles were used to implement and evaluate small tests of change throughout the implementation period.

Feedback Reports.

Monthly electronic reports were systematically reviewed to improve protocol adherence and decrease documentation errors locally. Feedback reports to front-line nurses and physicians were emailed monthly via password-protected excel reports to close the loop on common issues, starting in January 2016. In-person team meetings occurred at least monthly during the initial roll-out period and then as needed. The team addressed errors as discovered and conducted apparent cause analysis to develop corrections. Timely feedback on performance to clinical staff assisted in their ability to close the loop on corrections and, if needed, follow-up before discharge if the infant needed to be rescreened or evaluated. Protocol violations were infrequent. Therefore, interventions were limited to one-on-one notification and reeducation by unit clinical educators.

EHR Enhancements.

When incorrect nursing documentation continued, the team made further enhancements to the EHR to clarify the sequence labels of screening steps and to block both orders and documentation forms chosen out of correct sequence. In June 2016, the focus of the team was on creating improvements to electronic form language and the discharge summary to encourage correct documentation through the use of prompts and automatic defaults where appropriate. The changes became active in the system in July of 2016. A tailored educational module was also inserted into the monthly orientation for pediatric residents and nurses based on types of documentation errors discovered and survey results.

We designed other enhancements around the types of documentation errors occurring. These enhancements became active in the EHR system between August and March of 2017 (Fig. 2).

Measures and Analysis

The purpose of implementing CCHD screening is to ensure that the clinical team does not discharge infants with CCHD without a correct diagnosis. The global aim of this QI project was to verify that CCHD screening was performed and documented correctly. Inaccurate documentation and performance of screening may increase the risk of a missed case of CCHD.

The primary process measure was the percentage of protocol errors during the study period, and the secondary process measure was the percentage of documentation errors. The study team collected data by review of the monthly data reports pulled from the EHR using a customized report query and discussed the previous month’s data at the beginning of every month.

We selected patient satisfaction scores and overall length of stay (LOS) for the NICU as balancing measures to track whether interventions impacted overall care. Data was pulled from the Press Ganey database and the Children’s Hospitals Neonatal Database from the Children’s Hospitals Neonatal Consortium containing our local NICU dataset.

The Specific, Measurable, Actionable, Realistic and Timely (SMART) Aim, to reduce documentation errors and protocol violations by 75% and sustain this improvement for 6 months, guided team decision making and analysis. We used statistical process control charts to track these measures and inform progress. The percentage of error was calculated by dividing the number of documentation or protocol violations (respectively) by the total number of infants screened every month. Established rules for differentiating special cause and common cause variation were applied to the data.¹⁴

RESULTS

Between February 2014 and December 2018, Children’s National screened 2,214 infants (Fig. 4). The number of infants screened during the baseline period was 188. During the intervention period, we screened 1,564 infants. CCHD screening did not identify any new instances of CCHD during the study period. The total number of infants who passed was 2,212 (4 required a rescreen initially and passed upon rescreen). One infant failed (with values of 88% and 98%) but did not have CCHD or another pathology identified following evaluation. One infant was evaluated by an echocardiogram (protocol violation should have been rescreened) and found to have a small atrial septal defect (ASD).

Fig. 4.

Newborn NICU CCHD screening results from February 2014 to December 2018. This diagram shows the screening results from screening at Children’s National Health System between February 2014 and December 2018, where the total number of infants screened is at the top. The specific categories of those screens that were pass or fail and referred for further assessment are captured. ASD, atrial septal defect.

The average gestational age at birth for admissions between July 2015 and December 2018 was 35 weeks, and the median day of life age at admission was 5 days. The average age at the time of screening was 28 days (range: 25 hours to 66 days).

Protocol Violations

Protocol violations were 2.1% during the baseline period. In the first 6 months of the intervention period, the rate decreased to 0.6%, which we sustained for the remainder of the study period. During the intervention period, 26 protocol errors occurred in 1,564 infants screened. One patient who failed was inappropriately rescreened; 2 patients should have been rescreened but did not have a rescreen documented, and 23 errors stemmed from rescreens performed despite initial passing results. In 2 instances, the rescreen was delayed more than 2–3 hours after the initial screen. The protocol recommends rescreening after 1 hour (Fig. 5).

Fig. 5.

Quality improvement in critical congenital heart disease screening percent protocol violations. This control chart displays the average percentage of protocol violations over the period from July 2015 to December 2018. Based on 9 out of 12 data points below the centerline, in July of 2016, a new centerline was calculated around an average of 0.6% of protocol violations. This is a reduction from 2.1%, which is a 71% decrease. CL, centerline; UCL, upper control limit; LCL, lower control limit.

Documentation Errors

In this QI project, the number of documentation errors decreased from 23.5% in the baseline period to 1.2% in the intervention period. We sustained this decrease for the remainder of the intervention period.

During the intervention period, 71 documentation errors occurred out of the 1,564 infants screened and included (1) documenting a second or third screen without having documented an initial screen (56 errors) and (2) documenting incorrectly that an infant was not yet 24 hours of age (15 errors) (Fig. 6).

Fig. 6.

Quality improvement in critical congenital heart disease percent incorrect documentation. This control chart displays the average percentage of patients who had documentation errors related to CCHD screening. The percent with incorrect documentation is displayed over the period from July 2015 to December 2018. The results included a centerline shift using Nelson’s rules with 9 points below the centerline beginning in July 2016. This shift, from 23.5% to 1.2%, is significant and sustained over 2 years and represents a 95% decrease.

Balancing Measures

Average LOS for NICU patients remained constant at 20 days with a normal variation for the time between July 2015 and December 2018. July of 2017 was an outlier (35 days). That month there were 25 patients with LOS >80 days due to complications of extreme prematurity (average gestation age 25 weeks) and unrelated to newborn screening. Patient satisfaction scores for the NICU during this period stayed at or around 90% as captured in the Press Ganey database.

DISCUSSION

In this single-center QI project, feedback reports and multimodal EHR improvements decreased CCHD screening protocol deviations from 2.1% to 0.6%; documentation errors decreased from 23.5% to 1.2% during the intervention period. The team achieved its aim regarding reduction in documentation errors (95% decrease) and nearly achieved its aim regarding the reduction in protocol violations (71% decrease). These reductions were sustained for over 2 years.

CCHD is the most common birth defect. A missed diagnosis in the neonatal period can lead to significant morbidity and mortality. The current national algorithm endorsed by the AAP¹⁰ is subject to misinterpretation by both nurses and physicians and is a demonstrated area of need for QI/Quality Assurance.^15,16

Based on the findings of 6 federally funded CCHD newborn screening implementation projects, one of the most common challenges to implementation is the difficulty with algorithm interpretation.¹¹ These challenges may be exacerbated by factors specific to screening an NICU population, including a higher incidence of unresolved reasons for cyanosis related to prematurity or illness as well as a higher likelihood that the infant may have already received an echocardiogram before they are eligible for CCHD screening using pulse oximetry. Unlike in a well-infant nursery, where most infants can be screened at or around 24 hours of life, the timing will vary significantly in NICUs where infants cannot be screened until they are off supplemental oxygen and they may have medical conditions that can impact the screening. Further, intensive care nurseries in teaching hospitals, with a monthly rotation of trainees who are unfamiliar with the screening algorithm, may be particularly vulnerable to protocol errors and incorrect documentation. The transfer of neonates from birth hospitals to hospitals with NICUs creates an opportunity for a lapse in the communication of screening information obtained at the birth hospital. Two NICU-specific studies found protocol compliance in the NICU difficult to achieve. In one Texas multi-institution study, between 46.3% and 80% of infants were missed depending on screening characteristics of the infant (with the most premature group of less than 28 weeks being the least protocol compliant).¹⁷ An NICU-specific multicenter collaborative found that the number of infants not screened according to the protocol was approximately 1.1%.¹⁸

Leveraging the EHR to perform QI work has both advantages and disadvantages. Based on the timing of when the data showed special cause improvement (July 2016), the EHR enhancements likely aided in sustainability (despite the turnover of clinical staff and rotating trainees responsible for CCHD screening) but were less likely than the other interventions to have significantly contributed to the initial improvement. By comparison, a team in Washington state implemented a redesign of their EHR documentation system with autocalculated results and achieved a low 0.18% violation rate with 1 high quarter at 1.1%.⁷ Implementation in an NICU, which includes a less uniform and higher acuity patient population, may partially explain initially higher protocol errors despite implementing a system that autocalculated screening results. The implementing team requested the creation of the CCHD screening results report nearly 2 years before it became ready for use. The hospital was able to allocate resources to build the EHR report in response to the newborn screening law that required reporting. This law represented a new opportunity to obtain electronic data. Before January 2016, the inability to pull data or create reports was a significant barrier to being able to conduct QI.

Several strategies were key to the success of this project. The inclusion of an informatics nurse, who was able to develop and provide education related to the EHR enhancements, allowed the team to make timely refinements to the EHR. Bidirectional feedback from screeners allowed the team to achieve a reduction in instances of infants being unnecessarily rescreened despite an initial passing result. Some infants were screened as part of the discharge process despite having been already screened when the infant met the eligibility criteria. Reducing the number of unnecessary rescreens also represents cost savings to the organization by calculating nursing hours saved.

Another success of this QI project was improved communication of the CCHD screening results. Results are pulled into the discharge summary, creating a mechanism to notify out-patient providers of results following the infant’s discharge. Other centers have seen similar improvements in communication at discharge as a result of improvements to the EHR.¹⁹

The protocol implemented and recommended by the AAP is intended for use in infants in a well-baby nursery. Although this does not impact the QI aspects of this study, it may influence the screens ability to detect CCHD in the NICU population. However, the success of this project may be partially attributed to the academic setting and NICU environment, which has the benefit of a lower nurse to patient ratio. Six percent of infants included in this report were screened after being transferred to an acute care unit using the same NICU protocol. Although primarily implemented in the NICU, the EHR order set and enhancements were available hospital-wide, suggesting that the interventions could be implemented in other settings.

Limitations included the inability to confirm that all eligible infants completed CCHD screening, single-center project, and nonuniformity in the location and timing of screening. Additionally, patients screened were predominantly older than those screened in well-baby nurseries. Future directions include additional planned EHR refinements and a project to assess whether all eligible infants are screened before discharge.

CONCLUSIONS

At this large academic institution, following the implementation of systems-level QI interventions for newborn CCHD screening, we achieved and maintained a decreased rate of screening documentation and protocol errors. The interventions developed could be adapted and implemented into other programs screening for CCHD.

ACKNOWLEDGMENTS

We would like to thank nurses Ann-Marie Brown, Craig Woodside, and Jessica Eitel; Jacob Cheng and Michelande Ridore for their assistance with data queries and control charts; Jessica Colyer, Anit Saha, and Neil Bhattarai for their QI consultations; and Lindsay Attaway for her graphic design and editorial assistance.