Effectiveness of a Daily Rounding Checklist on Processes of Care and Outcomes in Diverse Pediatric Intensive Care Units Across the World

ABSTRACT

Background

Implementation of checklists has been shown to be effective in improving patient safety. This study aims to evaluate the effectiveness of implementation of a checklist for daily care processes into clinical practice of pediatric intensive care units (PICUs) with limited resources.

Methods

Prospective before–after study in eight PICUs from China, Congo, Croatia, Fiji, and India after implementation of a daily checklist into the ICU rounds.

Results

Seven hundred and thirty-five patients from eight centers were enrolled between 2015 and 2017. Baseline stage had 292 patients and post-implementation 443. The ICU length of stay post-implementation decreased significantly [9.4 (4–15.5) vs. 7.3 (3.4–13.4) days, p = 0.01], with a nominal improvement in the hospital length of stay [15.4 (8.4–25) vs. 12.6 (7.5–24.4) days, p = 0.055]. The hospital mortality and ICU mortality between baseline group and post-implementation group did not show a significant difference, 14.4% vs. 11.3%; p = 0.22 for each. There was a variable impact of checklist implementation on adherence to various processes of care recommendations. A decreased exposure in days was noticed for; mechanical ventilation from 42.6% to 33.8%, p < 0.01; central line from 31.3% to 25.3%, p < 0.01; and urinary catheter from 30.6% to 24.4%, p < 0.01. Although there was an increased utilization of antimicrobials (89.9–93.2%, p < 0.01).

Conclusions

Checklists for the treatment of acute illness and injury in the PICU setting marginally impacted the outcome and processes of care. The intervention led to increasing adherence with guidelines in multiple ICU processes and led to decreased length of stay.

INTRODUCTION

There has been significant evolution and maturation of critical care in resource limited countries [1, 2]. Several experiences have shown very positive results even with limited resources in terms of lives saved, and its scale-up may be a cost-effective tool as health systems expand [3, 4]. Incomplete knowledge of best practices by frontline health care providers and error-prone care delivery processes has been stated as a limitation to its sustainable growth [5, 6].

Quality improvement is the next major challenge in global health [7], as universal health coverage is scaled up in all regions. Simple interventions, such as timely appropriate antimicrobial treatment, timely recognition and treatment of cardiorespiratory failure, adequate tidal volume mechanical ventilation, prevention of hospital-acquired conditions, and optimizing mobility and sedation, require little-specialized equipment but are hugely contributory to successful outcomes in the critically ill [8].

A systematic approach to quality assurance with the use of checklists and electronic decision support algorithms have long been used in other complex industrial environments but have only recently been applied in acute care hospitals [9, 10]. The use of checklists has been proposed to improve patient safety in surgical settings and in ICUs in developed countries, with encouraging results on both process and outcome measures [11, 12]. Prior studies on incorporating a checklist into the daily rounding processes in the pediatric ICU have demonstrated improved teamwork while reducing the inertia of previous practice and the lack of agreement on goals and measures [13, 14]. Given local resources and a frequent lack of formal training in pediatric intensive care medicine, the expected benefit from a checklist approach to quality improvement processes in low- and middle-income countries are thought to be high.

Checklist for Early Recognition and Treatment of Acute Illness and iNjury in Pediatrics (CERTAINp) has been developed to standardize the approach to the evaluation and treatment of critically ill patients in diverse settings. We hypothesized that the implementation of the checklist would improve adherence to recommended critical care procedures and shorten the length of ICU stay in intensive care units with limited resources in low- and middle-income countries (LMIC). Our primary aim was to evaluate the effectiveness on process and patient outcomes after the implementation of a checklist into the clinical practice of pediatric intensive care units (PICUs) with limited resources in LMIC.

METHODS

This is a pragmatic before–after, prospective cohort study in eight PICUs from China, Congo, Croatia, Fiji, and India (Supplementary Table S2). The study centers were recruited through a global survey [15], conducted through the World Federation of Pediatric Intensive and Critical Care Societies, in which multiple centers expressed interest. After evaluations on resources and research infrastructure, ten centers were recruited. This study had three phases. Phase 1: baseline data collection (3 months or 20 patients per ICU). Phase 2: implementing checklist (3–6 months). Phase 3: post-implementation data collection (6 months or 40 patients per ICU) (Supplementary Fig. S1).

All pediatric (<18 years) patients who required intensive care and were admitted for the first time to the participating ICUs were included. Patients admitted for low-risk monitoring, planned ICU admissions for routine post-operative observation for <24 h after uncomplicated surgery, readmission or transfers from outside ICU, and ICU admissions of <24 h were excluded.

The checklist was designed to standardize the approach to the evaluation and treatment of the acutely decompensating patient. It was created by the investigators, informed by a survey of resources and availability of infrastructure from diverse international settings [15] and based upon best practice evidence, with validation at the primary site [16]. The checklist was available to the centers both as an electronic (web-based and mobile application) and paper format. The rounding checklist had 12 domains and 25 elements and included best practice guidelines for patient safety in the ICU including lung protective ventilation, deep vein thrombosis (DVT) prophylaxis, delirium prevention, and central-line infection prevention (Supplementary Fig. S2).

Before checklist implementation, the participating clinicians were trained by the implementation team through online training, including the study protocol, outcome measures and data collection. After training, the rounding team implemented the checklist in clinical practice. During the morning rounds, the physicians, trainees or nurses made clinical decisions based on the care processes recommendations. ‘Intervention’ was thus, applied at the level of the hospital, not at the level of the individual patient.

The primary outcome was the patient’s ICU length of stay; secondary outcomes included the length of hospital stay, mortality and adherence to recommended practices. Study data were collected and managed using REDCap electronic data capture tools hosted at Mayo Clinic [17]. Each site had a research coordinator responsible for capturing study data during the data collection period (3 months before implementation and 6 months post-implementation of the checklist). The coordinating center responsibilities were administered by the Mayo Clinic. The study database was housed and managed at Mayo Clinic, including enrollment tracking, basic data quality monitoring, data cleaning and outcome tracking.

Variables were described by medians and interquartile (IQR) ranges for non-normally distributed data, and percentages for categorical data. Mann–Whitney U-test was used to compare non-normally distributed data, and Pearson’s Chi-square to compare categorical data. The analysis was conducted using JMP version 13.0.0, and statistical significance was assessed at the 0.05 level. Patients without available data were excluded from the analysis. This study was approved by the Mayo Clinic Institutional Review Board (IRB) and ethics committees of participating institutions as per local requirements, with waiver of consent from the individual patients. The trial was registered at clinicaltrials.gov (NCT02398981).

RESULTS

A total of 10 PICUs were enrolled in the study. Two centers were excluded from analysis for incomplete data (only baseline data was collected), eight centers included in the final analysis. A total of 735 patients were included from April 2015 to December 2017. This included 292 patients in the pre-intervention group and 443 patients in the post-implementation group (Fig. 1). Out of the eight centers, four were from China which included a majority of the patient population in both pre- and post- implementation group [172/292 in pre (58%) and 220/443 (49.65%) in post] (Fig. 2).

Fig. 1.

Flow diagram of the study—patient enrollment in pre- and post-groups.

Fig. 2.

Number of patients enrolled per center in pre- and post-groups.

There was a significant difference in the age and sexual distribution in the pre- and post-implementation groups. The post-implementation group was younger (13.5 months in pre- vs. 9.4 months in the post) and had a proportionally higher number of male patients (56.2% males in pre vs. 63.4% males in the post). There was also a statistically significantly higher proportion of mechanically ventilated patients in the pre-intervention group compared with the post-intervention group (45.7% vs. 37.2%, p = 0.03). There was no difference in median pediatric index of mortality (PIM3) in the two groups (Table 1).

Table 1. Demographics and process of care in pre- and post-implementation group

Variables	Pre (N = 292)	Post (N = 443)	p value
Age (in months), median (IQR)	13.5 (3.5–45.8)	9.4 (2.5–41.3)	0.04*
Male, N (%)	164 (56.16)	281 (63.43)	0.04**
Pediatric Index of Mortality (PIM) (IQR)	4.2 (1.1–4.2)	4.2 (1.1–4.2)	0.24*
Number of patients mechanically ventilated (%)	116 (45.67)	126 (37.17)	0.03**
Mechanical ventilation daysa (N, %)	603 (42.58)	669 (33.84)	<0.01**
Central line daysb (N, %)	443 (31.29)	500 (25.30)	<0.01**
Urinary catheter daysc (N, %)	433 (30.58)	481 (24.37)	<0.01**
Antimicrobial used (N, %)	1275 (89.98)	1841 (93.17)	<0.01**

^aTotal cumulative days of mechanical ventilation out of total observation days: pre = 1416, post = 1977.

^bTotal cumulative central line days out of total observation days: pre = 1416, post = 1976.

^cTotal cumulative urinary catheter days out of total observation days: pre = 1416, post = 1974.

^dAntimicrobial medication observation days: pre = 1417, post = 1976.

^*Mann–Whitney U-test.

^**Chi-square test.

There was a significant decrease in exposure to mechanical ventilation (42.6% in the pre-intervention phase to 33.8% mechanical ventilation days out of total observation days in the post-intervention phase, p < 0.01), central line (31.3–25.3%) and urinary catheter utilization (30.6–24.4%) in the post-intervention group; however, there was increased utilization of antimicrobials (89.9–93.2%, p < 0.01) (Table 1).

There was a variable impact of the check list on compliance with the best practice guidelines. Processes of care improved for some elements with the introduction of the checklist (daily oral care, head-of-bed elevation, pressure ulcer prevention, antimicrobial use discussions and gastric ulcer prophylaxis in eligible patients), worsened with some elements (extubation readiness testing, goal sedation discussions, DVT prophylaxis) and did not change for the rest (Table 2).

Table 2. Compliance with best care practices in pre- and post-implementation groups

Variables		Pre	Post	p value
Ventilator-associated pneumonia prevention bundle elements,aN (%)	Daily oral care	516 (85.5)	649 (97.0)	<0.01*
	Head of the bed elevation at 30°	419 (74.1)	646 (97.4)	<0.01*
	Assessment of spontaneous breathing trial	354 (67.0)	367 (67.8)	0.778*
	Assessment of extubation readiness	400 (74.4)	349 (63.8)	<0.01*
Central line-associated bloodstream infection prevention bundle elements,bN (%)	Daily assessment for central line removal	300 (67.7)	344 (68.8)	0.72*
	Central line dressing clean and intact	436 (98.4)	492 (98.4)	0.98*
Catheter-associated urinary tract infection prevention bundle elements,cN (%)	Daily assessment for urinary catheter removal	263 (60.7)	289 (60.0)	0.84*
Pressure ulcer prevention,dN (%)	Every 2 h turning of immobile patient	946 (80.4)	1217 (95.2)	<0.01*
Delirium prevention bundle element,eN (%)	Goal sedation discussion during rounds	447 (97.1)	531 (94.6)	0.04*
DVT prevention,fN (%)	Discussion for need for DVT prophylaxis in eligible patients	164 (33.7)	123 (27.3)	0.03*
Gastric ulcer prevention,gN (%)	Prophylaxis in eligible patients	320 (55.8)	410 (69.1)	<0.01*
Judicious antimicrobial use,hN (%)	Assessment to continue or discontinue antimicrobial	1104 (86.6%)	1661 (90.3%)	<0.01*

^aTotal ventilator days of 606 in pre- and 669 in post-intervention phase.

^bTotal central line days of 443 in pre- and 500 in post-intervention phase.

^cTotal urinary catheter days of 433 in pre- and 481 in post-intervention phase.

^dTotal eligible patients of 1417 in pre- and 1976 in post-intervention phase.

^eTotal eligible patients on continuous sedation of 462 in pre- and 461 in post-intervention phase.

^fTotal eligible patients of 486 in pre- and 449 in post-intervention phase.

^gTotal eligible patients of 572 in pre- and 592 in post-intervention phase.

^hTotal eligible patient days of 1275 in pre- and 1841 in post-intervention phase.

^*Chi-square test.

Median ICU length of stay fell from 9.4 (IQR 4–15.5) days to 7.3 (IQR 3.4–13.4) days, p < 0.01. There was, however, no significant difference in the hospital length of stay, ICU mortality, hospital mortality or 28-day mortality. There was a slight, increase in reporting of various healthcare-associated patient safety events including, ventilator associated events rates, catheter associated urinary tract infection rates and rates of pressure ulcers (Table 3).

Table 3. Patient outcomes in pre- and post-implementation groups

Variables	Pre	Post	p value
ICU length (days), median (IQR)	9.4 (4–15.5)	7.3 (3.4–13.4)	0.01**
Hospital length (days),a median (IQR)	15.4 (8.4–25)	12.6 (7.5–24.4)	0.055*
ICU mortality, N (%)	42 (14.4)	50 (11.3)	0.22**
Hospital mortality, N (%)	42 (14.4)	50 (11.3)	0.22**
28-day mortality,bN (%)	47 (17.7)	60 (17.6)	0.98**
Catheter-associated central line infection, N (%)	13 (2.93%)	6 (1.2%)	0.06**
Ventilator-associated events, N (%)	146 (24.2%)	194 (29%)	0.054**
Catheter-associated urinary tract infections, N (%)	5 (1.15%)	11 (2.29%)	0.19**
Pressure ulcer,cN (%)	41 (2.89%)	73 (3.69%)	0.2**

^aHospital length unknown for 1 patient in pre-group and 14 patients in post-group.

^bTwenty-eight-day outcome unknown for 23 patients in pre-group and 102 patients in post-group.

^cPressure ulcer observation days: pre = 1417 and post = 1976.

^*Mann–Whitney U-test.

^**Chi-square test.

DISCUSSION

In this multicentric study, we have shown an effective implementation of a PICU checklist on a global scale. While prior collaborative quality improvement work on ICU processes of care has been done, this study, to our knowledge, was the first to implement these principles in ICUs of LMIC with limited resources.

Two centers dropped out of data collection in the post-implementation phase. In our study, four of eight centers were from the China region and close to 50% of our patients in both phases were from China. Center recruitment (except for Nyankunde, DR Congo center) was limited to centers with a relatively advanced ICU environment (e.g. mechanical ventilation, central line availability) and also pre-existing research infrastructure. This limited our ability to recruit many other ‘interested’ centers. The study material was only available in English and Chinese, which also limited our ability of recruitment in non-English speaking countries.

The two comparison groups of pre- and post-intervention phase were not similar in age, gender and percentage of patients on mechanical ventilation. This is likely reflective of seasonal changes. Since younger patients typically have higher mortality, this may explain why we were not able to show a mortality difference in this rather large study. However, PIM3 score in the two groups was comparable. We also did not collect specific patient diagnosis data, as the best practices apply to all patients, limiting comparison between the two groups. There was decreased utilization of critical care in the post-intervention group which may be reflective of the impact of the study itself (early extubation and central line removal leading to a decreased number of mechanical ventilation days and central-line days) or differences in the cohort. A significant decrease in the number of days for central lines and urinary catheters reflect the current recommendations to discuss daily the need for invasive lines, or urinary catheter on a daily basis and remove those no longer required for patient care [18]. In spite of increased daily discussion on the use of antibiotics, the actual proportion of patients on antibiotics increased in the post-intervention group, possibly due to seasonal variation in the patient population. It is also possible that the checklist itself paradoxically may have enforced higher antibiotic usage.

In our study, the impact on compliance with best practice recommendation was variable. While compliance increased for elements like daily oral care and head of the bed elevation, it decreased for other best practice recommendations like an assessment for extubation readiness and goal sedation discussion. It is possible that these centers were already highly compliant with these elements either natively or due to Hawthorne effect [19] and the implementation of new processes caused ‘confusion’ among staff. We tried to account for this by including a ‘washout’ period of education between the two phases, which may have been inadequate for units of high acuity in high staff turnover. It is also possible that there may be a significant difference if the elements were combined into a ‘bundle’ rather than analyzing individual elements of care [20].

In this study, we used ICU length of stay as our primary indicator as we felt it is most significantly impacted by the ICU processes like early extubation, judicious sedation and prevention of hospital-associated conditions [21]. Although, ICU length of stay can trend downwards without any benefit to the patient, or even to the detriment of the patient. There was no impact on the various measures of mortality, although the study was not adequately powered to detect such a change. Ventilator-associated events were used for its unbiased surveillance approach, rather than the more accepted definition of ventilator-associated pneumonia, which we acknowledge is only beginning to be studied in children [22–24].

The development of an online program and a robust data gathering framework allowed us to collect information from 10 PICUs around the world with limited resources for a total of 735 patients, significantly contributing to our current knowledge in multi-institutional global quality improvement projects. The information obtained contributes to much-needed information to allocate resources in pediatric critical care settings. The methodology used in this particular study highlights the accessibility to communicate data and information to improve workflows within different cultures and follow general guidelines to improve pediatric care worldwide.

There are limitations to our study, including the difficulty in having adequate personnel dedicated to quality improvement projects. All the data collected were entered by physicians interested in improving their practices within their environment. Ideally, personnel conducting compliance audits must not be a part of clinical team. In resource limited centers it was difficult to adhere to this standard and it is possible that this may have introduced bias. Due to high patient volumes in many of these centers, patient recruitment was impacted. There was an imbalance between the two groups, which may have impacted both the leading (process variables) indicators and lagging (outcome variables) indicators of the study. Lastly, the before–after comparison over a reasonably short period in a quality improvement initiative leads to a variety of possible confounders, as discussed above. Improvement in a process should be demonstrated by a run or control chart [25]. However, these are very resource intensive for manual data collection and were not practical in busy PICUs in developing countries. Hospital setting and resources had significant disparity, which may make generalization in one sitting difficult. Future studies focusing on specific age groups or disease states (such as severe pneumonia) may yield more generalizable data.

CONCLUSIONS

We have demonstrated the effect of an education-based quality improvement process using a best practice checklist in developing countries. This simple intervention led to increasing adherence with guidelines in multiple ICU processes and may have impacted the length of stay. A global coalition of diverse intensive care units is needed to identify what specific process changes should be adopted worldwide that could impact patient safety and outcome.

FUNDING

Study was supported by grant support of the Research Committee, Mayo Clinic Critical Care Integrated Medical Practice (IMP) and Center for Clinical and Translational Science (CCaTS) small grants program to principal investigator. The CCaTS award is supported financially by the Center for Clinical and Translational Science (CTSA), which is funded by grant from National Institutes of Health (NIH). This publication is made possible by Mayo Clinic CTSA grant number UL1TR000135 from the National Center for Advancing Translational Science (NCATS), a component of the NIH.

Conflict of interest: R.K. and O.G. have a financial conflict of interest related to adult ICU checklist software applications licensed to Ambient Clinical Analytics Inc. The related research has been reviewed by the Mayo Clinic Conflict of Interest Review Board and is being conducted in compliance with Mayo Clinic Conflict of Interest policies. Other authors have declared no conflict of interest.