Abstract
Objective To test a multifaceted collaborative quality improvement intervention designed to promote evidence based surfactant treatment for preterm infants of 23-29 weeks' gestation.
Design Cluster randomised controlled trial
Setting and participants 114 neonatal intensive care units (which treated 6039 infants of 23-29 weeks gestation born in 2001).
Main outcome measures Process of care measures: proportion of infants receiving first surfactant in the delivery room, proportion receiving first surfactant more than two hours after birth, and median time from birth to first dose of surfactant. Clinical outcomes: death before discharge home, and pneumothorax.
Intervention Multifaceted collaborative quality improvement advice including audit and feedback, evidence reviews, an interactive training workshop, and ongoing faculty support via conference calls and email.
Results Compared with those in control hospitals, infants in intervention hospitals were more likely to receive surfactant in the delivery room (adjusted odds ratio 5.38 (95% confidence interval 2.84 to 10.20)), were less likely to receive the first dose more than two hours after birth (adjusted odds ratio 0.35 (0.24 to 0.53)), and received the first dose of surfactant sooner after birth (median of 21 minutes v 78 minutes, P < 0.001). The intervention effect on timing of surfactant was larger for infants born in the participating hospitals than for infants transferred to a participating hospital after birth. There were no significant differences in mortality or pneumothorax.
Conclusion A multifaceted intervention including audit and feedback, evidence reviews, quality improvement training, and follow up support changed the behaviour of health professionals and promoted evidence based practice.
Introduction
Health services continue to show major gaps between routine practice and what the research evidence suggests is optimal patient care.1 In neonatology, systematic reviews indicate that prophylactic surfactant treatment of high risk preterm infants reduces risk of death and pneumothorax by 40%, and that earlier treatment is more effective than later treatment.2,3 Despite this evidence, few such infants routinely receive prophylactic surfactant treatment, and many infants, particularly those born at outlying hospitals, receive delayed treatment.4
Various strategies for promoting behaviour change and evidence based practice have been proposed.5-8 Experience from the Vermont Oxford Network suggests that multidisciplinary collaborative quality improvement based on four key “habits” (change, evidence based practice, systems thinking, and collaborative learning) modifies practice in neonatal intensive care units, improves clinical outcomes, and reduces costs.9,10
We therefore conducted a cluster randomised controlled trial11 to test whether teams in neonatal intensive care units exposed to a multifaceted collaborative quality improvement intervention based on the four key habits would administer the first dose of surfactant sooner after birth, and achieve improved patient outcomes for preterm infants of 23-29 weeks' gestation.
Methods
Vermont Oxford Network
The Vermont Oxford Network is a voluntary collaboration of health professionals whose mission is to improve the quality and safety of medical care for newborn infants and their families (www.vtoxford.org). The network maintains a database for very low birthweight infants at member hospitals.12
Sample size
According to network data for 1998, 16.4% of very low birthweight infants at member hospitals received surfactant in the delivery room (standard deviation of individual hospital proportions 21%), the median time to surfactant administration was 105 minutes, and the infant mortality was 16.8% (standard deviation of individual hospital proportions 7.5%). We set a sample size of 55 hospitals in each of the two study arms before starting the trial to give 90% power (two sided α = 0.05) to detect an increase to 29.4% in the proportion of infants receiving surfactant in the delivery room (13% difference). This sample size also had 90% power to detect a 48 minute difference in median time to surfactant administration and 80% power to detect a decrease in mortality to 12.8% (4% difference). We expected a mortality difference of 4% if delivery room surfactant treatment increased by 60% and if relative risk of death for infants receiving this treatment was 0.62.
To conform to the cluster randomisation, power calculations accounted for both intrahospital and interhospital variability.13,14
Eligibility, enrolment, and randomisation
Of the 300 North American hospitals in the Vermont Oxford Network, 178 were eligible to enter the trial and 114 enrolled (fig 1). A secure computer program assigned enrolled hospitals to one of two study arms using a completely randomised design. Assignments were concealed from investigators except for JDH, who notified hospitals in writing on 24 May 2000. Other investigators learnt the identity of the intervention hospitals at the workshop (see below).
Fig 1.
Flow of participants through study. Of 300 North American hospitals in the Vermont Oxford Network, 178 had neonatal units eligible to participate and 114 of these enrolled in the study
Components of the multifaceted intervention
Audit and feedback—In July 2000 intervention hospitals received confidential, individualised feedback from the Vermont Oxford Network including site-specific information and peer comparisons related to the administration and timing of surfactant, and delivery room practice for infants of 23-29 weeks' gestation born in 1998 and 1999.
Workshop—Intervention hospitals were invited to attend a workshop held from 16 to 18 September 2000 (see details of workshop on bmj.com). Of the 57 teams invited, 56 attended (53 physicians, 46 nurses, and 21 respiratory therapists). The workshop included didactic sessions, facilitated site team exercises, and multi-institutional group exercises designed to promote four key “habits” (change, evidence based practice, systems thinking, and collaborative learning).9,15
Ongoing support—Collaboration among intervention arm teams was fostered through quarterly conference calls and an email discussion list.
Control hospitals received centre-specific, confidential reports routinely prepared for members of the Vermont Oxford Network.
Outcome measures
Primary outcome measures consisted of process of care measures (surfactant treatment in the delivery room, first surfactant treatment more than two hours after birth (among those receiving surfactant), and time after birth when first surfactant dose was administered) and infant outcome measures (death before discharge from hospital and pneumothorax).
Secondary outcome measures, identified before the trial started, included common morbidities2,3 and co-interventions for preterm infants (see table 5 for details).
Table 5.
Secondary study outcomes for 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001.* Values are percentages unless stated otherwise
|
Odds ratios (95% CI)
|
||||
|---|---|---|---|---|
| Outcome | Intervention hospitals (n=57) | Control hospitals (n=57) | Unadjusted | Adjusted† |
| Delivery room resuscitation: | ||||
| Oxygen | 95.9 | 95.9 | 0.99 (0.53 to 1.85) | 1.00 (0.47 to 2.15) |
| Bag or mask | 68.4 | 64.9 | 1.17 (0.83 to 1.66) | 1.23 (0.86 to 1.76) |
| Endotracheal intubation | 78.0 | 69.8 | 1.54 (1.16 to 2.03) | 1.65 (1.19 to 2.29) |
| Adrenaline (epinephrine) | 5.7 | 6.7 | 0.86 (0.60 to 1.22) | 0.94 (0.65 to 1.36) |
| Cardiac compression | 8.4 | 8.5 | 0.99 (0.72 to 1.36) | 1.03 (0.74 to 1.44) |
| Cardiopulmonary resuscitation | 89.3 | 88.3 | 1.11 (0.82 to 1.49) | 1.14 (0.83 to 1.57) |
| Apgar score ≤3 at 1 minute | 32.5 | 30.3 | 1.10 (0.95 to 1.29) | 1.20 (0.99 to 1.44) |
| Respiratory: | ||||
| Chronic lung disease at 36 weeks | 42.1 | 38.5 | 1.16 (0.89 to 1.52) | 1.16 (0.87 to 1.55) |
| Respiratory distress syndrome | 86.6 | 88.5 | 0.84 (0.55 to 1.27) | 0.76 (0.49 to 1.19) |
| Conventional ventilation | 86.6 | 86.0 | 1.05 (0.77 to 1.44) | 1.10 (0.78 to 1.56) |
| High frequency ventilation | 31.3 | 30.7 | 1.03 (0.73 to 1.46) | 1.08 (0.65 to 1.80) |
| Corticosteroids given for chronic lung disease | 24.9 | 26.7 | 0.91 (0.68 to 1.22) | 0.87 (0.63 to 1.20) |
| Surfactant given at any time | 85.1 | 79.5 | 1.47 (1.07 to 2.01) | 1.55 (1.08 to 2.23) |
| Infection: | ||||
| Late bacterial | 13.2 | 13.7 | 0.95 (0.73 to 1.25) | 0.96 (0.73 to 1.28) |
| Coagulase negative staphylococci | 17.5 | 16.4 | 1.08 (0.80 to 1.45) | 1.04 (0.77 to 1.41) |
| Fungal | 3.3 | 4.4 | 0.75 (0.51 to 1.09) | 0.86 (0.59 to 1.25) |
| Nosocomial | 26.9 | 25.6 | 1.07 (0.84 to 1.36) | 1.05 (0.83 to 1.33) |
| Neurological: | ||||
| Cystic periventricular leucomalacia | 4.5 | 5.5 | 0.82 (0.56 to 1.19) | 0.85 (0.58 to 1.26) |
| Intraventricular haemorrhage | 28.0 | 32.5 | 0.81 (0.66 to 0.99) | 0.80 (0.63 to 1.00) |
| Severe intraventricular haemorrhage | 10.1 | 14.2 | 0.68 (0.56 to 0.83) | 0.70 (0.56 to 0.87) |
| Other: | ||||
| Patent ductus arteriosus | 44.4 | 38.5 | 1.27 (0.99 to 1.63) | 1.27 (0.96 to 1.67) |
| Retinopathy of prematurity | 57.2 | 56.3 | 1.04 (0.79 to 1.38) | 0.97 (0.71 to 1.35) |
| Severe retinopathy of prematurity | 13.5 | 13.1 | 1.04 (0.78 to 1.38) | 1.04 (0.71 to 1.52) |
Outcomes reported by study arm and birth location for clusters. Each hospital is considered a cluster.
Covariates in adjusted model include gestational age, maternal ethnicity (black, Hispanic, white, or other), teaching hospital (yes or no), type of neonatal intensive care unit (A, B, or C), and average annual volume of very low birthweight infants.
Statistical methods
The baseline year for comparing the two study arms began 25 May 1999. We assessed outcomes for infants born in 2001 (1 January to 31 December) with gestational ages of 23 weeks 0 days to 29 weeks 6 days, with birth weights of 401 to 1500 g, and without major birth defects. We pre-specified conducting primary analyses among all infants, and separately among infants born in the participating hospitals (inborn) and among infants transferred to a participating hospital after birth (outborn).
Our analyses were on an intention to treat basis. We analysed primary and secondary dichotomous outcomes using logistic regression. Generalised estimating equations accounted for within-hospital correlation.16 We adjusted analyses for infant covariates (gestational age, birth location, ethnicity) and hospital covariates (type of neonatal intensive care unit, teaching status, and annual volume of infant patients).
We used a log rank test with adjustment for clustering to test differences in time to administration of surfactant. We used proportional hazards models, accounting for clustering, adjusted for infant and hospital covariates to calculate adjusted hazard ratios (ratios > 1 indicate earlier surfactant).16 We used weighted linear mixed models to analyse changes over time in surfactant treatment and timing of treatment.
We calculated intraclass correlation coefficients17 for binary outcomes using SAS Proc Genmod and for time to first dose using Proc Mixed. The only covariate was trial arm, and complete trial data were used.
The statistician was masked to study assignments for analysis of primary outcomes. SAS version 8.2 (SAS Institute, Cary, NC) and STATA version 7.0 (College Station, TX) were used. Study inferences were based on the adjusted analyses.
Results
Baseline comparability
Table 1 shows the baseline characteristics of the participating hospitals. The hospitals in the two study arms were generally similar in types of neonatal intensive care unit, teaching status, and annual volume of preterm infant admissions, although more of the neonatal intensive care units in the control arm were type A (restricted assisted ventilation or no major surgery available).
Table 1.
Baseline characteristics of 114 hospitals in the Vermont Oxford Network that participated in trial of quality improvement intervention for surfactant treatment of preterm infants. Values are medians (interquartile ranges) unless stated otherwise*
| Characteristics | Intervention hospitals (n=57) | Control hospitals (n=57) |
|---|---|---|
| Type of neonatal intensive care unit (No (%) of units)†: | ||
| Type A | 7 (12) | 14 (25) |
| Type B | 37 (65) | 29 (51) |
| Type C | 13 (23) | 14 (25) |
| No (%) of teaching hospitals | 26 (45) | 25 (44) |
| Annual No of VLBW infants treated in neonatal unit | 75 (53-106) | 71 (41-114) |
| Treatment practices for VLBW infants in neonatal units: | ||
| Given antenatal corticosteroids (%) | 78 (72-83) | 77 (65-83) |
| Delivered by caesarean section (%) | 59 (54-67) | 55 (50-64) |
| Apgar score ≤3 at 1 minute (%) | 33 (24-38) | 30 (23-36) |
| Given surfactant (%) | 81 (69-87) | 85 (74-91) |
| Given surfactant in delivery room (%)‡ | 9 (0-33) | 4 (0-17) |
| Given first dose of surfactant after 2 hours (%) | 21 (12-39) | 21 (11-33) |
| Time to first dose of surfactant (minutes)§ | 75 (34-175) | 76 (53-120) |
| Infant outcomes: | ||
| Mortality (%) | 17 (11-21) | 15 (10-22) |
| Pneumothorax (%) | 8 (5-13) | 6 (4-10) |
VLBW=very low birth weight.
Characteristics are reported by study arm and birth location for clusters. Each hospital is considered a cluster.
Type A=restriction in providing assisted ventilation or no major surgery available. Type B=no restrictions on assisted ventilation, all major surgery except neonatal cardiac surgery. Type C=no restrictions on assisted ventilation, all major surgery including neonatal cardiac surgery.
Data collection began in 2000.
Time to first dose for all infants, censored at one week.
Infant characteristics in the assessment year
Tables 2 and 3 show the characteristics of the infants in the 2001 assessment year. The proportion of white infants was higher at intervention hospitals.
Table 2.
Median (interquartile range) characteristics of infants treated in 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001*
|
All infants
|
Infants born at participating hospitals
|
Infants transferred to participating hospital after birth
|
||||
|---|---|---|---|---|---|---|
| Intervention hospitals (n=57) | Control hospitals (n=57) | Intervention hospitals (n=57) | Control hospitals (n=57) | Intervention hospitals (n=57) | Control hospitals (n=57) | |
| No of infants treated per neonatal unit | 45 (35-75) | 40 (25-68) | 40 (33-64) | 36 (22-56) | 6 (3-12) | 5 (2-12) |
| Birth weight (mean grams) | 942 (902-968) | 925 (896-973) | 937 (902-963) | 920 (889-960) | 963 (890-1063) | 989 (907-1085) |
| Gestational age (mean weeks) | 27 (26-27) | 27 (26-27) | 27 (26-27) | 27 (26-27) | 27 (26-27) | 27 (26-27) |
| Female (%) | 47 (41-51) | 45 (41-51) | 47 (41-51) | 47 (42-53) | 50 (33-63) | 40 (25-50) |
| Maternal ethnicity (%): | ||||||
| Black | 24 (8-40) | 29 (11-50) | 26 (9-40) | 29 (11-50) | 13 (0-50) | 15 (0-68) |
| Hispanic | 6 (2-17) | 6 (3-28) | 6 (2-15) | 7 (2-30) | 0 (0-20) | 0 (0-16) |
| White | 57 (43-70) | 44 (24-68) | 59 (43-72) | 44 (25-68) | 50 (25-86) | 50 (8-92) |
| Asian | 1 (0-4) | 0 (0-3) | 1 (0-4) | 0 (0-3) | 0 (0-0) | 0 (0-0) |
| Other | 0 (0-3) | 0 (0-4) | 0 (0-2) | 0 (0-2) | 0 (0-0) | 0 (0-0) |
| Small for gestational age (%) | 10 (6-12) | 9 (7-13) | 11 (6-13) | 10 (7-13) | 0 (0-2) | 0 (0-5) |
| Multiple birth (%) | 26 (20-35) | 24 (16-31) | 27 (21-33) | 25 (17-33) | 0 (0-24) | 0 (0-20) |
| Given antenatal corticosteroids (%) | 77 (68-85) | 77 (68-86) | 83 (73-89) | 84 (75-91) | 33 (20-50) | 40 (19-65) |
| Delivered by caesarean section (%) | 62 (54-70) | 58 (49-63) | 61 (54-71) | 59 (50-64) | 50 (23-71) | 50 (29-60) |
| Apgar score ≤3 at 1 minute (%) | 32 (25-39) | 30 (24-36) | 32 (24-39) | 29 (21-38) | 38 (25-50) | 32 (17-50) |
Characteristics reported by study arm and birth location for clusters. Each hospital is considered a cluster.
Table 3.
Characteristics of infants treated in 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001*
|
All infants
|
Infants born at participating hospitals
|
Infants transferred to participating hospital after birth
|
||||
|---|---|---|---|---|---|---|
| Intervention hospitals (n=57) | Control hospitals (n=57) | Intervention hospitals (n=57) | Control hospitals (n=57) | Intervention hospitals (n=57) | Control hospitals (n=57) | |
| Total No of infants treated in neonatal units | 3313 | 2726 | 2911 | 2314 | 402 | 412 |
| Mean (SD) birth weight (grams) | 936 (258) | 930 (259) | 932 (258) | 924 (262) | 967 (257) | 963 (238) |
| Mean (SD) gestational age (weeks) | 27 (2) | 27 (2) | 27 (2) | 27 (2) | 26 (2) | 27 (2) |
| Female (%) | 47 | 46 | 47 | 47 | 47 | 40 |
| Maternal ethnicity (%): | ||||||
| Black | 30 | 34 | 30 | 33 | 28 | 42 |
| Hispanic | 8 | 17 | 8 | 16 | 12 | 17 |
| White | 56 | 44 | 57 | 46 | 52 | 35 |
| Asian | 4 | 2 | 4 | 2 | 4 | 2 |
| Other | 2 | 3 | 2 | 3 | 4 | 4 |
| Small for gestational age (%) | 9 | 10 | 10 | 11 | 4 | 6 |
| Multiple birth (%) | 27 | 24 | 28 | 26 | 18 | 16 |
| Given antenatal corticosteroids (%) | 75 | 76 | 80 | 82 | 39 | 41 |
| Delivered by caesarean section (%) | 59 | 58 | 61 | 59 | 50 | 51 |
| Apgar score ≤3 at 1 minute (%) | 33 | 30 | 32 | 29 | 40 | 34 |
Characteristics reported by study arm and birth location without regard to cluster.
Primary outcomes
Delivery room surfactant treatment was significantly higher in the intervention than in the control arm for all infants (adjusted odds ratio 5.38 (95% confidence interval 2.84 to 10.20)), for the infants born at participating hospitals (inborn) (adjusted odds ratio 6.16 (3.03 to 12.54)), and for the infants transferred to participating hospitals after birth (outborn) (adjusted odds ratio 1.99 (1.12 to 3.53)) (table 4). First surfactant treatment more than two hours after birth was significantly lower in the intervention than in the control arm for all infants (0.35 (0.24 to 0.53)), inborn infants (0.29 (0.18 to 0.47)), and outborn infants (0.57 (0.37 to 0.89)). Infants in the intervention arm received surfactant significantly sooner after birth than did infants in the control arm (median 21 minutes (interquartile range 10-128) v 78 minutes (29-410), adjusted hazard ratio 1.57 (95% confidence interval 1.42 to 2.07)). For inborn infants, median times were 18 minutes and 75 minutes in the intervention and control arms (adjusted hazard ratio 1.83 (1.50 to 2.23); for outborn infants the median times were 74 and 103 minutes (adjusted hazard ratio 1.30 (1.00 to 1.69)). When restricted to infants who received surfactant, the median times to first dose for all infants were 15 and 52.5 minutes in the intervention and control arms (adjusted hazard ratio 1.75 (1.47 to 2.09)).
Table 4.
Dichotomous primary study outcomes for 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001.* Values are percentages unless stated otherwise
|
Odds ratios (95% CI)
|
||||
|---|---|---|---|---|
| Intervention hospitals (n=57) | Control hospitals (n=57) | Unadjusted | Adjusted† | |
| Surfactant given in delivery room: | ||||
| All infants | 54.7 | 18.2 | 5.41 (3.30 to 8.87) | 5.38 (2.84 to 10.20) |
| Inborn infants‡ | 58.0 | 18.4 | 6.15 (3.62 to 10.43) | 6.16 (3.03 to 12.54) |
| Outborn infants‡ | 26.9 | 17.2 | 1.77 (1.01 to 3.11) | 1.99 (1.12 to 3.53) |
| First dose of surfactant given after 2 hours: | ||||
| All infants | 9.4 | 24.9 | 0.31 (0.21 to 0.45) | 0.35 (0.24 to 0.53) |
| Inborn infants‡ | 7.4 | 23.4 | 0.26 (0.17 to 0.40) | 0.29 (0.18 to 0.47) |
| Outborn infants‡ | 26.4 | 36.0 | 0.64 (0.40 to 1.00) | 0.57 (0.37 to 0.89) |
| Infant mortality: | ||||
| All infants | 17.8 | 18.2 | 0.97 (0.81 to 1.16) | 1.01 (0.79 to 1.30 |
| Inborn infants‡ | 17.5 | 17.7 | 0.99 (0.82 to 1.19) | 1.08 (0.83 to 1.42) |
| Outborn infants‡ | 20.2 | 20.3 | 0.99 (0.68 to 1.45) | 0.74 (0.52 to 1.06) |
| Infant pneumothorax: | ||||
| All infants | 6.6 | 7.4 | 0.89 (0.68 to 1.17) | 0.89 (0.67 to 1.18) |
| Inborn infants‡ | 6.6 | 7.2 | 0.92 (0.69 to 1.22) | 0.92 (0.68 to 1.25) |
| Outborn infants‡ | 6.2 | 9.1 | 0.66 (0.39 to 1.13) | 0.58 (0.33 to 1.03) |
Outcomes reported by study arm and birth location for clusters. Each hospital is considered a cluster.
Covariates in adjusted model include gestational age, maternal ethnicity (black, Hispanic, white, or other), teaching hospital (yes or no), type of neonatal intensive care unit (A, B, or C), and average annual volume of very low birthweight infants.
Inborn infants are those born in the participating hospitals. Outborn infants are those born at another hospital and transferred to a participating hospital after birth.
There were no significant differences in mortality or pneumothorax. There was a trend towards decreased pneumothorax for outborn infants in the intervention arm (adjusted odds ratio 0.58 (0.33 to 1.03)).
There were significant interactions between treatment arm and location of birth for surfactant administration in the delivery room (P < 0.003) and for first surfactant treatment more than two hours after birth (P < 0.001) resulting from larger effect sizes for inborn infants than for outborn infants (table 4). There was a significant interaction between treatment arm and gestation for first surfactant treatment more than two hours after birth among inborn infants (P = 0.01) due to changes in the size, but not the direction, of the effect across gestational age. The differences between treatment arms for this measure were greatest at 25 weeks and smallest at 29 weeks.
The intraclass correlation coefficients were 0.31 for surfactant administration in the delivery room, 0.09 for first surfactant treatment more than two hours after birth, 0.25 for time to first surfactant dose, and 0.01 for both mortality and pneumothorax.
Secondary outcomes
Intervention arm infants were significantly more likely to be intubated in the delivery room (adjusted odds ratio 1.65 (1.19 to 2.29)) and to receive surfactant at any time (1.55 (1.08 to 2.23)) (table 5). The overall proportions of infants who were intubated and received conventional ventilation or high frequency ventilation were similar in the two arms (adjusted odds ratios 1.10 (0.78 to 1.56) and 1.08 (0.65 to 1.80) respectively). Severe intraventricular haemorrhage (grades 3 or 418) was significantly lower in the intervention arm (0.70 (0.56 to 0.87)); there was a trend towards a decrease for any intraventricular haemorrhage (grades 1 to 418) (0.80 (0.63 to 1.00)). There was a trend towards increased risk of patent ductus arteriosus in the intervention arm (1.27 (0.96 to 1.67)).
Changes in 2000 and 2001
We saw persistent reductions in the median time of the first surfactant dose and in interhospital variability for intervention hospitals starting in the last quarter of 2000, after the workshop (fig 2). The intervention hospitals showed significant changes in 2000 and 2001 for the proportion of all infants who received surfactant and time of first dose (both P < 0.001), but not the control hospitals (P = 0.81 and P = 0.18 respectively). The changes were significantly different between the two arms (time by treatment arm interaction, both P < 0.001).
Fig 2.
Median (interquartile range) time after birth at which first dose of surfactant was administered to preterm infants in neonatal intensive care units by calendar quarter. Units in intervention group were notified of their status in May 2000, were given individualised feedback in July 2000, and were invited to a quality improvement workshop in September 2000
Discussion
The multifaceted intervention tested in this trial produced significant improvement in the timing of surfactant use for very low birthweight infants in neonatal intensive care units. We observed improvements both for infants born at participating hospitals (inborn) and for those born outside and transferred after birth (outborn), although the effect was larger for the infants born inside (unsurprising given delays between birth and transfer and the involvement of non-study teams in providing delivery room care and initial stabilisation for outborn infants).
Comparison with other studies
Systematic reviews suggest that audit and feedback,5 interactive workshops,6 and multidisciplinary collaboration7 can modify professional practice and improve patient outcomes. A systematic review of 117 studies assessing implementation of practice guidelines, including 46 cluster randomised trials of multifaceted interventions, showed small to modest improvements in the processes of patient care.8 Eight studies that evaluated combinations of educational materials, educational meetings, and audit and feedback showed a median 3% improvement in target outcome (range 2.6% to 9.0%). The 37% increase we observed in delivery room surfactant treatment is large in comparison.
Patient outcomes
On the basis of results from systematic reviews,2,3 we expected that if our intervention resulted in earlier surfactant treatment it would also lead to reductions in death and pneumothorax. However, we observed no significant reductions in either death or pneumothorax. There are several possible explanations. Firstly, our wide confidence intervals for death and pneumothorax are compatible with the effects observed in the systematic reviews. Secondly, our trial was not designed to test the efficacy of prophylactic or early surfactant treatment, since infants were not randomly assigned to these treatment options. Thirdly, our study may have had insufficient power to detect an effect on mortality. We had 80% power to detect a 4% difference in mortality, assuming a 60% increase in surfactant administration in the delivery room and a relative risk for death of 0.6. However, we observed only a 37% increase in delivery room surfactant administration. Furthermore, a relative risk of death of 0.6 for infants receiving delivery room surfactant, although based on the systematic reviews, may have been too large an expected effect: antenatal corticosteroid therapy, which reduces the risk of death of preterm infants, has increased and time after birth to first dose of surfactant has decreased since the original trials were performed.4,19 These changes may reduce the additional benefit expected from prophylactic surfactant.
The intervention resulted in significantly higher rates of endotracheal intubation in the delivery room. This was an anticipated trade off for earlier surfactant treatment, since infants must be intubated to receive surfactant. Importantly, the intervention did not increase the overall intubation rate since the rates of conventional and high frequency ventilation were similar in the two study arms.
Earlier surfactant treatment has not been shown to reduce the risk for intraventricular haemorrhage or to increase the risk of patent ductus arteriosus.2,3 The decreased risk of severe intraventricular haemorrhage in the intervention arm in our trial might have resulted from changes in the composition and performance of delivery room resuscitation and stabilisation teams or other unmeasured changes in care, suggesting subjects for future study. The trends towards decreased risk of pneumothorax in infants transferred to the hospital after birth and increased risk for patent ductus arteriosus should be interpreted with caution.
A test of collaborative improvement
Collaborative improvement initiatives have been applied in a variety of healthcare settings.15,20 Typical elements of the initiatives include audit and feedback, self assessment, multidisciplinary teams, quality improvement training, evidence review, setting aims, and establishing social networks for information sharing and peer accountability.
Systematic reviews show that audit and feedback can improve professional practice, but effects are generally modest.5,8 The minimal feedback in routine reports provided by the Vermont Oxford Network to our control hospitals did not result in changes of timing for surfactant treatment, suggesting that feedback alone is insufficient to result in the large practice changes we observed.
We designed the two day workshop for the intervention hospitals based on four key habits for clinical improvement to be multidisciplinary and highly interactive (see bmj.com). Hospital teams learnt the habit for change involving the PDSA (plan, do, study, act) improvement model and left the workshop having identified their own measurable improvement aims and having designed several initial PDSA improvement cycles.21
During the workshop, the teams applied the habit for evidence based practice,22 reviewing evidence from systematic reviews, reflecting on their own practices, and assessing trade-offs for different treatment strategies. The intervention did not promote a specific practice guideline. After the workshop, the teams reviewed and modified their aims involving the staff back home. The literature on innovation suggests that personal reflection by clinical leaders, resulting in self imposed goals, is essential in obtaining the commitment to change.23 The workshop encouraged participants to apply systems thinking and look beyond the clinical aims to underlying structures, patterns, and processes within their hospitals. Social networking is an important contributor to the success of collaborative initiatives.20,23 Workshop exercises, discussion periods, the opening dinner, conference calls, and the email discussion list were designed to promote collaborative learning within and among teams.
Limitations of this study
Our study was not designed to determine the effects of individual components of the multifaceted intervention, or its cost. Although only 114 of the 178 eligible hospitals enrolled in the study, enrolled and non-enrolled hospitals had similar characteristics, and our findings are probably generalisable to other neonatal intensive care units. However, we cannot predict the generalisability of our findings to other clinical settings involving different target practices and evidence, and we cannot predict whether a different workshop faculty would achieve similar results. Our trial is unlikely to suffer from the problems reported in a recent review of cluster trials since randomisation was secure and blinded, and there was no attrition of clusters.24
Conclusion
A multifaceted collaborative improvement intervention—including audit and feedback, evidence reviews, quality improvement training based on four key habits, and follow up support—changed the behaviour of neonatologists and promoted evidence based practice.
What is already known on this topic
The quality improvement collaborative is a promising intervention to encourage evidence based practices and has been used in a variety of healthcare settings
What this study adds
A multifaceted collaborative quality improvement intervention based on four key habits was used to promote evidence based surfactant therapy for preterm infants
It produced significant changes in the timing of surfactant use and a 37% increase in surfactant treatment in the delivery room
It did not result in significant changes in mortality or pneumothorax, however, but the study's power to detect these changes may have been low
Supplementary Material
Participants in the Vermont Oxford Network, and details of the intervention workshop appear on bmj.com
Contributors: All authors contributed to the planning, conduct, and reporting of this study. JDH is guarantor for the study.
Funding: Funded by a grant from the Agency for Healthcare Research and Quality (R01 HS10528, Horbar JD, principal investigator). The authors' work was independent of the funding agency.
Competing interests: JDH is chief executive and scientific officer, RFS is a director, and JHC is an employee of the Vermont Oxford Network. JB received funding for research from the Vermont Oxford Network. RFS has acted as a consultant and spoken at meetings sponsored by various manufacturers of surfactant products, including Ross Laboratory Division of Abbott Laboratories and Chiesi Pharmaceuticals. RFS has also received grant funding from Ross Laboratories to conduct research regarding surfactant therapy. PEP has served as a paid consultant to the Vermont Oxford Network and other clients who sponsor improvement collaboratives. GS has served as a paid consultant to Dey Laboratories, a manufacturer of surfactant.
Ethical approval: The study was approved by the University of Vermont Institutional Review Board.
References
- 1.Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington DC: National Academy Press, 2001.
- 2.Soll RF, Morley CJ. Prophylactic versus selective use of surfactant in preventing morbidity and mortality in preterm infants. Cochrane Database Syst Rev 2001;(2): CD000510. [DOI] [PubMed]
- 3.Yost CC, Soll RF. Early versus delayed selective surfactant treatment for neonatal respiratory distress syndrome. Cochrane Database Syst Rev 2000;(2): CD001456. [DOI] [PubMed]
- 4.Horbar JD, Carpenter J, Buzas J, Soll RF, Suresh G, Bracken MB, et al. Timing of initial surfactant treatment for infants 23 to 29 weeks gestation: is routine practice evidence based? Pediatrics. 2004;113: 1593-602. [DOI] [PubMed] [Google Scholar]
- 5.Jamtvedt G, Young JM, Kristoffersen DT, Thomson O'Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2003;(3): CD000259. [DOI] [PubMed]
- 6.Thomson O'Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2001;(2): CD003030. [DOI] [PubMed]
- 7.Zwarenstein M, Bryant W. Interventions to promote collaboration between nurses and doctors. Cochrane Database Syst Rev 2000;(2): CD000072. [DOI] [PubMed]
- 8.Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8(6): iii-iv, 1-72. [DOI] [PubMed] [Google Scholar]
- 9.Horbar JD, Plsek P, Leahy K. NIC/Q 2000: establishing habits for improvement in neonatal intensive care units Pediatrics 2003;111: e397-410. [PubMed] [Google Scholar]
- 10.Horbar JD, Rogowski J, Plsek PE, Delmore P, Edwards WH, Hocker J, et al. Collaborative quality improvement for neonatal intensive care. Pediatrics 2001;107: 14-22. [DOI] [PubMed] [Google Scholar]
- 11.Campbell MK, Elbourne DR, Altman DG for the CONSORT Group. CONSORT statement: extension to cluster randomized trials. BMJ 2004;328: 702-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Vermont Oxford Network. Database manual of operations. Release 4.0. Burlington, VT: Vermont Oxford Network, 1999.
- 13.Lee E, Dubin N. Estimation and sample size considerations for clustered binary responses. Stat Med 1994;13: 1241-52. [DOI] [PubMed] [Google Scholar]
- 14.Collett D. Modelling survival data in medical research. 2nd ed. London: Chapman and Hall, 2003.
- 15.Plsek PE. Quality improvement methods in clinical medicine. Pediatrics. 1999;103: 203-14. [PubMed] [Google Scholar]
- 16.Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold, 2000.
- 17.Campbell MK, Grimshaw JM, Elbourne DR. Intracluster correlation coefficients in cluster randomized trials: empirical insights into how they should be reported. BMC Med Res Methodol 2004;4: 9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Papile LA, Burstein J, Burstein R, Koffler H. Incidence and evolution of subependymal and intraventricular hemorrhage: a study of infants with birth weights less than 1500 gm. J Pediatr. 1978;92: 529-34. [DOI] [PubMed] [Google Scholar]
- 19.Horbar JD, Badger GJ, Carpenter JH, Fanaroff AA, LaCorte M, Phibbs R, et al for the members of the Vermont Oxford Network. Trends in mortality and morbidity for very low birth weight (VLBW) infants, 1991-1999. Pediatrics 2002;110: 143-51. [DOI] [PubMed] [Google Scholar]
- 20.Wilson T, Berwick DM, Cleary PD. What do improvement collaboratives do? Experience from seven countries. Jt Comm J Qual Saf 2003;29: 85-93. [DOI] [PubMed] [Google Scholar]
- 21.Langley GJ, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide. A practical approach to enhancing organizational performance. San Francisco CA: Jossey-Bass, 1996.
- 22.Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine. How to practice and teach EBM. 2nd ed. Edinburgh: Churchill Livingstone, 2000.
- 23.Berwick DM. Disseminating innovations in health care. JAMA 2003;289: 1969-75. [DOI] [PubMed] [Google Scholar]
- 24.Puffer S, Torgerson DJ, Watson J. Evidence for risk of bias in cluster randomized trials: review of recent trials published in three general medical journals. BMJ 2003;327;785-9. [DOI] [PMC free article] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.