Can differences in obstetric outcomes be explained by differences in the care provided? The MFMU Network APEX Study

Abstract

Objective

To determine whether hospital differences in the frequency of adverse obstetric outcomes are related to differences in care.

Study Design

The Assessment of Perinatal EXcellence (APEX) cohort of 115,502 women and their neonates born in 25 hospitals in the United States between March 2008 and February 2011. Hierarchical logistic regression was used to quantify the amount of variation in postpartum hemorrhage, peripartum infection, severe perineal laceration, and a composite adverse neonatal outcome among hospitals that is explained by differences in patient characteristics, hospital characteristics, and the obstetric care provided.

Results

115,502 women were included in the study. For most outcomes, between 20 and 40% of hospital differences in outcomes were related to differences in patient populations. After controlling for patient-, provider- and hospital-level factors, multiple care processes were associated with the predefined adverse outcomes, but these care processes did not explain significant variation in the frequency of adverse outcomes among hospitals. Ultimately, between 50 and 100% of the inter-hospital variation in outcomes was unexplained.

Conclusion

Hospital differences in the frequency of adverse obstetric outcomes could not be explained by differences in frequency of types of care provided.

Obstetric admissions are a leading cause of hospitalization in the United States. Accordingly, there has been an increasing demand for quality measurement from multiple stakeholders. Quality measures typically take two forms – outcome measures, such as frequency of peripartum infection, which reflect the actual outcomes that patients have, and process measures, such as frequency of episiotomy, which reflect adherence to, or avoidance of, a given type of care.^1,2

However, several uncertainties remain about obstetric outcome and process measures and their ability to represent quality care. There is controversy whether, and to what extent, hospital differences in outcomes are actually due to differences in the characteristics of their patient population; correspondingly, case-mix adjustment has been used inconsistently.^3,4 Also, there is often an implicit assumption that those hospitals that perform best on process measures will have the best outcomes as well.⁵ Yet this assumption has not been proven in obstetrics.

In fact, there are several potential contributors to the frequency of adverse outcomes, including patient characteristics (such as maternal age), hospital characteristics (such as the types of obstetric providers or continual availability of interventional radiology), and the types of care that are provided (such as the frequency of cesarean delivery). Although poorly understood, the extent to which each of these categories explains hospital differences in outcomes is important in determining the adequacy of quality measures. For example, if all variation in an outcome were due to differences in patient populations, it would make little sense to use that outcome to represent a hospital’s quality. On the other hand, if much of the variation in an outcome were not due to differences in patient populations, but differences in a particular process of care, the use of both specific outcome and process measures would be better supported.

The specific aim of the present study was to assess whether, and to what extent, hospital differences in the frequency of adverse obstetric outcomes are related to patient and hospital characteristics, and to types of care provided.

METHODS

Study Design

The Assessment of Perinatal EXcellence (APEX) study is an observational study designed to assist in the development of quality measures for intrapartum obstetrical care. This study was approved by the Institutional Review Board at each participating institution under a waiver of informed consent. Full details of the study design have been previously published.⁶

In summary, patients eligible for data collection were those who delivered on randomly selected days between March 2008 and February 2011 at any of the 25 hospitals in the Eunice Kennedy Shriver National Institute of Child Health and Human Development Maternal-Fetal Medicine Units (MFMU) Network, were at least 23 weeks of gestation, and had arrived at the hospital with a live fetus. Days were chosen via computer-generated random selection, with enrollment from larger hospitals limited in order to avoid overrepresentation of patients from these hospitals. The medical records of all eligible women and their neonates were abstracted by trained and certified research personnel at the clinical centers. Patient data that were recorded included demographic characteristics (including, in order to assess the diversity of the cohort, race and ethnicity as reported in the chart), details of the medical and obstetrical history, types of intrapartum and postpartum care, and obstetric outcomes. In addition, characteristics of the providers who cared for the patients and the hospitals in which they delivered were collected. Maternal data were collected until discharge and neonatal data were collected until discharge or until 120 days of age, whichever came first.

Outcomes

The five a priori primary outcomes were 1) venous thromboembolism, 2) postpartum hemorrhage (PPH), 3) peripartum infection, 4) severe perineal laceration, restricted to women with vaginal singleton deliveries with no shoulder dystocia, and stratified by spontaneous (SVD), forceps (FVD) and vacuum (VVD) vaginal delivery, and 5) a composite neonatal adverse outcome, restricted to term (≥ 37 weeks of gestation), non-anomalous singleton infants. Additional details regarding the definitions of these outcomes are detailed elsewhere.⁶

Statistical Analysis

Sample size for the APEX cohort was based on thromboembolism in cesarean deliveries, which was expected to have the lowest frequency (0.175% overall and 0.550% in cesarean deliveries) of the five a priori primary outcomes, using techniques that consider the cluster design.^7,8,9 Assumptions included: 2-sided type I error = 0.01 and the proportion of deliveries without an associated process measure = 25%. The sample size estimate was based on 30,000 cesarean deliveries. Conservatively assuming a cesarean frequency of 25%, a total sample size of 120,000 would enable the detection of an odds ratio of 2.75 for the association between a process measure and outcome with at least 80% power for the outcome of thromboembolism. Assuming an odds ratio of 1.5, and event frequencies ranging from 2.4% to 8.0% for the remaining four outcomes (PPH, peripartum infection, severe perineal laceration in vaginal deliveries, and the composite neonatal adverse outcome in term non-anomalous singletons), power was estimated to range from 83% to 99%; power was > 99% for these four outcomes assuming an odds ratio of 2.0. Due to fewer than expected thromboembolism events (0.03% overall) this outcome was not further evaluated.⁶

For each of the adverse obstetric outcomes, hierarchical logistic regression with hospital random effects was used to quantify the amount of variation in outcomes among hospitals that is due to: 1) patient characteristics, 2) provider and hospital characteristics, and 3) the types of care provided (process measures). The initial regression equation included only the hospitals as random-effect terms. In each successive stage of the model, another level of variables – i.e., the patient characteristics, hospital characteristics, or care characteristics – were added as fixed effects. Per the methods used by Synnes et al,¹⁰ each equation contained a random effects term (b_o), and it is the standard deviation (σ) of this term that serves to quantify the overall variation in outcome frequency across the hospitals. The difference in the value of σ as each set of characteristics is added to the model then quantifies the amount of variation between hospitals explained by the additional characteristics. Odds ratios and 99% confidence intervals (CIs) for each hospital, using the hospital with the median observed outcome frequency as the referent, were also obtained from these hierarchical models.

Patient, provider, hospital, and care characteristics eligible for multivariable models were selected a priori for each outcome based on a plausible association with the outcome (i.e., face validity). Details regarding the methods and results for selection of the patient characteristics has been reported previously.⁶ The provider and hospital characteristics eligible for multivariable models included: the specialty of the attending provider, years since the attending provider graduated from medical/midwifery school, nurse-to-patient ratio during the shift that delivery occurred, a hospital’s annual delivery volume (expressed in quartiles), the existence of a prenatal electronic medical record, the occurrence of a structured review of laboring patients attended by both nursing staff and attending providers, and the availability of a 24-hour anesthesia service dedicated to the labor and delivery unit. The presence of a 24-hour in-house attending obstetric provider, a 24-hour in-house neonatologist or pediatrician, and a 24-hour in-house interventional radiology service also were evaluated. For each outcome, after the patient characteristics that were previously selected for risk-adjustment were forced into the model,⁶ a backwards selection method was utilized with a P<0.05 to determine which provider and hospital characteristics were to remain in the regression for each outcome.

After a model that included patient, provider, and hospital characteristics was established, we examined which types of care (i.e., process measures) provided, selected a priori, were associated with each outcome. Eligible process measures included: elective delivery prior to 39 weeks of gestation without documented lung maturity, cervical dilation at admission among women in spontaneous labor, labor induction, proportion of labor with oxytocin augmentation, maximum dose of oxytocin, duration (minutes) of active stage (5 cm to 10 cm, or 5 cm to cesarean delivery), vaginal exams per hour in the first stage of labor; duration (minutes) from complete dilation (10 cm) to start of pushing, duration (minutes) from start of pushing to delivery, vaginal delivery, episiotomy, and type of anesthesia (epidural/regional or general). The process measures were individually added to patient and hospital characteristics-adjusted models that were restricted to women eligible for the type of care being assessed (e.g., labor induction was not assessed among women with a placenta previa, as women with this diagnosis would not be eligible to receive induction). In order to facilitate interpretation, process measures that were initially explored as continuous variables were dichotomized for use in the final regression model based on clinical relevance and assessment of plots using a locally weighted scatterplot smoothing technique (LOESS). Process measures significantly associated with a greater frequency of an adverse obstetric outcome were identified and used to derive a composite process measure “exposure score” which was calculated, per the methods by Peterson et al,¹¹ as the proportion of the care processes that a patient was eligible to receive that were actually received by the patient. Thus, if a patient received 3 of the 4 care processes significantly associated with the outcome of interest, her composite exposure was 75%.

SAS software (SAS Institute, Cary, NC), was used for the analyses. All tests were two-tailed. P<0.01 was used to define statistical significance and 99% CIs were estimated when directly testing a hypothesis (i.e., examining the association between the process measures and outcomes) and to identify hospital outliers. P<0.05 and 95% CIs were estimated for model building and other descriptive analyses.

RESULTS

During the study period, data were collected on 115,502 women and their neonates, as well as on 1797 different delivery attending providers at 25 hospitals. Characteristics of these patients, and their providers and hospitals, are provided in Tables 1 and 2. As shown, women were delivered by a variety of types of providers, and these providers had a range of experience. Hospital characteristics, including availability of medical services (e.g., obstetric anesthesia), the presence of electronic medical records, and the attendance of providers at structured obstetric patient review, varied as well.

Table 1.

Maternal (n = 115,502) and neonatal (n = 118,422) characteristics of the study population

		No. (%)
MATERNAL CHARACTERISTICS
Age, y
	< 20	10,187 (8.8)
	20–24.9	24,299 (21.0)
	25–29.9	31,101 (26.9)
	30–34.9	30,570 (26.5)
	≥ 35	19,345 (16.8)
Race/ethnicitya
	Non-Hispanic white	52,040 (45.1)
	Non-Hispanic black	23,878 (20.7)
	Non-Hispanic Asian	5999 (5.2)
	Hispanic	27,291 (23.6)
	Other	5083 (4.4)
	Not documented	1211 (1.1)
Body mass index at delivery,b kg/m2
	< 25	14,242 (12.6)
	25–29.9	41,268 (36.5)
	30–34.9	32,088 (28.4)
	35–39.9	15,088 (13.3)
	≥ 40	10,481 (9.3)
Cigarette use during pregnancy		11,370 (9.9)
Cocaine or methamphetamine use during pregnancy		830 (0.7)
Insurance status
	Uninsured/self-pay	11,989 (10.5)
	Government-assisted	45,125 (39.4)
	Private	57,462 (50.2)
Prenatal careb		107,510 (97.9)
Obstetric history
	Nulliparous	46,773 (40.5)
	Prior vaginal delivery only	49,865 (43.2)
	Prior cesarean only	8872 (7.7)
	Prior cesarean and vaginal	9963 (8.6)
Any hypertension		13,272 (11.5)
Diabetes mellitus
	None	106,706 (92.4)
	Gestational	6999 (6.1)
	Pregestational	1734 (1.5)
Anticoagulant use during pregnancy		920 (0.8)
Multiple gestation		2815 (2.4)
Polyhydramnios		940 (0.8)
Oligohydramnios		4700 (4.1)
Placenta previa		467 (0.4)
Placenta accreta		158 (0.1)
Placental abruption		930 (0.8)
PROM/PPROMb		6004 (5.3)
GBS status
	Negative	68,918 (59.7)
	Positive	24,390 (21.1)
	Unknown	22,194 (19.2)
NEONATAL CHARACTERISTICS
Presentation at delivery
	Vertex	111,174 (94.1)
	Breech	6010 (5.1)
	Nonbreech malpresentation	931 (0.8)
Gestational age at delivery, wk
	230–276	1256 (1.1)
	280–336	4282 (3.6)
	340–366	10,024 (8.5)
	370–376	10,914 (9.2)
	380–386	20,723 (17.5)
	390–396	37,695 (31.8)
	400–406	23,876 (20.2)
	410–416	8998 (7.6)
	≥ 420	654 (0.6)
Birthweight, g
	< 2500	12,498 (10.6)
	2500–3999	96,708 (81.7)
	≥ 4000	9186 (7.8)
Size for gestational age
	Small	11,530 (9.7)
	Appropriate	97,774 (82.6)
	Large	9088 (7.7)

Abbreviations: PROM/PPROM = premature rupture of the membranes or preterm premature rupture of the membranes; GBS = group B streptococcus.

^aRace/ethnicity was reported in the chart;

^bN = 113,167 with body mass index data; N = 109,773 with prenatal care visit data; N = 113,446 with PROM/PPROM data.

Table 2.

Characteristics of the study population’s attending providers and hospitals

		No. (%)
Specialty of attending at delivery
	General obstetrics and gynecology	84,057 (72.8)
	Midwife	7808 (6.8)
	Family medicine	3728 (3.2)
	Maternal-fetal medicine	18,954 (16.4)
	No attending at delivery	859 (0.7)
Years since attending at delivery graduated medical or midwifery school
	0–9.9 (includes no attending at delivery)	26,717 (23.4)
	10–14.9	21,793 (19.1)
	15–20.9	19,880 (17.4)
	20–24.9	16,248 (14.2)
	25+	29,428 (25.8)
Nurse-to-patient ratio at deliverya
	< 1	31,781 (27.6)
	1 – 1.9	58,263 (50.7)
	2 – 2.9	15,804 (13.7)
	3+	9160 (8.0)
Patient delivered at hospital where prenatal electronic medical record available
	No	47,727 (41.3)
	Sometimes	35,083 (30.4)
	Yes	32,692 (28.3)
Patient delivered at hospital with 24-hour in-house obstetric anesthesia service
	No	13,150 (11.4)
	Yes	102,352 (88.6)
Patient delivered at hospital with 24-hour in-house attending obstetric provider
	No	13,823 (12.0)
	Yes	101,679 (88.0)
Patient delivered at hospital with attending providers and/or nurses present for structured obstetric patient reviewb
	No obstetricians present at review	21,106 (18.3)
	Obstetricians but no nurses present at review	38,052 (32.9)
	Both obstetricians and nurses present at review	56,344 (48.8)
Patient delivered at hospital with 24-hour in-house interventional radiology available
	No	79,452 (68.8)
	Yes	36,050 (31.2)
Patient delivered at hospital with 24-hour in-house attending neonatologist or pediatrician
	No neonatologist, no pediatrician	12,532 (10.9)
	Pediatrician, no neonatologist	4363 (3.8)
	Neonatologist	98,314 (85.3)

^aTotal number of nursing hours worked in L&D during the 8-hour shift divided by 8, divided by the numbe of patient admissions during the 8-hour shift;

^bOfficial board sign-out at shift change or other structured patient review.

The frequencies of the selected outcomes were as follows: PPH 2.29% (95% CI 2.20% – 2.38%), peripartum infection 5.06% (95% CI 4.93% – 5.19%), severe perineal laceration at SVD 2.16% (95% CI 2.06% – 2.27%), severe perineal laceration at FVD 27.56% (95% CI 25.54% – 29.57%), severe perineal laceration at VVD 14.51% (95% CI 13.34% – 15.67%), composite neonatal adverse outcome 2.73% (95% CI 2.63% – 2.84%).⁶ As previously reported, the frequency of the selected adverse outcomes varied widely and differed significantly among hospitals (P<0.001 for all).⁶ The type of care experienced by patients at different hospitals varied widely as well (Table 3).The frequency of labor induction among women who were eligible for such an intervention, for example, ranged among hospitals from 21% to 37%. Oxytocin at rates greater than 20 mU/minute was rarely administered to laboring women at some hospitals, but this practice occurred in nearly 50% of women who received oxytocin at other hospitals. There was a more than twenty-fold difference in the frequency of delayed pushing among women who reached the second stage, and a difference in the frequency of vaginal delivery that ranged from 61% to 80%. Delivery practices varied as well, with a 50-fold difference in the frequency of episiotomy among women who had a vaginal delivery and more than a ten-fold difference in the use of general anesthesia at cesarean delivery.

Table 3.

Observed hospital frequencies of types of obstetric care

	Lowest Percent	Median Percent	Highest Percent
Labor inductiona	20.8	28.2	37.1
Dilation ≤ 2 cm at admissionb	6.6	13.6	25.9
Maximum oxytocin ≥ 20 mU/minutec	8.7	17.6	46.3
≥ 80% of labor augmented with oxytocind	1.0	10.1	22.6
≥ 1 hour between complete dilation and initiation of pushinge	0.8	10.9	21.2
≥ 2 hours between initiation of pushing to deliverye	4.4	9.1	19.2
≥ 8 hours active phasef	2.9	8.3	19.2
< 1 vaginal exam per every 3 hours in first stageg	2.9	21.0	43.7
Vaginal deliveryh	60.6	70.1	79.5
Episiotomyi	0.7	7.0	35.4
Epidural/regional anesthesiaj	45.3	77.7	89.7
General anesthesiak	1.1	6.5	14.8
Elective delivery < 39 weeks without documented fetal lung maturityl	0.2	0.5	12.2

^aIn patients with no previa and no history of classical, T, or J cesarean (N = 113,049);

^bIn patients at term with intact membranes and spontaneous intended labor with no previa and cervical dilation measured within one hour before or after L&D admission (N = 46,068);

^cIn patients who received oxytocin in labor (N = 58,228);

^dIn patients with spontaneous intended labor admitted to L&D before delivery (N = 61,157);

^eIn patients who reached complete after intended labor (N = 60,290);

^fIn patients with intended labor who reached active stage (5 cm) with a term non-anomalous singleton pregnancy (N = 71,571);

^gIn patients with intended labor managed in hospital for greater than 1 hour during first stage (N = 81,826);

^hIn all patients (N = 115,502);

ⁱIn patients with a vaginal delivery and no shoulder dystocia (N = 77,071);

^jIn patients with non-operative vaginal delivery of a singleton, no shoulder dystocia and reached complete after intended labor (N = 70,362);

^kIn patients with a cesarean delivery (N = 36,201);

^lIn patients with a term non-anomalous singleton pregnancy (N = 98,509).

Presented in Table 4 are associations of processes measures (individual and composite exposure score) with the studied outcomes. Even after controlling for patient, provider and hospital characteristics, particular types of obstetric care remained associated with the outcomes of interest.

Table 4.

Adjusted odds ratios (99%CIs) between the types of obstetric care and adverse obstetric outcomes

Process Measure		Postpartum hemorrhagea	Peripartum infectionb	Severe perineal laceration at SVDcd	Severe perineal laceration at FVDce	Severe perineal laceration at VVDcd	Composite neonatal adverse outcomefg
N		105,987	110,205	68,144	1898	3515	89,279
Labor induction		1.20 (1.04–1.37)	1.22 (1.13–1.33)	1.04 (0.90–1.21)	1.05 (0.78–1.42)	0.92 (0.70–1.21)	1.18 (1.05–1.34)
Dilation ≤ 2 cm at admission		h	1.58 (1.37–1.82)	h	h	h	h
Maximum oxytocin ≥ 20 mU/minute		1.61 (1.33–1.95)	1.30 (1.16–1.44)	h	h	h	h
≥ 80% of labor augmented with oxytocin		1.08 (0.78–1.50)	1.63 (1.42 –1.87)	h	h	h	h
≥ 1 hour between complete dilation and initiation of pushing		1.67 (1.22–2.28)	h	1.29 (1.04–1.59)	1.10 (0.74–1.64)	0.94 (0.65–1.34)	1.13 (0.89–1.45)
≥ 2 hours between initiation of pushing to delivery		4.02 (3.10–5.23)	h	1.88 (1.51–2.34)	1.21 (0.87–1.69)	1.55 (1.15–2.09)	1.83 (1.46–2.28)
≥ 8 hours active stage		h	h	h	h	h	1.32 (1.08–1.62)
< 1 vaginal exam per every 3 hours in first stage		h	1.18 (1.07–1.30)	h	h	h	1.18 (1.01–1.38)
Vaginal delivery		0.19 (0.16–0.22)	0.52 (0.47–0.56)	h	h	h	0.72 (0.63–0.83)
Episiotomy		h	1.22 (1.04–1.43)	2.47 (2.08–2.93)	1.24 (0.87–1.79)	1.99 (1.51–2.62)	h
Epidural/regional anesthesia		h	h	0.88 (0.73–1.06)	(small N precludes analysis)	0.90 (0.57–1.45)	h
General anesthesia		3.61 (2.98–4.37)	h	h	h	h	h
Elective delivery < 39 weeks without documented fetal lung maturity		h	h	h	h	h	1.39 (0.67–2.89)
Composite process measure exposure score (percent of care associated with fewer adverse outcomes that was received; referent is received 100% of care eligible for)		0–67%: 4.69 (3.89–5.64) 75–83%: 2.25 (1.79–2.83)	0–57%: 1.88 (1.68–2.11) 60–86%: 1.89 (1.70–2.09)	0–67%: 2.18 (1.88–2.54)	N/A	0–50%: 2.64 (1.96–3.55)	0–67%: 1.65 (1.43–1.91) 75–83%: 1.34 (1.16–1.56)

Abbreviations: SVD = spontaneous vaginal delivery; FVD = forceps-assisted vaginal delivery; VVD = vacuum-assisted vaginal delivery.

^aAdjusted for age, insurance status, prenatal care, obstetric history, any hypertension, diabetes mellitus, anticoagulant use, multiple gestation, previa, accreta, abruption, birthweight, attending specialty, years since attending graduated medical or midwifery school;

^bAdjusted for age, BMI, cigarette use, insurance status, obstetric history, diabetes mellitus, PROM/PPROM, GBS status, gestational age at delivery, attending specialty, years since attending graduated medical or midwifery school, nurse-to-patient ratio, prenatal EMR present, attending providers and/or nurses present for structured obstetric patient review, hospital volume;

^cAmong women with a singleton delivery and no shoulder dystocia;

^dAdjusted for age, BMI, cigarette use, insurance status, obstetric history, birthweight, attending specialty, prenatal EMR present;

^eAdjusted for age, BMI, cigarette use, insurance status, obstetric history, birthweight, prenatal EMR present;

^fAmong women with a term, non-anomalous singleton infant;

^gAdjusted for BMI, cigarette use, cocaine or methamphetamine use, insurance status, prenatal care, obstetric history, any hypertension, diabetes mellitus, PROM/PPROM, size for gestational age, attending specialty, round-the-clock in-house attending pediatrician available;

^hEmpty cells reflect that this process measure was not assessed for this outcome.

eFigures 1a – d (Supplement) represent the hospital differences in postpartum hemorrhage and how those differences are affected by the sequential addition of independent variables in the different categories (i.e., patient, provider/hospital, and care). For example, eFigure 1a (Supplement) illustrates the odds ratio for each hospital (identified by the numbers 1 to 25 on the x-axis) for the outcome of PPH derived from the logistic regression model without any risk-factor adjustment. Hospitals differ significantly from one another (P<0.01) and some hospitals (represented in red) have significantly higher or lower odds of an outcome than the reference hospital (i.e. 99% confidence intervals do not include 1.0). If patient, hospital, and process characteristics are associated with the outcomes, as they are entered into the regression model, variation among the odds ratios of the hospitals should lessen. If all variation were explained by these characteristics, the odds ratios associated with each hospital would be 1.0.

The results of adjusting only for patient characteristics are shown in eFigure 1b (Supplement), with the results obtained after the further addition of provider/hospital characteristics shown in eFigure 1c (Supplement). There is a progressive reduction in the variation of the odds ratios, as illustrated by the hospitals’ odds ratio point estimates that have “migrated” from their original positions and towards the line representing an “odds ratio = 1”. However, when care variables are entered into the model, either as a single variable such as “labor induction” (data not shown) or as a composite exposure score (eFigure 1d [Supplement]), the odds ratios associated with each hospital are largely unchanged. Graphical representations for the odds ratios associated with each stage of the model for the other outcomes are presented in the eFigures 2–6 (Supplement).

Table 5 presents the variation between hospitals (σ) associated with each stage of the hierarchical logistic regression for each outcome. For infection, none of the inter-hospital variation was explained by patient characteristics, whereas for the other outcomes between 20 and 40% (% difference between the σs) of the hospital’s variation in outcomes was related to differences in patient populations. About 20% of the variation in hospital PPH frequency was related to provider/hospital factors. However for the other outcomes there was little evidence that inter-hospital outcome variation was related to provider/hospital factors. In no case did differences in types of obstetric care account for much of the variation in observed outcomes. Ultimately, between 50 and 100% of the inter-hospital variation in outcomes was unexplained.

Table 5.

Variation (σ) in outcome frequency across the hospitals, crude and after adjustments for patient, provider/hospital, and care characteristics

		Variation (σ) (standard error)
	Denominator size for each outcome	Crude hierarchical regression	Hierarchical regression with patient characteristics	Hierarchical regression with patient and provider/hospital characteristics	Hierarchical regression with patient, provider/hospital, and care characteristics
Postpartum hemorrhage	105,987	0.20 (0.06)	0.16 (0.05)	0.13 (0.04)	0.13 (0.04)
Peripartum Infection	110,205	0.18 (0.05)	0.21 (0.06)	0.18 (0.06)	0.18 (0.06)
Severe perineal laceration at SVDa	68,144	0.15 (0.05)	0.09 (0.03)	0.09 (0.03)	0.09 (0.03)
Severe perineal laceration at FVDa	1898	0.33 (0.13)	0.25 (0.11)	0.26 (0.12)	N/A
Severe perineal laceration at VVDa	3515	0.20 (0.09)	0.15 (0.08)	0.15 (0.08)	0.14 (0.08)
Composite neonatal adverse outcomeb	89,279	0.17 (0.05)	0.10 (0.04)	0.09 (0.03)	0.09 (0.03)

Abbreviations: SVD = spontaneous vaginal delivery; FVD = forceps-assisted vaginal delivery; VVD = vacuum-assisted vaginal delivery.

^aAmong women with a singleton delivery and no shoulder dystocia;

^bAmong women with a term, non-anomalous singleton infant.

Comment

In this study, we investigated the relationship between differences in obstetric care patterns and outcomes among hospitals. Several findings are notable. Despite the fact that the hospitals in the study were either university or university-affiliated and part of a single research network, the frequencies of obstetric practices were vastly different. After controlling for differences in patient populations and hospital characteristics, several types of obstetric care were found to be associated with adverse obstetric outcomes. Nevertheless, this association did not translate into a capability to explain the hospital differences in adverse outcomes that were found.

This lack of explanatory power is in contrast to that discerned for care processes in some other disciplines. For example, Synnes et al¹⁰ examined variation in the frequency of intraventricular hemorrhage among neonates in the intensive care unit. In an analysis similar to ours, after controlling for patient and hospital factors, they were able to demonstrate that differences in acidosis treatment, vasopressin use, and surfactant use could account for differences in inter-hospital rates of intraventricular hemorrhage. Similarly, studying adults with cardiac disease, Petersen et al¹¹ demonstrated that adherence to particular types of management (such as beta-blocker use) could explain differences in hospitals’ adjusted-mortality rates.

Process measures, however, have not been well demonstrated to explain inter-hospital variation in obstetric outcomes. The inability to do so in the obstetrical population we studied has implications with regard to obstetric quality measurement and its interpretation. “Process measures” quantify adherence to a given type of care. Hospitals are often judged according to their adherence to selected process measures, with the implicit assumption that the hospitals that perform best on selected measures will have the best health outcomes. Yet, Draycott et al⁵ have called attention to the fact that this relationship need not hold. Further, they cite examples to illustrate that belief in an inexorable relationship between process measures and outcomes may hinder quality improvement if there is undue focus on process measures, which may be relatively easy to measure, and less attention paid to actual outcomes.

Our findings support Draycott et al’s⁵ contention that although process measures may be associated with an adverse outcome, the hospitals that perform “best” on those measures, or combinations of those measures, do not necessarily have the best risk-adjusted rates of obstetric morbidity. This may be because the labor and delivery process is complex and dynamic, and the evidence base for “best practice” remains poor. Indeed, the wide variation in the use of different obstetric practices – starting from the time a woman is admitted, continuing through her labor, and present at her delivery – are another manifestation of the lack of consensus for what constitutes best care during many aspects of labor.

These data do not imply that process measurement lacks any value. Process measurement may provide insight into types of care that hospitals wish to perform more frequently and may help direct internal improvement initiatives. Also, although we believe we have selected and analyzed process measures that are most likely to be associated with variation in outcomes, there are other process measures that exist and we cannot rule out the possibility that these unstudied measures would have a relationship with inter-hospital variation of outcomes. Nevertheless, such relationships have not been demonstrated, and our findings suggest that the care factors underlying inter-hospital variation in obstetric outcomes remain poorly understood, and that the practice of ranking individual hospital obstetric quality based on frequency of adherence to certain process measures may provide poor insight into which hospitals actually achieve the best outcomes.

Appendix A

In addition to the authors, other members of the Eunice Kennedy Shriver National Institute of Child Health and Human Development Maternal-Fetal Medicine Units Network are as follows:

Northwestern University, Chicago, IL – G. Mallett, M. Ramos-Brinson, A. Roy, L. Stein, P. Campbell, C. Collins, N. Jackson, M. Dinsmoor (NorthShore University Health System), J. Senka (NorthShore University Health System), K. Paychek (NorthShore University Health System), A. Peaceman

Case Western Reserve University-MetroHealth Medical Center, Cleveland, OH – B. Mercer, C. Milluzzi, W. Dalton, T. Dotson, P. McDonald, C. Brezine, A. McGrail

Columbia University, New York, NY – M. Talucci, M. Zylfijaj, Z. Reid (Drexel U.), R. Leed (Drexel U.), J. Benson (Christiana H.), S. Forester (Christiana H.), C. Kitto (Christiana H.), S. Davis (St. Peter's UH.), M. Falk (St. Peter's UH.), C. Perez (St. Peter's UH.)

University of Utah Health Sciences Center, Salt Lake City, UT – K. Hill, A. Sowles, J. Postma (LDS Hospital), S. Alexander (LDS Hospital), G. Andersen (LDS Hospital), V. Scott (McKay- Dee), V. Morby (McKay-Dee), K. Jolley (UVRMC), J. Miller (UVRMC), B. Berg (UVRMC)

University of North Carolina at Chapel Hill, Chapel Hill, NC – K. Dorman, J. Mitchell, E. Kaluta, K. Clark (WakeMed), K. Spicer (WakeMed), S. Timlin (Rex), K. Wilson (Rex)

University of Texas Southwestern Medical Center, Dallas, TX – L. Moseley, M. Santillan, J. Price, K. Buentipo, V. Bludau, T. Thomas, L. Fay, C. Melton, J. Kingsbery, R. Benezue

University of Pittsburgh, Pittsburgh, PA – H. Simhan, M. Bickus, D. Fischer, T. Kamon (deceased), D. DeAngelis

The Ohio State University, Columbus, OH – C. Latimer, L. Guzzo (St. Ann's), F. Johnson, L. Gerwig (St. Ann's), S. Fyffe, D. Loux (St. Ann's), S. Frantz, D. Cline, S. Wylie, P. Shubert (St. Ann's)

University of Alabama at Birmingham, Birmingham, AL – M. Wallace, A. Northen, J. Grant, C. Colquitt

University of Texas Medical Branch, Galveston, TX – J. Moss, A. Salazar, A. Acosta, G. Hankins

Wayne State University, Detroit, MI – N. Hauff, L. Palmer, P. Lockhart, D. Driscoll, L. Wynn, C. Sudz, D. Dengate, C. Girard, S. Field

Brown University, Providence, RI – P. Breault, F. Smith, N. Annunziata, D. Allard, J. Silva, M. Gamage, J. Hunt, J. Tillinghast, N. Corcoran, M. Jimenez

The University of Texas Health Science Center at Houston-Children’s Memorial Hermann Hospital, Houston, TX – S. Blackwell, F. Ortiz, P. Givens, B. Rech, C. Moran, M. Hutchinson, Z. Spears, C. Carreno, B. Heaps, G. Zamora

Oregon Health & Science University, Portland, OR – J. Seguin, M. Rincon, J. Snyder, C. Farrar, E. Lairson, C. Bonino, W. Smith (Kaiser Permanente), K. Beach (Kaiser Permanente), S. Van Dyke (Kaiser Permanente), S. Butcher (Kaiser Permanente)

The George Washington University Biostatistics Center – E. Thom, Y. Zhao, P. McGee, V. Momirova, R. Palugod, B. Reamer, M. Larsen

Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, MD – S. Tolivaisa