The Association between Hospital-level Obstetric Quality Indicators and Maternal and Neonatal Morbidity

Abstract

Importance

In an effort to improve the quality of care, several obstetric-specific quality measures are now monitored and publically reported. The extent to which these measures are associated with maternal and neonatal morbidity is not known.

Objective

To examine whether 2 Joint Commission obstetric quality indicators are associated with maternal and neonatal morbidity.

Design, Setting, and Participants

Population-based observational study using linked 2010 New York City discharge and birth certificate datasets. All delivery hospitalizations were identified and two perinatal quality measures were calculated. Published algorithms were used to identify severe maternal morbidity (delivery associated with a life threatening complication or performance of a life-saving procedure) and morbidity in non-anomalous term newborns (births associated with complications such as birth trauma, hypoxia, and prolonged length of stay). Mixed-effects logistic regression models were used to examine the association between maternal morbidity, neonatal morbidity, and hospital-level quality measures while risk-adjusting for patient sociodemographic and clinical characteristics.

Exposure

Two Joint Commission perinatal quality measures: 1) elective (non-medically indicated) deliveries at >= 37 and < 39 weeks of gestation and 2) cesarean delivery performed in low-risk mothers.

Main Outcomes and Measures

Individual and hospital level maternal and neonatal morbidity.

Results

Severe maternal morbidity occurred among 2.4% of 115,742 deliveries and neonatal morbidity occurred among 7.8% of 103,416 non-anomalous term newborns. Rates for elective deliveries performed before 39 weeks of gestation ranged from: 15.5 to 41.9 per 100 deliveries among 41 hospitals. There were 11.7 to 39.3 cesareans per 100 deliveries performed in low-risk mothers. Overall maternal morbidity ranged from 0.9 to 5.7 mothers with complications per 100 deliveries and 3.1 to 21.3 neonates with complications per 100 deliveries. The maternal quality indicators elective delivery before 39 weeks of gestation and cesarean delivery performed in low-risk mothers were not associated with severe maternal complications (RR, 1.00; 95% CI: 0.98–1.02 and RR, 0.99, 95% CI: 0.96–1.01, respectively) or neonatal morbidity (RR, 0.99; 95% CI: 0.97–1.01 and RR, 1.01, 95% CI: 0.99–1.03, respectively).

Conclusions and Relevance

Rates for the quality indicators elective delivery before 39 weeks of gestation and cesarean delivery performed in low risk mothers varied widely in New York City hospitals as did maternal and neonatal complications rates. However, there were no correlations between the quality indicator rates and maternal and neonatal morbidity. Current quality indicators may not be sufficiently comprehensive for guiding quality improvement in obstetric care.

Although great progress has been made in reducing obstetric complications, they persist. Severe maternal complications include renal failure and eclampsia or the need for lifesaving interventions such as prolonged mechanical ventilation or transfusions.^1,2 Neonatal complications may occur in low-risk term infants and include hypoxia and shock.³ Severe maternal morbidity occurs in about 60,000 women (1.6 per 100 deliveries) annually in the US and 1 in 10 term infants experience neonatal complications.^3,4 Variation in complication rates between hospitals exists and suggests that the quality of obstetric care can be improved.⁵ Over a third of maternal deaths and severe morbidities, and a significant proportion of neonatal mortality and morbidity may be preventable by changes in patient, clinician, and system factors.^4,6–9

As part of its core measure set, the Joint Commission now recommends two perinatal quality measures that address important aspects of obstetric care during childbirth: 1) elective deliveries performed prior to 39 weeks of gestation and 2) cesarean deliveries performed in low-risk nulliparous women.¹⁰ The elective delivery measure, which includes non-medically indicated deliveries associated with medical induction or cesarean delivery over 37 weeks and prior to 39 weeks gestation, is also mandated by the Centers for Medicare and Medicaid Services.¹¹ The elective delivery before 39 weeks of gestation indicator is intended to reduce neonatal complications among term infants. Assessing rates of cesarean delivery performed in low-risk patients is intended to reduce unnecessary variation in cesarean delivery rates. Both of these measures may be associated with maternal outcomes.^12,13 However, how well hospital performance on these quality indicators correlate with maternal or neonatal morbidity is not known.

We investigated whether elective deliveries performed prior to 39 weeks of gestation and cesarean deliveries performed in low-risk nulliparous women were associated with severe maternal or neonatal morbidity in New York City Hospitals.

METHODS

Study Sample

We used birth certificate data linked with New York State discharge abstract data - The Statewide Planning and Research Cooperative System (SPARCS) for all delivery and newborn hospitalizations in New York City in 2010. Data linkage was conducted by the New York State Department of Health. Ninety-seven percent of maternal and 98% of infant discharge abstracts were linked with infant birth certificates. Institutional Review Board approvals were obtained from the New York City Department of Health and Mental Hygiene, the New York State Department of Health, and the Icahn School of Medicine at Mount Sinai. A waiver of consent was approved by the Icahn School of Medicine. Delivery hospitalizations were identified based on ICD-9-CM diagnosis and procedure codes and DRG delivery codes.¹⁴ Singleton non-anomalous term newborns were identified using the newborn flag. Gestational age was ascertained from birth certificate data and congenital anomalies from SPARCS data.

Identifying Severe Maternal Morbidity

We used a Centers for Disease Prevention and Controls (CDC)-endorsed algorithm to identify severe maternal morbidity, using diagnoses for life-threatening conditions and procedure codes for life-saving procedures that is currently used for surveillance.² (See: http://www.cdc.gov/reproductivehealth/MaternalInfantHealth/SevereMaternalMorbidity.html) Codes were selected by experts and reviewed for their association with in-hospital maternal mortality. The algorithm excludes hospitalizations with a length of stay less than the 90th percentile as calculated separately for vaginal, primary, and repeat cesarean deliveries.² Length of stay reclassification was not applied to in-hospital deaths, transfers or delivery hospitalizations with severe complications identified by procedure codes as recommended.²

Identifying Neonatal Morbidity at Term

We identified neonatal morbidity at term based on diagnoses and procedure codes as defined by Korst et al. to monitor childbirth morbidity using hospital discharge data.^3,15,16 The list includes ICD-9 code-based categories that capture indicators of death or neonatal complications (e.g., birth trauma, disseminated intravascular coagulation, neonatal intensive care unit (NICU) procedures, renal failure, respiratory conditions, necrotizing enterocolitis, shock, neonatal length of stay >5 days, neonatal death) and was ascertained from SPARCS data.³

Quality Measures

We used birth certificate data and SPARCS data to calculate two perinatal quality measures using algorithms designated by the Joint Commission.¹⁰ The first, elective deliveries at >= 37 and < 39 weeks of gestation was defined as all deliveries that were associated with medical inductions of labor or cesarean delivery at >= 37 and < 39 weeks of gestation as a proportion of all deliveries at >= 37 and < 39 weeks of gestation. All conditions possibly justifying delivery prior to 39 weeks gestation were excluded, as specified.¹⁰ We also excluded cesarean deliveries that were associated with a trial of labor but were not inductions. The second measure, cesarean deliveries in low-risk women, was defined as the proportion of cesarean deliveries among nulliparous patients with singleton vertex deliveries of newborns at ≥37 weeks gestation. We excluded patients with ICD-9 codes that signify contraindications to vaginal deliveries.¹⁰ Gestational age, parity, multiple birth, vertex presentation and trial of labor were ascertained from birth certificate data. ICD-9 codes were obtained from SPARCS.

For each hospital we calculated the rate of elective deliveries and low-risk cesarean deliveries. No risk adjustment is required for the elective delivery measure, but direct standardization by maternal age is suggested for the cesarean delivery measure. We standardized this indicator for each hospital by 5-year age groups using the observed maternal age distribution in the entire sample.¹⁰

Covariates

To risk-adjust hospital-level rates of maternal morbidity we used variables from birth certificate records, including mothers’ sociodemographics (maternal age, self-identified race and ethnicity, parity, education), prenatal care visits, and clinical and obstetric factors (multiple births, history of previous cesarean delivery). We ascertained patient insurance status from SPARCS. We also included diagnoses codes for patient risk factors that could lead to maternal morbidity but were likely present on admission to the hospital (e.g. diabetes, hypertension, obesity, premature rupture of membranes, disorders of placentation). These conditions have been used to risk-adjust for severe maternal morbidity,¹⁷ cesarean deliveries, and other maternal outcomes (Appendix A).^18,19

We risk-adjusted neonatal morbidity by maternal sociodemographics (maternal age, self-identified race and ethnicity, parity, education), behaviors (prenatal care visit, tobacco use, illicit drug use), maternal clinical factors (hypertension or diabetes during or before pregnancy), obstetric factors (previous cesarean delivery, premature rupture of membranes), and infant factors (gestational age, size for gestational age).²⁰ Small for gestational age (SGA) was defined as a birthweight <10th percentile and large for gestational age (LGA) was defined as a birthweight >90th percentile for gestational age and sex.²¹

We obtained teaching status from the American Hospital Association, ownership and nursery level from the New York State Department of Health, and volume of deliveries in each hospital from SPARCS to assess how other hospital characteristics are correlated with measures of quality and morbidity.

Statistical Analyses

We compared the sociodemograpic characteristics and clinical conditions of women with and without severe maternal morbidity and infants with and without neonatal morbidity using Chi Square tests.

We used mixed-effects logistic regression with a random hospital-specific intercept to generate risk-standardized severe maternal morbidity rates (SSMMR) and risk-standardized neonatal morbidity rates (SNMR) at term for each hospital. The models included covariates described above. Hospital risk-standardized rates were computed from these models using methods recommended by CMS Hospital Compare.²² (See Appendix B for more detail).

We used Spearman’s rank correlations to assess correlations among hospital-level rates of severe maternal morbidity, neonatal morbidity, and the two measures of hospital quality. We divided hospitals into quartiles based on each quality measure and examined hospital rankings on both risk-standardized morbidity measures using the nonparametric Kruskal-Wallis Test. Quality measures were coded as continuous for our primary multivariable analyses. Our final models explored the association of the two quality measures with severe maternal morbidity and then with neonatal morbidity using mixed-effects logistic regression in three steps: (1) with no adjustment, (2) with adjustment for patient characteristics, and (3) with adjustment for patient and hospital-level characteristics, including ownership, volume, level of nursery and teaching status. Risk ratios were calculated using recycled predictions, which yields the marginal effect of a variable while holding all other patient characteristics constant;²³ confidence intervals were estimated using the delta method.²⁴ Intraclass correlations were calculated to measure the between hospital variance as a proportion of the total variance.

Sensitivity analyses included coding quality measures in quartiles; examining the association of the quality measures with severe maternal morbidity without blood transfusion; and using two simpler measures of neonatal outcomes: prolonged length of stay (>5 days) and NICU admission.

All statistical analysis was performed using the SAS system software version 9.3 (SAS Institute Inc, Cary, NC). A p value of less than .05 was considered statistically significant and two-sided tests were utilized.

RESULTS

Patient Sample Characteristics, severe maternal morbidity, neonatal morbidity at term

Of the 43 hospitals with deliveries, we excluded 2 hospitals with annual delivery volumes less than 5 births and 1,360 deliveries with missing hospital identifiers. The final sample included 115,742 deliveries of which 2,732 (2.4%) were associated with severe maternal morbidity. Of the 119, 793 newborns identified, we excluded 4,672 multiple births, 4,447 congenital anomalies, and 7,258 births with gestational age <37 weeks. The final sample included 103,416 newborns and 8,057 (7.8%) were associated with neonatal morbidity. The associations of sociodemographic and clinical characteristics with severe maternal morbidity and neonatal morbidity are described in Tables 1 and 2.

Table 1.

Socio-demographic and Clinical Characteristics for Deliveries by Presence of Severe Maternal Morbidity

	Severe Maternal Morbidity
	Yes (n=2,732) N (%)	No (n=113,010) N (%)	Risk Ratio2	95% CI2
Maternal Age
<20	203 (7.4)	6,597 (5.8)	0.95	0.81–1.09
20–29	1,183 (43.3)	50,670 (44.8)	ref
30–34	686 (25.1)	31,442 (27.8)	1.11	1.01–1.22
35–39	446 (16.3)	18,694 (16.5)	1.18	1.05–1.32
40–44	185 (6.8)	5,151 (4.6)	1.55	1.31–1.79
45+	29 (1.1)	456 (0.4)	1.96	1.25–2.68
Maternal Race
White	1,282 (46.9)	65,558 (58.1)	ref
Black	1,086 (39.8)	29.225 (25.9)	1.33	1.19–1.46
Asian	347 (12.7)	17,432 (15.4)	1.45	1.26–1.64
Other	17 (0.6)	795 (0.7)	0.90	0.48–1.32
Hispanics	1,023 (37.5)	35,780 (31.7)	1.33	1.20–1.46
Maternal Education
Less than HS	859 (31.4)	26,908 (23.8)	1.22	1.10–1.35
HS	666 (24.4)	25,493 (22.6)	1.12	1.01–1.23
Greater than HS	1,207 (44.2)	60,609 (53.6)	ref
Insurance
Commercial	725 (26.5)	42,294 (37.4)	ref
Uninsured	42 (1.5)	1,258 (1.1)	1.03	0.78–1.36
Medicaid	1,944 (71.2)	68,585 (60.7)	1.09	0.97–1.22
Other	21 (0.8)	873 (1.2)	1.05	0.60–1.51
Prenatal visits
0–5	326 (11.9)	6,943 (6.1)	1.34	1.18–1.51
6–8	381 (14.0)	13,172 (11.7)	1.08	0.96–1.19
≥9	1,932 (70.7)	90,384 (80.0)	ref
Unknown	93 (3.4)	2,511 (2.2)	1.21	0.94–1.48
Parity
Nulliparous	1,220 (44.7)	52,210 (46.2)	ref
Multiparous	1,512 (55.3)	60,800 (53.8)	0.68	0.62–0.74
Type of Pregnancy
Singleton	2,534 (92.8)	110,696 (98.0)	ref
Multiple	198 (7.3)	2,314 (2.1)	2.85	2.44–3.26
Previous Cesarean	783 (28.7)	16,816 (14.9)	2.32	2.11–2.53
Comorbidities
Cardiac Disease	50 (1.83)	510 (0.45)	2.94	2.10–3.78
Renal Disease	19 (0.70)	37 (0.03)	4.84	2.36–7.32
Musculoskeletal Disease	13 (0.48)	284 (0.25)	2.79	0–8.54*
Digestive Disorder**	<10 (<3.7)	85 (0.08)	1.56	0–3.26*
Blood Disease	1,082 (39.60)	10,611 (9.39)	4.95	4.55–5.35
Mental Disorders	114 (4.17)	2789 (2.47)	1.18	0.96–1.40
CNS disease	45 (1.65)	965 (0.85)	1.54	1.10–1.98
Rheumatic Heart Disease**	<10 (<3.7)	48 (0.04)	2.17	0.43–3.92
Disorder Placentation	279 (10.21)	1,609 (1.42)	4.72	4.14–5.30
Chronic Hypertension	74 (2.71)	1376 (1.22)	1.35	1.04–1.67
Pregnancy Hypertension	474 (17.35)	5,430 (4.80)	2.55	2.29–2.81
Lupus**	<10 (<3.7)	159 (0.14)	0.36	0–1.21*
Collagen Vascular Disorder**	<10 (<3.7)	36 (0.03)	0.76	0–2.53*
Rheumatoid Arthritis**	<10 (<3.7)	91 (0.08)	0.29	0–1.03*
Diabetes	51 (1.87)	689 (0.61)	1.36	0.30–2.43
Diabetes Complicating Pregnancy	55 (2.01)	764 (0.68)	1.08	0.26–1.91
Obesity	150 (5.49)	3,635 (3.22)	1.17	0.98–1.37
Asthma/Chronic bronchitis	186 (6.8)	5,035 (4.5)	1.07	0.91–1.23
Delivery method1
Cesarean Delivery	1,902 (69.6)	36,196 (32.1)
Vaginal delivery	890 (30.4)	78,938 (67.9)
Hospital Characteristics1
Hospital Ownership
Public	725 (26.5)	42,294 (37.4)
Private	2,007 (73.5)	70,716 (62.6)
Teaching Status
Not Teaching	36 (1.3)	2,603 (2.3)
Teaching	2,696 (98.7)	110,407 (97.7)
Nursery Level
Level 2	272 (10.0)	14,118 (12.5)
Level 3–4	2,460 (90.0)	98,892 (87.5)
Delivery Volume 3
Low	499 (18.3)	12,417 (11.0)
Medium	580 (21.2)	20,796 (18.4)
High	672 (24.6)	30,069 (26.6)
Very High	981 (35.9)	49,728 (44.0)

¹Variables are not included in multivariable analysis for calculation of risk adjusted severe maternal morbidity.

²Derived from a logistic regression model adjusting for maternal sociodemographics, clinical characteristics, and comorbid conditions.

³Interquartile range is shown in Table 3 (see Delivery Volume)

^*We estimated confidence intervals for the risk ratios using the delta method as implemented in the “nlcom” command in Stata 12. In a few cases where the confidence interval on the coefficient was especially large, the delta method generated a 95% CI lower bound of the risk ratio that was less than zero. We truncated the confidence limit at 0.

^**Number of observation not shown since cell size is less than 10

Table 2.

Sociodemographic and Clinical Characteristics for Deliveries by Presence of Moderate and Severe Neonatal Term Morbidity among Term Non-anomalous¹ Newborns

	Severe Neonatal Morbidity at Term
	Yes (n=8,057)	No (n= 95,359)	Risk Ratio2	95% CI2
	N (%)	N (%)
Maternal Age
<20	632 (7.8)	5,464 (5.7)	0.93	0.85–1.01
20–29	3,734 (46.3)	43,226 (45.3)	ref
30–34	2,027 (25.2)	26,649 (28.0)	1.05	0.99–1.11
35–39	1,249 (15.5)	15,559 (16.3)	1.13	1.06–1.21
40–44	382 (4.8)	4,144 (4.4)	1.22	1.10–1.35
45+	33 (0.4)	317 (0.3)	1.37	0.93–1.81
Maternal Race
White	4,407 (54.7)	57,606 (60.4)	ref
Black	2,743 (34.1)	23,640 (24.8)	1.21	1.14–1.29
Asian	988 (12.3)	15,335 (16.1)	0.93	0.85–1.00
Other	167 (2.1)	2459 (2.6)	0.73	0.46–0.99
Hispanics	2,761 (34.3)	30,135 (31.6)	1.45	1.35–1.55
Maternal Education
Less than HS	2,152 (26.7)	22,530 (23.6)	1.04	0.97–1.10
HS	1,851 (23.0)	21,457 (22.5)	1.00	0.95–1.06
Greater than HS	4,005 (49.7)	51,065 (53.6)	ref
Unknown	49 (0.6)	307 (0.3)	1.10	0.74–1.47
Insurance
Commercial	2,426 (30.1)	35,596 (37.3)	ref
Uninsured	188 (2.3)	1,730 (1.8)	1.19	1.01–1.37
Medicaid	5,378 (66.8)	57,329 (60.1)	1.18	1.10–1.26
Other	65 (0.8)	704 (0.7)	1.47	1.12–1.82
Illicit Drug Use	174 (2.2)	630 (0.7)	2.73	2.08–3.39
Tobacco Use	662 (8.2)	3408 (3.6)	1.42	1.26–1.59
Prenatal visits
0–5	659 (8.2)	5,117 (5.4)	1.18	1.08–1.27
6–8	999 (12.4)	10,393 (10.9)	1.08	1.01–1.15
≥9	6,110 (75.8)	77,949 (81.7)	ref
Unknown	290 (3.6)	1,900 (2.0)	1.21	1.06–1.37
Multiparous	3,599 (44.7)	51,850 (54.4)	0.6	0.57–0.63
Previous Cesarean	988 (12.3)	10,157 (10.7)	1.45	1.35–1.55
Premature Ruptured Membranes	274 (3.4)	2,961 (3.1)	1.07	0.94–1.19
Maternal Hypertension	488 (6.1)	2,620 (2.7)	1.63	1.49–1.78
Maternal Diabetes	550 (6.8)	4,634 (4.9)	1.31	1.20–1.42
Gestational age
37	1,035 (12.9)	6,989 (7.3)	1.60	1.48–1.71
38	1,430 (17.8)	17,152 (18.0)	1.04	0.97–1.11
39	2,605 (32.3)	34,341 (36.0)	0.97	0.92–1.03
40	2,120 (26.3)	27,848 (29.2)	ref
41 plus	867 (10.8)	9,029 (9.5)	1.2	1.11–1.29
Newborn weight
Small for gestational age	492(6.1)	3423 (3.6)	1.54	1.40–1.67
Appropriate for gestational age	7,242 (89.9)	89,726 (94.1)	ref
Large for gestational age Delivery method3	323(4.0)	2210 (2.3)	1.73	1.55–1.91
Cesarean Delivery	3,552 (44.1)	27,064 (28.4)
Vaginal delivery	4,505(55.9)	68,295(71.6)
Hospital Characteristics3
Hospital Ownership
Public	2,070 (25.7)	16,861 (17.7)
Private	5,987 (74.3)	78,498 (82.3)
Teaching Status
Not Teaching	139 (1.7)	2,301 (2.4)
Teaching	7,918 (98.3)	93,058 (97.6)
Nursery Level
Level 2	771 (9.6)	12,469 (13.1)
Level 3–4	7,286 (90.4)	82,890 (86.9)
Delivery Volume4
Low	1,381 (17.1)	10,157 (10.7)
Medium	1,711 (21.2)	17,537 (18.4)
High	2,166 (26.9)	25,679 (26.9)
Very High	2,799 (34.7)	41,986 (44.0)

¹Congenital anomalies identified using ICD-9 codes given in Korst et al.³

²Derived from a logistic regression model adjusting for maternal sociodemographics, clinical characteristics, and comorbid conditions.

³Variables not included in multivariable analysis.

⁴Interquartile range is shown in Table 3 (see Delivery Volume)

Hospital Characteristics and Performance on Perinatal Quality and Morbidity Measures

Table 3 shows that the majority of hospitals were private, had Level 3/4 nurseries, and were teaching hospitals. Hospital performance per 100 deliveries ranged from: 15.5 to 41.9 for elective deliveries before 39 weeks; 11.7 to 39.3 for age-standardized rates of cesarean deliveries in low-risk nulliparous women; 0.9 to 5.7 for risk-standardized severe maternal morbidity rates; and 3.1 to 21.3 for risk-standardized neonatal morbidity rates among these hospitals. Both quality measures were correlated with each other (correlation=0.45; p=0.003). Severe maternal morbidity and neonatal morbidity at term were also correlated (correlation=0.39; p = 0.01).

Table 3.

New York City Delivery Hospital Characteristics and Performance Indicators (n=41)

Characteristics	%/Median	n/[IQR]	Range
Hospital Ownership
Public	26.8	11	--
Private	73.2	30	--
Teaching Status
Not Teaching	4.9	2	--
Teaching	95.1	39	--
Nursery Level
Level 2	17.1	7	--
Level 3–4	82.9	34	--
Delivery Volume	2,493	[1,670 – 3,895]	447–7,554
Quality Indicator 1: Elective deliveries before 39 weeks1	27.7	[23.1 – 31.1]	15.5–41.9
Quality Indicator 2: Cesarean in low risk women2	21.0	[17.5 – 24.1]	11.7–39.3
Unadjusted severe maternal morbidity3	2.4	[1.5 – 3.8]	0.5–11.0
Adjusted severe maternal morbidity4	2.6	[1.9 – 3.2]	0.9–5.7
Unadjusted severe neonatal morbidity at term 5	7.7	[5.8–10.6]	3.1–24.7
Adjusted severe neonatal morbidity at term6	8.0	[6.0–9.5]	3.1–21.3

¹per 100 deliveries ≥37 and < 39 weeks gestation. For this measure, numerator = 4,677 and denominator = 17,393.

²per 100 low-risk deliveries to nulliparous women with a term, singleton baby in a vertex position. For this measure, numerator = 6,209 and denominator = 31,981.

³per 100 deliveries.

⁴per 100 deliveries adjusted for maternal characteristics as described in Table 1.

⁵per 100 deliveries of singleton, term, non-anomalous babies

⁶per 100 deliveries of singleton, term, non-anomalous babies adjusted for maternal and neonatal characteristics as described in Table 2

Hospital Rankings on Quality Measures and Risk-standardized Maternal and Neonatal Morbidity

Hospital rankings on both quality measures were not associated with hospital rankings for maternal or neonatal morbidity as demonstrated in Figure 1. In fact, among the 10 hospitals with the best performance (lowest rates) on elective deliveries, only 3 were in the lowest quartile of risk-standardized severe maternal morbidity. Among the 10 hospitals with the best performance (lowest rates) on low-risk cesarean delivery measure, only 3 were in the lowest quartile of risk-standardized severe maternal morbidity. The rankings were similarly discordant for neonatal morbidity. Nonparametric correlations between obstetric quality indicators and both maternal and neonatal morbidity were not significant (c-section and maternal morbidity: −0.01, p=0.94; c-section and neonatal morbidity: −0.10, p=0.52; elective delivery and maternal morbidity: 0.14, p=0.39; elective delivery and neonatal morbidity: −0.14, p=0.39).

Figure 1.

Risk adjusted maternal (A and B) and neonatal morbidity (C and D) by quality indicators. Hospitals ranked in the top 10 on the quality measures (lowest rate for elective and cesarean deliveries) are shown as black circles, hospital ranked in the bottom 10 on these quality measures (highest rate for elective and cesarean deliveries) are shown as black triangles.

Mixed-Effects Logistic Regression Models

In mixed-effects models including both quality measures as independent variables, neither quality indicator was associated with severe maternal morbidity (Model 1, Table 4); the risk ratio (RR) for elective delivery was 1.00 (95% CI: 0.97–1.03) and for low-risk cesarean deliveries was 0.98 (0.95–1.01). In models which also included patient-level variables (Model 2) and patient-level and other hospital-level variables (Model 3), the risk ratios for both quality measures remained essentially unchanged. Sensitivity analyses with the quality measures coded in quartiles and analyses excluding blood transfusions from the severe maternal morbidity outcome corroborated our findings. There was a substantial between-hospital variation in severe maternal morbidity: the intraclass correlation coefficient (ICC) was 28% in the empty model (without any adjustment); after adjusting for patient characteristics ICC was reduced to 22%.

Table 4.

Association between Hospital-level Quality Indicators and Characteristics with Severe Maternal and Neonatal Morbidity

	Severe Maternal Morbidity			Severe Neonatal Morbidity at Term
Hospital Level Variables	Model 1 – Unadjusted	Model 2* – Adjusted for Maternal Characteristics	Model 3 – Adjusted for Maternal and Hospital Characteristics	Model 1 – Unadjusted	Model 2# – Adjusted for Maternal and Neonatal Characteristics	Model 3 – Adjusted for Maternal, Neonatal and Hospital Characteristics
	RR (95% CL)	Adjusted RR (95% CL)	Adjusted RR (95% CL)	RR (95% CL)	Adjusted RR (95% CL)	Adjusted RR (95% CL)
Quality Indicator 1 – Elective Deliveries < 39 weeks (per one percent from mean)	1.00 (0.97–1.03)	1.00 (0.98–1.03)	1.00 (0.98–1.02)	1.00 (0.98–1.03)	1.01 (0.99–1.02)	0.99 (0.97–1.01)
Quality Indicator 2 – Cesarean Delivery for Low Risk Nulliparous Women (per one percent from mean)	0.98 (0.95–1.01)	0.99 (0.96–1.01)	0.99 (0.96–1.01)	1.00 (0.98–1.03)	1.01 (0.99–1.03)	1.01 (0.99–1.03)
Public Hospital (ref=private)			1.36 (0.96–1.76)			1.11 (0.82–1.39)
Teaching Hospital (ref=non-teaching)			1.17 (0.37–1.98)			1.08 (0.52–1.65)
Delivery Volume in Quartiles (ref=very high)
Low			1.41 (0.92–1.89)			1.71 (1.20–2.23)
Medium			1.19 (0.76–1.63)			1.30 (0.87–1.95)
High			0.83 (0.56–1.09)			1.27 (0.86–1.68)
Level II Nursery (ref = Level 3–4)			0.71 (0.47–0.94)			0.65 (0.46–0.84)

^*Maternal characteristics adjusted for in this model include age, race, ethnicity, insurance, parity, education, prenatal visits, multiple births previous cesarean delivery, and comorbidities (e.g. hypertension, diabetes). A full list of comorbidities is included in Table

^#Maternal and neonatal characteristics adjusted for in this model include maternal factors (age, race, ethnicity, insurance, parity, education, prenatal visits, tobacco use, illicit drug use, hypertension or diabetes during pregnancy, previous cesarean delivery, premature rupture of membranes), and infant factors (gestational age, size for gestational age).

Our findings were similar in models examining the two obstetric quality measures and neonatal morbidity (Table 4). Volume of deliveries and nursery were associated with neonatal morbidity. The association between the two obstetric quality measures with neonatal length of stay > 5 days and NICU admission showed similar results. In a multilevel model without adjustment, the ICC was 20% for neonatal morbidity. The variance between hospitals remained (ICC=16%) after accounting for patient characteristics.

DISCUSSION

Severe maternal morbidity and neonatal morbidity at term remain important health issues. We found that in New York City hospitals 2.4% of all mothers and 7.8% of term non-anomalous newborns have major complications and that these rates vary substantially between hospitals. Severe maternal morbidity rates varied four to five-fold between hospitals and there was a seven-fold variation in neonatal morbidity at term between hospitals. Although there was substantial variation in morbidity rates, they were not correlated with the performance measures designed to assess hospital-level obstetric quality of care.

Measuring quality of care in obstetrics is complex: it involves assessing care for two separate individuals (mother and baby). Improving the quality of care requires reducing the use of obstetric interventions which can harm babies, (e.g. early delivery) and mothers (e.g. unnecessary cesarean delivery) and avoiding suboptimal care such as underutilization of antenatal steroids or antibiotics which can lead to neonatal or maternal complications or death. Measures of overutilization may be important obstetric quality indicators and enable the tracking of obstetric intervention utilization, however the measures we assessed do not reflect the quality of care in terms of severe maternal or neonatal morbidity.

The Joint Commission set of perinatal quality measures includes three other indicators in addition to the two we studied. Those measures assess other dimensions of perinatal care – breastfeeding, healthcare associated infections among very low birth weight infants, and antenatal steroid use among premature deliveries. The measures may be indicators of obstetrical quality, but the outcomes primarily concern newborns and not their mothers and two of the measures only assess care provided in the neonatal period. Our findings suggest that other quality measures be developed that focus on suboptimal care. Examples include whether hemorrhage and preeclampsia protocols are used in the delivery suite.^25–29 Given that we did not find a relationship between the quality indicators we examined and maternal or neonatal morbidity, these measures do not adequately capture obstetrical hospital quality.

Our results are consistent with findings from the Maternal Fetal Medicine Network and studies in NICUs demonstrating that performance assessment based on isolated measures do not accurately characterize the overall quality of care in a hospital.^20,30 Numerous other studies have found a lack of correlation between quality indicators and the clinical outcomes they are supposed to reflect. Hospital Compare measures only predict small differences in hospital-risk adjusted mortality rates for heart failure, pneumonia and acute MI.³¹ Adherence with infection control practices were not associated with lower perioperative infection rates (http://www.ncbi.nlm.nih.gov/pubmed/20571014). There was no association between compliance with home management care quality measures and ED visits and readmissions for pediatric asthma.³² Adherence to perioperative prophylaxis for VTE was not associated with VTE rates and surveillance bias resulted in hospitals that more rigorously looked for VTE to be unfairly penalized.³³ The growing list of quality measures often do not reflect the care they are supposed to measure, and capture only a narrow slice of hospital quality.^34,35 There is a need to reassess how these measures are designed and implemented, and to think more broadly about constructing meaningful quality measures tightly linked with patient outcomes.

In our analyses, the number of deliveries at a hospital and the availability of high level nursery facilities were associated with neonatal morbidity at term. These hospital characteristics have been consistently related to outcomes for very low birthweight infants.³⁶ The association is less clear for outcomes of lower risk infants.^37,38 Few studies have investigated how hospital characteristics affect severe maternal morbidity.

Our analysis has limitations. We used administrative (ICD-9 procedure and diagnosis codes) and birth certificate data rather than medical chart review for our analysis. Both birth certificate and SPARCS data may be limited by the reliability of certain types of information contained in these databases. Previous assessments demonstrated good reliability for age, race, and gestational age.³⁹ Previous studies of birth certificate data used for hospital quality assessment studies showed that they perform as well as medical records information.⁴⁰

Algorithms for morbidity based on diagnosis and procedure codes are imperfect. However, discharge data in obstetrics are reliable for delivery and procedure codes, and moderately reliable for comorbidities.^41,42 It is possible that the codes utilized in our analyses inadequately captured the outcomes of interest. However, many of the diagnosis codes included in the severe maternal morbidity algorithm (e.g. eclampsia, shock, acute renal failure) are designated as major complications and comorbidities codes (MCC) in DRG coding and when present may affect DRG assignment for obstetric deliveries, making these diagnoses less likely to be under reported. In a sensitivity analyses we found that coding for prolonged neonatal length of stay and NICU admission (outcomes known to be less subject to coding problems) were not associated with the two quality measures. If administrative codes inadequately captured the clinical information we assessed, we would have expected null findings in the patient-level analysis. This was not the case. We were able to demonstrate significant associations between age, race, parity, education, prenatal care, pregnancy type, and multiple comorbidities with severe maternal morbidity.

Although we have no reason to believe that systematic coding biases (e.g., the probability that a hospital under-codes maternal morbidity is a positive function of its cesarean or elective delivery rates) affected our results; the existence of random coding errors could bias our findings toward the null. Our estimate for severe maternal morbidity is higher than the recent national estimate of 1.6% that was derived using similar methods,⁴ but is consistent with data from New York City showing significantly higher than average maternal mortality rates,⁴³ and the rates of neonatal morbidity at term are consistent with previous findings.³ Further, without medical chart review we were unable to address the recent Joint Commission recommendation to exclude women with specific types of previous uterine surgery (e.g. classical cesarean delivery and myomectomy) from the measure on elective delivery before 39 weeks. We constructed risk-adjustment models that included multiple comorbidities and clinical conditions consistent with those in previous studies. Our sensitivity analyses examining the association of the quality measures with subcomponents of both outcomes measures (severe maternal morbidity without blood transfusion, prolonged neonatal length of stay, and NICU admission) reinforced our findings. By linking hospital discharge and birth certificate data we were able to control for maternal confounders, such as self-identified race, education and prenatal visits, associated with maternal and neonatal morbidity.

CONCLUSION

Performance on elective delivery before 39 weeks of gestation, cesarean deliveries performed on low-risk mothers and maternal and neonatal morbidity varied widely between New York City hospitals. The obstetric quality indicators we examined were not associated with lower morbidity. Our findings highlight the need for an expanded array of obstetric quality measures.