Evaluating Risk-Adjusted Cesarean Delivery Rate as a Measure of Obstetric Quality

Abstract

OBJECTIVE

To validate the risk- adjusted cesarean delivery rate as a measure of obstetric quality through its association with maternal and neonatal outcomes for all pregnancies (model 1) and in singleton primiparous patients (model 2).

METHODS

We constructed a population-based cohort of 845,651 patients from 401 hospitals representing all deliveries in California and Pennsylvania between 2004 and 2005. We used linked birth certificate and hospital admission records for mother and infant to estimate the correlation between risk-adjusted cesarean delivery and a composite of adverse maternal outcomes, adverse neonatal outcome, and four obstetric patient safety indicators from The Agency for Healthcare Research and Quality (AHRQ).

RESULTS

In both models, risk-adjusted cesarean delivery rates were negatively correlated with both the maternal and neonatal composite outcomes and the AHRQ patient safety indicators for birth trauma, injury with instrumented vaginal delivery and cesarean delivery. Approximately 60% of the 107 hospitals with lower-than-expected risk-adjusted cesarean delivery rates had a higher-than-expected rate of at least one of the six adverse outcomes, compared to 19.6% of the hospitals with a higher-than-expected, risk-adjusted cesarean delivery rate and 36.1% of the hospitals with expected rates (p<0.001).

CONCLUSION

Lower-than-expected, risk-adjusted cesarean delivery rates in all patients or when restricted to a more homogeneous group of term singleton primiparous patients are associated with higher-than-expected adverse maternal or neonatal outcomes. Higher-than-expected risk-adjusted cesarean delivery rates do not result in improved outcomes.

INTRODUCTION

Despite the over 4 million deliveries in the United States each year¹ there are currently no uniformly accepted measures of obstetric quality.² A valid obstetric quality measure should have face validity, in which both obstetricians and patients believe that it measures the quality of obstetric care. The measure should also have construct validity demonstrating that hospitals that perform well on the quality measure of interest also perform well on other possible measures of quality. Additionally, the measure should be reproducible across different patient populations and across different time periods.²

The risk adjusted cesarean delivery (RACD) rate has historically been a proposed quality measure in obstetric care given its face validity, easy measurability and construct validity demonstrated in prior work where a high cesarean delivery (CD) rate at individual hospitals was associated with other markers of poor quality of care, such as infections, severe perineal lacerations, and neonatal complications.^2–4 However, there are several issues with RACD as a quality measure. First, obstetricians argue that using all CD in the measure is inappropriate, because in some situations CD is the standard of care. This criticism diminishes its face validity. Second, prior studies use data from nearly 10 years ago when the cesarean delivery rate was significantly lower;^3–7 these results have not been validated using more recent data or in additional states. Finally, these studies do not compare the association of a hospital’s RACD to other measures, such as the Agency for Healthcare Research and Quality (AHRQ) patient safety indicators, as additional measures of its construct validity. These studies also find those hospitals with a lower-than-expected cesarean delivery rate have higher rates of maternal infection, longer lengths of stay, and neonatal asphyxia than the hospitals in the expected rate group.^{3, 4} These results suggest that both higher and lower-than-expected rates may be associated with adverse maternal and neonatal outcomes, although more evidence is needed.

With these concerns, we seek to validate RACD as a measure of obstetric quality by measuring the correlation, or the statistical relationship, between RACD rate and important maternal and neonatal outcomes in recent data from multiple states. Combined with prior data, this new analysis will help measure the reproducibility of prior RACD results across different periods of time and across different states.

METHODS

Study Design and Cohorts

We evaluated RACD as a quality measure in 2 separate population-based cohorts of women to improve the face validity of the results. The first, general model includes all women in the dataset to evaluate RACD as a general quality metric for all delivering women. The second, restricted model includes only term singleton primiparous women without a history of a prior cesarean delivery. This second model is important because of biases inherent in the general model that is based on all deliveries. These biases include: 1) the role of patient choice in delivery mode after a cesarean delivery; 2) the differential ability of hospitals to perform vaginal birth after cesarean deliveries, 3) the increased speed of labor and the lower likelihood of parous women with prior vaginal delivery to have a cesarean section, and 4) the higher likelihood of fetal heart rate abnormalities and need for cesarean delivery in preterm infants. All analyses were performed on each cohort separately.

Data Source

We collected birth certificates from all deliveries occurring in California and Pennsylvania between January 1, 2004 and June 30, 2005 and linked to death certificates by each state’s department of health using name and date of birth. These linked records were then matched to maternal and newborn hospital discharge records using previous methods.⁸ California data were linked by the state department of health using established algorithms and Pennsylvania data were linked in a similar fashion internally at our center. Using these techniques, we matched over 98% of all birth certificates in the two states to maternal and newborn hospital records. The Institutional Review Boards of The Children’s Hospital of Philadelphia and the departments of health in California and Pennsylvania approved this study.

Birth certificates were excluded if they had a gestational age less than 23 weeks or greater than 44 weeks, a birth weight less than 400 grams or greater than 8000 grams, or if the birth weight was more than 5 standard deviations from the mean birth weight for the recorded gestational age in the cohort. Cesarean deliveries were identified from an ICD-9CM code of 669.7x in the maternal delivery record or a notation of a Cesarean section delivery in the birth certificate. Cesarean deliveries for previa, herpes, malpresentation & cord prolapse were excluded because medical standards of care support the delivery of these women via cesarean section. Hospitals with fewer than 50 deliveries were combined into two small hospital groups, one for California and one for Pennsylvania, because the small number of deliveries at each individual hospital result in less stable assessments of the outcomes at each individual hospital.⁹

Study Outcomes

To evaluate the construct validity of RACD, we measured the correlation between risk adjusted cesarean delivery rate and 6 outcome measures. A composite maternal outcome included wound infection (ICD-9CM codes 674.1x, 674.2x, 674.3x), post-delivery hemorrhage (ICD-9CM codes 641.3x, 641.8x, 641.9x, 660.0x, 660.1x, 660.2x, 660.3x, 667.1x), and blood transfusion (ICD-9CM codes 99.0 99.00 99.02 99.03 99.04). A composite neonatal outcome included neonatal death rate defined as any death during the initial birth hospitalization from death certificate records, neonatal asphyxia (ICD-9CM codes 768.5, 768.6, 768.9), birth injury ( ICD-9 codes 767.2, 767.4, 767.5, 767.6, 767.7, 767.8, 767.9 ), and neonatal seizure ( ICD-9 codes 779.0, 780.3, 780.39, 780.31). Four patient safety indicators (PSI) from the Agency for Healthcare Research and Quality (AHRQ) were also examined: birth trauma (PSI 17) (ICD-9CM codes 767.2, 767.4–767.9), injury with instrumented vaginal delivery (VD) (PSI 18), non-instrumented VD (PSI 19), or CD (PSI 20).

Our risk adjustment model included covariate variables based on their association with one or more study outcomes, the likelihood that a patient with these covariates would receive a CD, biologic plausibility, and previous work.^{5, 10} These variables included maternal comorbid conditions and neonatal congenital anomalies grouped by affected organ system. These maternal and neonatal comorbidities, shown in Table 1, were identified by ICD-9CM codes. The c-statistics for all patients (general model) and the restricted cohort were 0.866 and 0.616 respectively.

Table 1.

Comorbidity codes for risk adjustment

Comorbidities	Codes
Congenital Anomalies
GI	560.2, 750.3, 750.4, 750.5, 750.7, 750.8, 750.9, 751.0, 751.1, 751.2, 751.3, 751.4, 751.5, 751.60, 751.61, 751.69, 751.7, 751.8, 751.9, 756.70, 756.79, 777.1 and Birth Certificate
GU	753.0, 753.10, 753.12, 753.14, 753.15, 753.19, 753.2x, 753.3, 753.4, 753.6, 753.7, 753.8, 753.9, 756.71 and Birth Certificate
CNS	741.0x, 741.9x, 742.0, 742.1, 742.2, 742.3, 742.4, 742.59, 742.8, 742.9 and Birth Certificate
Pulmonary	519.4, 553.3, 748.3, 748.4, 748.6x, 748.8, 748.9, 750.6, 756.6 and Birth Certificate
Cardio	424.0, 424.1, 425.1, 425.3, 745.10, 745.11, 745.12, 745.19, 745.2, 745.3, 745.0, 745.60, 745.61, 745.69, 746.01, 746.09, 746.1, 746.2, 746.3, 746.4, 746.5, 746.6, 746.7, 746.81, 746.82, 746.83, 746.84, 746.85, 746.87, 746.89, 746.9, 747.1x, 747.21, 747.22, 747.29, 747.4x and Birth Certificate
Skeletal	756.50, 756.51, 756.55, 756.56, 756.59 and Birth Certificate
Chromosomes	758.3, 758.5, 758.89, 758.9, 759.4, 759.7, 759.89, 759.9 and Birth Certificate
Maternal Comorbidities
Disorders of Placentation	641.0x, 641.1x, 641.2x
Chronic Hypertension	642.0x, 642.1x, 642.2x
Cord Abnormality	663.0x, 663.1x, 663.5x
Preterm Labor	644.0x, 644.2x
PROM	658.1x, 658.2x
Chorioamnionitis	658.4x, 659.2x, 659.3x
GU Tract Infection	646.6x
PIH	642.4x 642.5x, 642.7x
Oligohydramnios	658.0x
Amniocentesis	75.1 and Birth Certificate
Cord Prolapse	663.0x, 762.4 and Birth Certificate
Blood Transfusion	99.0, 99.00, 99.02, 99.03, 99.04
Lupus	710.0
Other Collagen Vascular	710.1 710.2 710.3 710.4 710.5 710.8 710.9
Rheumatoid Arthritis	714.x

Data Analysis

We first calculated the expected number of RACD in the following manner. Logistic regression was performed using all of explanatory variables (comorbidities, complications, birth weight, year, etc). The model’s results were used to calculate the probability a given patient would receive a CD, known as the expected value. The expected values for each patient in a given hospital were summed to derive the expected rate of CD at that hospital. The expected number for the composite of maternal outcomes, neonatal outcomes, and each individual AHRQ PSI were also calculated using similar methods. We then compared the expected rate of each outcome to the observed rate of the outcome at each hospital by using the following formula:

$$\frac{(\text{observed}\text{number}\text{of}\text{events})-(\text{expected}\text{number}\text{of}\text{events})}{\text{Number}\text{of}\text{deliveries}}$$

After determining hospital-level differences between observed and expected RACD rates, we measured the construct validity of RACD in two ways. First, we performed a Pearson correlation analysis between RACD and each outcome measure, using the two models described. This correlation analysis evaluated the statistical relationships between two observed data values (RACD and 1 of the 6 maternal and neonatal outcomes). A positive correlation was signified by a positive coefficient value whereas a negative correlation was signified by a negative value. Second, we assigned hospitals into statistically lower-than-expected, as expected, and statistically higher-than-expected categories for RACD and each of the 6 outcome measures using previous methods of Haberman.^{11, 12} The Haberman method adjusts the standard error, and thus the statistical significance, of each (O-E)/N value to correctly account for the fact that the expected event model used the same patients as the observed values at each hospital. We then examined the relationship between RACD and the 6 outcomes using these rankings. These methods paralleled other methods to identify outlier hospitals in public reported data.⁹ All statistical analyses reported two-tailed P values with a statistical significance level of 5% after adjusting for multiple comparison testing using the methods of Bonferroni. All analyses were performed using SAS (version 9.2).

RESULTS

Hospital and Patient Demographics

A total of 401 hospitals were evaluated. Forty-two hospitals were grouped together into 2 small hospital groups (24 CA hospitals and 18 PA hospitals) leaving 361 hospital groups in our study. Initially, 957,438 birth records were identified for this project; 111,787 met the exclusion criteria, leaving 845,651 births in the final cohort in the overall model and 274,371 in the model restricted to term singleton primiparous patients. Table 2 demonstrates the average demographic characteristics for the patients by the general and restricted models.

Table 2.

Patient demographic characteristics in both models

Characteristic	Model 1(ALL)	Model 2(Term Primiparous Singleton)
	% or mean	%
Mean Age	28.00	25.61
Race-White	39.83	42.91
Black	6.48	5.92
Asian	8.57	10.24
Hispanic	41.59	36.75
Other	3.53	4.19
Mean gestational age at delivery	38.83	39.40
Term	84.08	100.00
Primiparous	38.23	100.00
Prior CD	14.93	0.00
Multiples	1.55	0.00
Chronic Hypertension	0.77	0.60
Pre-gestational Diabetes	0.75	0.50
Gestational Diabetes	5.19	3.97

Construct Validity

Table 3 shows the Pearson’s correlation coefficients between the corresponding RACD model and each adverse outcome. For both cohorts, there was a negative correlation between RACD and each of the 6 outcomes, which ranged from a Pearson’s correlation coefficient of −0.08 to −0.38. The maternal composite outcome was most negatively correlated with RACD in both models. These correlations were all statistically significant except for the correlation between RACD and PSI 19, injury with non-instrumented vaginal delivery. With respect to individual outcomes, there was a statistically significant correlation between RACD and maternal hemorrhage (r=−0.369, p<0.001) and wound complications (r=−0.342, p<0.001).

Table 3.

Correlation analyses between RACD and maternal and neonatal outcomes and AHRQ Patient safety indicators

	Model 1 (95% CI)	P Value	Model 2 (95% CI)	P Value
Maternal Composite	−0.383 (−0.49,−0.28)	<.0001	−0.324 (−0.41,−0.24)	<.0001
Neonatal Composite	−0.208 (−0.31, −0.10)	<.0001	−0.165 (−0.29, −0.04)	0.002
PSI 17(Birth trauma)	−0.140 (−0.23, −0.05)	0.008	−0.107 (−0.19, −0.02)	0.042
PSI 18 Injury with instrumented VD)	−0.172 (−0.33,−0.02)	0.001	−0.132 (−0.28,0.01)	0.012
PSI 19 (Injury with non-instrumented VD)	−0.080 (−0.20,0.04)	0.130	−0.054 (−0.17, 0.07)	0.307
PSI 20 (Injury with CD)	−0.185 (−0.27,−0.10)	<.0001	−0.163 (−0.25, −0.08)	0.002

We next determined which hospitals had a significantly higher or lower-than-expected rate of RACD for each model and each of the 6 adverse outcomes. We then identified the RACD status for hospitals that had higher-than-expected rates of each of the 6 additional adverse outcomes. As with the correlation analysis, hospitals with higher-than-expected rates of the 6 adverse outcomes were more likely to have lower-than-expected RACD rates in both the general cohort and the restricted cohort. In the general cohort, 59.8% of the 107 hospitals with lower-than-expected RACD rates had a higher-than-expected rate of at least one of the six adverse outcomes, compared to 19.6% of the 102 hospitals with a higher-than-expected RACD rate and 36.1% of the as expected group. A similar result was seen with the restricted cohort (Table 4). Compared to the expected RACD group, a statistically similar percentage of hospitals with higher-than-expected rates of RACD had higher-than-expected rates of the other six outcome measures. Average hospital level characteristics for hospitals in the lower-than-expected, as-expected, and higher-than-expected RACD rate groups are shown in Table 5.

Table 4.

Habermans tests to compare rankings for RACD and 6 outcomes in both models

		Percentage of Outcome Higher-than-Expected
		Maternal Composite	Infant Composite	PSI 17	PSI 18	PSI 19	PSI 20
Model 1	Lower -than -Expected (N=88)	29 (33.0%)	9 (10.2%)	14 (15.9%)	22 (25.0%)	21 (23.9%)	10 (11.4%)
	Expected (N=187)	30 (16.0%)	11 (5.9%)	8 (4.3%)	15 (8.0%)	30 (16.0%)	5 (2.7%)
	Higher -than -Expected (N=86)	3 (3.5%)	2 (2.3%)	4 (4.7%)	5 (5.8%)	8 (9.3%)	0 (0.0%)
	p-value	< 0.001	0.04	0.001	< 0.001	0.001	< 0.001
Model 2	Lower -than -Expected (N=53)	22 (41.5%)	6 (11.3%)	7 (13.2%)	17 (32.1%)	11 (20.8%)	8 (15.1%)
	Expected (N=238)	37 (15.6%)	15 (6.3%)	18 (7.6%)	20 (8.4%)	39 (16.4%)	7 (2.9%)
	Higher -than -Expected (N=70)	3 (4.3%)	1 (1.4%)	1 (1.4%)	5 (7.1%)	9 (12.9%)	0 (0.0%)
	p-value	< 0.001	0.13	0.04	< 0.001	0.14	< 0.001

^*p-values calculated by the Haberman method

Table 5.

Hospital characteristics by RACD rate group^*

Characteristic	Lower-than-expected (N=87)	As expected (N=186)	Higher-than-expected (N=86)	p-value
NICU Level				0.31
Level I	37 (42.5%)	105 (56.5%)	55 (64.0%)
Level II	7 (8.1%)	11 (5.9%)	6 (7.0%)
Level IIIA	8 (9.2%)	13 (7.0%)	4 (4.7%)
Level IIIB	15 (17.2%)	33 (17.7%)	10 (11.6%)
Level IIIC+	20 (23.0%)	24 (12.9%)	11 (12.8%)
Rural location	6 (6.9%)	28 (15.1%)	3 (3.5%)	0.014
Teaching hospital	14 (16.1%)	16 (8.6%)	2 (2.3%)	0.006
Median delivery volume, 18 months [Inter-quartile range]	2564 [1254–4080]	1496 [662–2674]	1935 [1278–3639]	< 0.001

^*Characteristics do not include 2 small volume hospital groups (N=359 instead of 361)

DISCUSSION

Many clinical fields have greatly accelerated their efforts to improve safety and quality, using techniques such as performance measurement, regionalization and specialization, and communication of best practices. Although the use of RACD rate as an obstetric quality metric has been promising, it has several deficiencies: its lack of acceptance by the provider community, the continued rising rate, and the influence of factors outside of the hospital’s control, such as a patient’s choice to demand a CD. Similar to prior work,^{3, 4} our results in a cohort of over 360 hospitals with almost 1 million deliveries suggest that hospitals that perform too few cesarean deliveries have higher-than-expected rates of other outcomes, such as maternal and neonatal complications. We speculate that in some instances, patients benefit from having a cesarean section, and that \ hospitals who do not act fast enough to perform a cesarean delivery may have a higher rate of adverse outcomes. Additionally, although hospitals with higher-than-expected RACD do not have higher-than-expected rates of other adverse outcomes, performing more RACD was not associated with improved outcomes. This overuse of medical health care may have significant negative consequences for many women,^13–15 and results in higher costs to patients and society.

Our findings in regards to the use of RACD rate as a quality measure should be placed into the context of the prior literature. Initial measures of obstetric quality used the raw cesarean delivery rate, which was undesirable given the lack of risk adjustment, and maternal mortality rate, which is a rare occurrence. RACD was promising since it is easily measured, and historically a high rate had excellent construct validity given its association with poor maternal and neonatal outcomes.^{3, 4} Conversely, a lower-than-expected RACD rate was also associated with poorer neonatal outcomes in three separate studies.^{4, 7, 16} These findings suggest RACD may be a possible quality measure, either higher or lower-than-expected.

There are several plausible explanations for our findings that demonstrate an association between worse maternal and neonatal outcomes and AHRQ PSIs when RACD rate is lower-than-expected, but no difference when the RACD is higher-than-expected. First, in many clinical settings, outcomes improve with volume. The increasing volume of cesarean delivery^{1, 17} may lead to a decline in complications such as hemorrhage and infection. Second, improvements in sterilization techniques and operative room procedures may result in a lack of correlation between infection and higher-than-expected cesarean rate in our study. Third, changes in practice patterns due to external, non-medical forces such as the liability climate and cesarean delivery by maternal request may increase the number of CD for non-medical reasons.¹⁸ In each case, secular changes in the use of cesarean section may have led to our results differing from prior work.

Performing too many cesarean deliveries was not associated with adverse maternal and neonatal outcomes nor was it associated with improved outcomes or protective effects. This lack of a correlation between a higher-than-expected CD rate and adverse outcomes should not suggest that a higher-than-expected cesarean rate is desirable. Specifically, it likely reflects an over-utilization of medical care and the performance of unnecessary procedures. This fact may represent a different aspect of poor quality that is not being measured through these methods. For example, Medicare patients residing in geographic areas with high intensity practice patterns, i.e. those areas with high quantity of health care services per capita, have been shown to have a higher 5 year mortality rate with conditions such as myocardial infarction and colorectal cancer, reflecting a lower quality of care.^19–21 Further, in a cross sectional analysis of Medicare enrollees, Fisher and colleagues demonstrated that in areas with greater hospital bed capacity, there is increased hospital utilization without detectable mortality benefit.²²

Finally, the time period of our study reflects the increasing cesarean rate compared to previously published literature. Using data from previous years to construct our expected rate of cesarean section, when the initial studies were performed, would drastically change our results: with this older data as the baseline, almost all hospitals would have higher-than-expected RACD rates in 2004–2005. Therefore the metric of RACD would not differentiate hospitals of different levels of quality. This result shows the importance of continually testing the construct validity and reproducibility of any obstetric quality measure.

This study has several limitations. First, we identified comorbidities and complications using administrative data including ICD-9 codes, not primary chart abstraction. This method allows for the inclusion of a population-based cohort of patients not otherwise obtainable. However, some comorbidities may have been under-coded or over-coded, resulting in a misclassification bias. Second, although the results are adjusted for a large number of observable factors, unobserved changes in the maternal population may be responsible for the observed changes in outcomes. The concurrent observation of lower-than-expected RACD rates and higher-than-expected adverse outcomes does not prove causality. However, the observation that the majority of hospitals with a lower-than-expected RACD rate have higher-than-expected rates for 1 or more adverse outcomes suggests that an association is present.

In conclusion, in a cohort of 361 hospitals, a lower-than-expected RACD rate is associated with a higher-than-expect rate of several adverse maternal and neonatal outcomes. However, with the rapidly rising CD rate, performing too many cesarean deliveries is not beneficial and may have significant negative future reproductive consequences to women.^13–15 The lack of an association between higher-than-expected RACD rates and higher rates of adverse obstetric outcomes suggests that RACD may not be a sustainable measure of obstetric quality. These findings underscore the importance of developing novel ways to measure all aspects of obstetric quality to ensure that women are receiving the safest and most effective obstetric care available.