Distinguishing High-Performing from Low-Performing Hospitals for Severe Maternal Morbidity: A Focus on Quality and Equity

Abstract

Objective:

To investigate which organizational factors, policies, and practices distinguish hospitals with high compared with low risk-adjusted rates of severe maternal morbidity.

Methods:

Using a positive deviance approach, this qualitative study included 50 semi-structured interviews with healthcare professionals (obstetrics and gynecology chairs, labor and delivery medical directors, nurse managers, frontline nurses, physicians or nurses responsible for quality and safety, chief medical officers) in four low-performing and four high-performing hospitals in New York City. Hospital performance was based on risk-adjusted morbidity metrics from previous research. Major topics explored were structural characteristics (e.g. staffing, credentialing), organization characteristics (e.g. culture, leadership, communication, use of data), labor and delivery practices (e.g. use of standardized evidence-based practices, teamwork), and racial and ethnic disparities in severe maternal morbidity. All interviews were audiotaped, professionally transcribed, and coded using NVivo software. Researchers blinded to group assignment conducted qualitative content analysis. Researchers wrote analytic memos to identify key themes and patterns emerging from the interviews, highlight illustrative quotes, and draw qualitative comparisons between the two hospital clusters with different (but unrevealed) performance levels.

Results:

Six themes distinguished high- from low-performing hospitals. High- performing hospitals were more likely to have: 1) senior leadership involved in day-to- day quality activities and dedicated to quality improvement, 2) a strong focus on standards and standardized care, 3) strong nurse–physician communication and teamwork, 4) adequate physician and nurse staffing and supervision, 5) sharing of performance data with nurses and other frontline clinicians, and 6) explicit awareness that racial and ethnic disparities exist and that racism and bias in the hospital can lead to differential treatment.

Conclusion:

Organizational factors, policies, and practices at multiple levels distinguish high from low-performing hospitals for severe maternal morbidity. Findings illustrate the potential for targeted quality initiatives to improve maternal health and reduce obstetric disparities arising from delivery in low-performing hospitals.

Precis:

Organizational factors, policies, and practices at multiple levels distinguish high from low-performing hospitals for severe maternal morbidity.

Introduction

Rates of severe maternal morbidity (SMM), or unexpected outcomes of labor or delivery that have significant short- or long-term health impacts¹, have doubled over the last decade¹. Further, there is substantial hospital-level variation in risk-adjusted SMM. In New York City (NYC), a woman’s SMM risk during her delivery hospitalization varies seven-fold across hospitals even after accounting for patient case mix. This variation contributes to significant racial and ethnic disparities in SMM because Black and Latinx women more often give birth in hospitals with high SMM. Delivery hospital may explain as much as 48% of the Black-White disparity and up to 37% of the Latinx-White disparity in SMM in NYC hospitals^2,3.

Studies have identified hospital characteristics associated with SMM such as delivery volume, neonatal intensive care level, urban or rural location, hospital ownership, and percent of admissions covered by Medicaid^2,4,5. However, these factors leave much hospital-level variation in risk-adjusted SMM unexplained and are not easily modifiable, leaving hospitals with unclear direction for improvement. Further, few if any studies have garnered leadership and front-line staff input on maternal health disparities. In other disciplines, a positive deviance approach has been used to identify factors associated with higher performance and to improve quality of care⁶. A key postive deviance premise is that solutions to a problem lie within the members of the community involved, and that high-performing community members have knowledge and skills that can improve the performance of others⁷. This approach is uniquely suited to advance health equity research in its ability to provide context to and identify root causes of quantitative disparities in hospital performance⁸.

In this study, we applied a positive deviance approach^8,9 to learn which organizational factors, policies, and practices distinguish high from low-performing hospitals for SMM. We also aimed to explore views and actions hospitals did or did not take to recognize and address racial and ethnic disparities in maternal mortality and SMM.

Methods

The positive deviance framework^8,9 examines quantitative variation within a cohort (i.e., between hospitals or health care professionals) in performance on a specific measure, groups together high performers, and uses qualitative inquiry to understand characteristics that distinguish high from average or below-average performers. We applied this framework to characterize structural and organizational attributes of NYC hospitals with low rates of severe maternal morbidity. In prior quantitative research^2,3, we divided 40 NYC hospitals into three tertiles by their risk-adjusted SMM rate, an aggregate measure of hospital performance on maternal health that maps onto Black-White and Latinx-White disparities (Box 1). We defined high-performers as hospitals in the first tertile (low morbidity) and low-performers as hospitals in the third tertile (high morbidity)^2,3.

Box 1. Methodology for measuring and ranking hospital-level severe maternal morbidity rates.

We previously^2,3 measured severe maternal morbidity (SMM) in New York City (NYC) hospitals using vital statistics birth records linked to statewide hospital discharge data, containing International Classification of Diseases, ninth revision, Clinical Modification (ICD-9) diagnosis and procedure codes for the delivery hospitalization. We defined SMM using a published algorithm of diagnoses for life-threatening conditions (eg, renal failure, eclampsia) and procedure codes for life-saving procedures (eg, hysterectomy, ventilation, blood transfusion) defined the Centers for Disease Control and Prevention^1,55. We risk-adjusted hospital-level SMM for mothers’ sociodemographic characteristics (maternal age, self-identified race and ethnicity, parity, education, patient insurance coverage), prenatal care visits, clinical and obstetric factors (multiple pregnancy, history of previous cesarean delivery, body mass index), and diagnoses for patient risk factors that could lead to maternal morbidity but were likely present on admission to the hospital (eg, diabetes, hypertension, obesity, premature rupture of membranes, disorders of placentation). We used mixed-effects logistic regression with a random hospital-specific intercept to generate risk-standardized SMM rates for each hospital and ranked hospitals from lowest to highest risk-standardized SMM. We evaluated Black-White and Latina-White disparities in hospital use by comparing the cumulative distribution of births between racial and ethnic groups across hospitals ranked by risk-standardized morbidity. Additional detail on this methodology is provided elsewhere^2,3.

For the present qualitative study, we purposively sampled four hospitals from each of the previously defined performance clusters. We sampled from high and low performing clusters for sufficient contrast to suggest attributes that may lead to variation in quality. We aimed to include hospitals with similar delivery volume, percent Black or Latinx deliveries, and percent Medicaid deliveries across clusters. Because of our focus on disparities, we purposively selected hospitals with high proportions of Black or Latinx deliveries and aimed to have hospitals with high proportion of Black or Latinx deliveries in each cluster to help identify factors that distinguish high versus low performers in this context. Two hospitals declined to participate in this study and were replaced.

From September 2017 through October 2018, we conducted semi-structured interviews with six to eight clinicians and administrators in each hospital. Consultations with a scientific advisory group for this study, which included participation from community organizations, informed the research design and topics included in the interview guide. We introduced the study to chairs of obstetrics and gynecology by email and followed up with a phone call, during which the goals of the study were introduced, and potential participants (based on role) and site coordinators were identified. We aimed to interview similar roles across all hospitals. Those interviewed included chief medical officers (CMO); Chairs of Obstetrics & Gynecology, frontline labor and delivery (L&D) nurses; and physicians or nurses in charge of quality and safety for obstetrics. Each site visit was conducted by two to three researchers. The qualitative research team and interviewees were blinded to hospital ranking. All participants provided written informed consent; interviews were conducted in-person and were between 60 and 90 minutes long. Interviews followed a semi-structured discussion guide with open-ended questions (Appendix 1, available online at http://links.lww.com/xxx).

We used a published theoretical framework on hospital perinatal quality¹⁰ and explored the following topics: structural characteristics (e.g. physician and nurse staffing, credentialing), organization factors (e.g. culture, leadership, physician–nurse communication, staff-patient and family communication, quality improvement, training, quality data and how it is shared), labor and delivery (L&D) practices (e.g. use of evidence-based practices, teamwork), and questions about racial and ethnic SMM disparities (e.g. whether disparities occur and reasons for their existence). Interviews were audiotaped and transcribed, with all organizational and individual identifiers redacted. The Institutional Review Board of the Icahn School of Medicine at Mount Sinai approved the study protocol.

We used qualitative content analysis, a systematic process of coding and identifying themes and patterns in codes, to analyze qualitative interviews^11,12. Analysts were blinded to hospital performance. Researchers generated initial codes based on the interview protocols and transcripts. At the outset, two analysts coded the same five interviews to test the codebook for clarity and ensure consistency. After discussion of the initial coding exercise, codes and definitions were revised and clarified. Codes were compared over successive interviews to extract recurrent themes across the data. A total of 50 interviews were coded using NVivo 12 software.

After completion of coding, data analysts developed analytic memos to identify key themes and patterns emerging from the interviews, highlight illustrative quotes, and draw qualitative comparisons between the two hospital clusters with different (but unrevealed) performance levels. Memos were written on each topic (described above). Memos summarized results for individual hospitals, for the two groups of hospitals with different but unrevealed performance, and for all hospitals together.

Results

The study hospitals varied in teaching status, size, geographic location within NYC, and socioeconomic status of their obstetric population. Data use agreements with the New York State Department of Health and the New York City Department of Health and Mental Hygiene do not allow for characteristics of the eight hospitals to be presented. Table 1 summarizes hospital characteristics for the sample in the aggregate. Approximately 60% of interview respondents were female, White, and had been in practice > 20 years. The characteristics of the 50 interviewed hospital staff are listed in Table 2.

Table 1.

Characteristics of Eight New York City Study Hospitals

Characteristic	Mean or n (%)
Teaching hospital, n (% of hospitals)	8 (100.0)
Level 3 or 4 nursery, n (% of hospitals)	8 (100.0)
Delivery volume
Mean volume	3872
<2500 deliveries, n (% of hospitals)	4 (50.0)
≥2500, n (% of hospitals)	4 (50.0)
Medicaid coverage
Mean percent Medicaid births	62.7
<50% births, n (% of hospitals)	3 (37.5)
≥50% births, n (% of hospitals)	5 (62.5)
Race and ethnicity of obstetric population
Percent Black, mean	27.5
Percent Latinx, mean	31.3

Table 2.

Demographic characteristics of clinicians and administrators (n=50) interviewed at eight New York City hospitals

Characteristic	n (%)
Age
≤45	13 (26.0)
>45	37 (74.0)
Gender
Male	20 (40.0)
Female	30 (60.0)
Race a
Asian	5 (10.0)
Black	7 (14.0)
Hispanic or Latino	4 (8.0)
Multiracial or Middle Eastern	4 (8.0)
White	30 (60.0)
Ethnicity a
Hispanic or Latino	7 (14.0)
Non-Hispanic	43 (86.0)
>20 Years in Practice	32 (64.0)
>10 Years at Current Institution	28 (56.0)
Position
Chair of Obgyn	7 (14.0)
Physician Director of L&D	7 (14.0)
Physician or Nurse Quality & Safety Lead	6 (12.0)
Nurse Manager of L&D	8 (16.0)
Front Line L&D Nurse	7 (14.0)
Chief Medical Officer	7 (14.0)
Otherb	8 (16.0)

Obgyn=obstetrics and gynecology, L&D=labor and delivery

^aRace and ethnicity were self-reported by interviewees.

^bChief Quality Officer, Chief of MCH Nursing, Vice Chair Obgyn, MFM Attending.

There were several similar organizational factors and policies across hospital clusters. For example, all hospitals reported staffing and physical space challenges. Most hospitals used quality tools such as simulations, huddles, checklists, online modules, grand rounds, and communication tools such as SBAR¹³. In both clusters, frontline staff and nurses were less likely to have access to quality data than were physicians. None of the hospitals reported stratifying quality metrics by race and ethnicity.

Six themes distinguished high- from low-performing hospitals. High-performers were more likely to have: 1) senior leadership involved in day-to-day quality activities and dedicated to quality improvement, 2) a strong focus on standards and standardized care, 3) strong nurse–physician communication and teamwork, 4) adequate physician and nurse staffing and supervision, 5) sharing of performance data with nurses and other frontline clinicians, and 6) explicit awareness that racial and ethnic disparities exist and that racism and bias in the hospital can lead to differential treatment. We describe the domains and key themes, with representative quotations from study participants. Additional quotations are provided in Appendix 2, available online at http://links.lww.com/xxx.

Senior leaders in high-performing hospitals were more often involved in day-to-day activities (e.g., participating in rounds) and expressed a dedication to quality improvement. Frontline nurses and physicians perceived hospital leadership as accessible and responsive. Staff described how leadership engaged them in identifying quality issues, with monitoring oriented toward learning from adverse events, understanding system failures, and improving care. Conversely, clinicians in low-performing hospitals described leadership as disconnected from frontline staff and quality improvement work. In addition, while leadership turnover had occurred in both clusters, negative opinions about leadership changes were raised only in low-performing hospitals.

I know our hospital leadership believes and I believe… that the goal of hearing about errors is not to point the finger, assign blame, punish. It’s to understand what processes failed, what systems failed that allowed this individual to have made the error. That’s really where our focus is.

– CMO (high-performing cluster)

In the quality of care…when I was hired, I remember a young lady telling me specifically, “It’s about lack of leadership here. They need a strong leader”. – Chief Maternal-Child Health (MCH) Nursing (low-performing cluster)

Administrative and clinical staff in high-performing hospitals described more of a focus on quality measurement and standardization to reduce variation in obstetric care quality. Departmental and hospital leadership described having formalized processes for auditing clinical best practices and using quarterly physician report cards to motivate clinicians to improve their care.

I firmly believe in standardization as being key to quality and positive outcomes. In fact--that’s one of my main goals is to try to decrease variation in how care is provided for each diagnosis.

– CMO (high-performing cluster)

Administrators from low-performing hospitals, however, described staff resistance to standardization, and some clinicians discussed intuition and experience as equally or more valuable.

So, [standardization] it’s very important to me. I feel variability is the enemy of safety, and you’ve got to make things consistent. I’ve tried to drive as much as I can towards lack of variation…I’ve hit resistance every step of the way with both nursing and physicians…” – Obstetrics & Gynecology (Obgyn) Chair (low-performing cluster)

Hospital staff in both clusters discussed nurse–physician communication as an important aspect of patient safety but differed in how they perceived communication in their facility. Clinicians and administrators in high-performing hospitals often described a respectful and collaborative work environment in which nurses felt comfortable asking for clarification and voicing opinions to physicians and empowered to verbalize disagreement when necessary. There was also often a clear chain of reporting within and across departments in high-performing hospitals.

The most important thing that works for us is the teamwork. It’s interdisciplinary. The doctors ask me, they do respect what I think…We’re a team dynamic that makes us work well. – L&D Frontline RN (high-performing cluster)

Talking about it, discussing what I see, what you see, coming up with a plan. If either party is still not comfortable with that plan, it’s usually a second attending that’s called—the back-up and the charge nurse. Or the entire team may be called. And if still there’s concern, it’s escalated then to the chair. It’s escalated to nursing supervision. There’s nursing supervision here 24/7. – RN Manager L&D (high-performing cluster)

Low-performing hospital staff described communication deficiencies and tension between nurses and physicians. Nurses discussed feeling blamed for adverse outcomes and not respected as equal members of the care team.

…the nurse is always the first to be thrown under the bus. – L&D RN (low-performing cluster)

…the climate is bad. The nurses do not feel empowered to speak up to the physicians. They feel pushed in a corner. – L&D RN Manager (low-performing cluster)

High-performing hospitals had experienced and competent nurses, obstetricians, and specialist physicians.

We have good people, well-trained people, and very experienced people… One group always keeps an attending in house, so there are always three attending obstetricians here. There are always four residents here. There are always PAs here. There are always midwives here and not to mention anesthesiologists, medical students.... – Obgyn Chair (high-performing cluster)

Low-performing hospital administrators cited difficulty staffing sufficient generalist and specialist Obgyn physicians. Further, while hospitals in both clusters reported nurse shortages, respondents from high-performing hospitals discussed having resources within the facility to increase staffing to meet demand, while low-performing hospitals had to draw on agency nurses to fill staffing gaps.

We actually just succeeded in petitioning back for those lines to all be filled, but in the meantime we’ve been using a lot of agency, which itself is probably not one of the safest things unless the nurse really has great experience…. – RN Manager L&D (low-performing cluster)

Leadership at high-performing hospitals discussed processes and systems for sharing data with frontline clinicians. Data sharing occurred at multiple levels, including direct performance feedback to clinicians as well as wider departmental quality meetings and grand rounds. In one scenario described by the medical director of a high-performing hospital, data feedback led to improvement in the hospital’s episiotomy rate, an established measure of perinatal quality of care:

…we saw our rate was… higher than the national average. So we had a presentation in M&Ms [morbidity and mortality conference] and we gave the physicians the data for their own practice…and we saw that in one group [practice]…one person had an 18% [episiotomy rate] and another one had a 3%... we were able to show, at least infer, that a lot of it was done by rote...Our latest one [episiotomy rate] was eight or 10%, so we were able to come in the national average just by highlighting those - - educating everybody. – Medical Director L&D (high-performing cluster)

In contrast, respondents from low-performing hospitals did not describe data feedback, and some stated outright that there was little if any effort to communicate performance metrics to frontline staff. As one L&D nurse manager in a low-performing facility stated, “To be honest, I have not seen any data”.

When asked about racial and ethnic disparities in maternal health outcomes, participants from both hospital clusters spoke about social determinants of health such as immigration status, insurance coverage, and education, pointing to factors outside of hospital care as underlying causes. However, participants from high-performing hospitals were more likely to discuss racism and bias explicitly and recognized the potential for racism and bias in hospitals to contribute to differential medical care.

I think that every physician and provider has some bias, somehow. I mean, that’s human nature. Medical Director L&D (high-performing cluster)

Respondent: I deal a lot with the staffing…we have had people, attendings, who go to the clinic because we need them to go. And they clearly don’t want to be there. They don’t want to take care of those patients…so then they don’t go anymore…

Interviewer: So, I’m very curious. What is it that turns people off about working in that setting?

Respondent: I think racism is the main thing. – MD Quality & Safety (high-performing cluster)

In contrast, clinicians in the low-performing cluster denied that there was differential care in their facility by race, or solely attributed disparities to factors outside the hospital. They appeared to place more blame on patient-level factors such as individual health knowledge and behaviors.

I don’t see any evidence of racism in medicine. When somebody comes in they are treated just like the next person. We may categorize them in a higher risk than other patients but it’s not that every Black patient that comes in here is going to be considered high-risk…I don’t see any evidence of racism. – Obgyn Attending (low-performing cluster)

I think that some people...don’t seek the appropriate prenatal care…They’re not paying attention to their weight. They’re not paying attention to diabetes. You know, a family influence…”‘…– Chief MCH Nursing (low-performing cluster)

Discussion

We found that high- and low-performing hospitals for SMM faced similar organizational challenges, consistent with prior research investigating hospital quality in the perinatal setting^14,15. Despite these similarities, six themes emerged from our positive deviance approach that distinguished high- from low-performing hospitals. Given previous findings that delivery in low-performing hospitals plays a determining role in generating ethnic and racial inequity in SMM^2,3, the present study illustrates the potential for improvement in maternal outcomes as well as reduced inequity in outcomes through better care organization and practice.

Two of the themes identified in our study, senior management support for quality improvement and data feedback to clinicians, have been highlighted in positive deviance research in other fields of medicine.^6,16–18 Staff in high-performing hospitals described a dynamic learning process in which leadership engaged frontline clinicians in identifying and troubleshooting quality issues. High-performing hospitals also demonstrated a commitment to standardization to reduce variation in care, which is a focus of quality improvement in obstetrics. The Alliance for Innovation on Maternal Health (AIM) patient safety bundles propose standardized, evidence-based practices, including standards for managing severe hypertension and cardiac disorders in pregnancy, obstetric hemorrhage, and safe reduction of primary cesarean delivery¹⁹.

Another distinguishing theme, strong communication among clinical teams, is also emphasized in research on quality of care in obstetrics^20–28 and other specialties^29–34. Communication and teamwork deficiencies are among the strongest predictors of surgical errors ^35,36 and medical complications in adverse event or incident reports^31,37, and are key drivers of poor maternal and infant outcomes^38,39. In contrast, the literature on adequate staffing and supervision in obstetrics in scarce, although evidence from neonatology shows that staffing ratios play a critical role in patient safety and disparities^15,40–44, and that nurse staffing shortages are disproportionately prevalent in hospitals with a high volume of Black patients^15,43,44.

Although none of the hospitals had any focus on structured monitoring of quality indicators by race and ethnicity, awareness of racial and ethnic inequity distinguished the organizational culture in high-performing hospitals. To our knowledge, this attribute on a hospital level has not been previously identified, although bias in maternal health care delivery, both explicit biases and implicit or unconscious attitudes, has been documented extensively in the scientific literature and news media^45–47. Recently, the Centers for Disease Control and Prevention added discrimination and interpersonal and structural racism as contributing factors in their standardized data system for maternal death reviews³⁸. AIM and the Council on Patient Safety in Women’s Health Care developed a patient safety bundle to address peripartum disparities which includes recommendations for supporting bias research and trainings, addressing social determinants of health and structural racism in enhanced maternal mortality and severe maternal morbidity reviews, and promoting a culture of equity⁴⁸. The AIM bundle also encourages hospitals to monitor process and outcome metrics stratified by race and ethnicity and regularly disseminate stratified performance data to hospital staff and leadership⁴⁸. Further, there is legislative momentum toward building awareness and competence among health professionals to combat racism and discrimination in obstetric care⁴⁹.

Our results raise the hypothesis that hospital learning collaboratives focused on optimizing organizational practices and policies, increasing clinician and staff awareness and education on maternal health disparities, and addressing structural racism may be important tools for improving equity in maternal outcomes. Some health systems have adopted multipronged approaches to maternal health disparities reduction – with targets for clinical practice; faculty, staff, and trainee education; and research development – that may serve as models for others⁵⁰. Evaluating interventions such as the implementation of implicit bias trainings and disparities dashboards to track and report disaggregated perinatal metrics on labor and delivery units is a priority for future research.

Limitations of our study include the potential for social desirability bias to influence participants’ reporting of hospital culture, personnel, and organizational processes. However, respondents were not aware of their hospital’s ranking, and unlike similar health care research, we blinded qualitative data analysts to hospital performance. Data collected at a single point in time may not reflect most current hospital practices or changes over time. We were not able to explore fully the role of financial resources in risk-adjusted morbidity rates. We were limited in our ability to evaluate the influence of hospital demographics due to confidentiality restrictions. Our findings represent health care provider perspectives, and do not address hospital quality from patient or community viewpoints. Existing qualitative studies have focused on deficiencies in the patient experience that explain maternal health disparities^51–54. Further research applying a positive deviance lens to the patient perspective may help to identify features that differentiate high- versus low-performing hospitals. Finally, findings reflect the narratives of a small sample of staff in eight NYC hospitals and may not generalize more broadly.

We used a positive deviance framework and qualitative research paradigm to understand the role of hospital quality in maternal health disparities. Findings suggest that hospital policies and practices focused on improving nurse–physician communication, data sharing, standardization, adequate staffing, and building an antiracist culture may lead to improved and more equitable maternal outcomes.