Skip to main content
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2024 Jan 1.
Published in final edited form as: Qual Manag Health Care. 2022 Apr 4;32(1):8–15. doi: 10.1097/QMH.0000000000000347

Texas hospital’s perspectives about NICU performance measures: A mixed-methods study

Julie R Doherty a, Andrew Schaefer a, David C Goodman a
PMCID: PMC9530051  NIHMSID: NIHMS1736654  PMID: 35383729

Abstract

Background and Objectives:

This study was conducted to assess Texas hospital leaders’ perspectives about neonatal intensive care (NICU) performance measures.

Methods:

We conducted an explanatory mixed-methods study. First, we sent a survey and a copy of the Dartmouth Atlas of Neonatal Intensive Care to clinical and administrative leaders of 150 NICU’s in Texas. We asked respondents to review the chapter that reported Texas-specific results and respond to a variety of open and closed-ended questions about the overall usefulness of the report. Second, we conducted semi-structured qualitative interviews with a subset of survey respondents to better understand their perspectives.

Results:

The survey had a 50% hospital response rate. Respondents generally found the report to be interesting and useful and 87.7% of all respondents reported being in favor of receiving future reports with their own hospital’s data benchmarked against anonymous peers. All of the specific measures in the Atlas were found to be of interest and valuable, with NICU admissions and special care days rating among the most interesting and useful. In the semi-structured interviews, respondents expressed that a report with performance data would serve as a mechanism to drive change by identifying opportunities for improvement.

Conclusion:

Texas hospital NICU leaders are interested in routinely receiving more information about their own NICU’s performance anonymously benchmarked against their peers. This would facilitate a greater understanding of a unit’s functionality, as well as accelerate clinically appropriate quality improvement initiatives, which together have the potential to deliver better newborn care at lower costs for all Texans.

Keywords: performance reporting, neonatal intensive care, information dissemination, survey research

Introduction

Information on the relative performance of health care systems is instrumental to providers and policy makers for identifying possible areas in need of improvement and for tracking changes over time.1,2,3 Most performance measures have focused on adult populations where patient numbers and morbidity levels are high. Measurement of pediatric patient populations is less common, but includes some notable examples such as for very low birth weight newborns in the Vermont Oxford Network4,5 and those with Cystic Fibrosis by the Cystic Fibrosis Foundation.6 While these registries have detailed clinical information, they are limited to diagnosed patients receiving care at registry member providers.

Population-based measures can be particularly useful because they capture overall populations. Depending on the data source, populations can be enrolled prior to illness and observation can continue after receiving inpatient medical care. The drawbacks to this approach are that necessary data are often unavailable and sometimes do not include clinical information useful for risk adjustment.

This study reports Texas hospital stakeholder’s perspectives on receiving hospital-level measures of newborn medical care in a unique cohort of Texas Medicaid insured newborns.7,8 Both high and low risk newborn cohorts for the period of 2010 to 2014 served as the study populations regardless of NICU admission status. Observation began with maternal prenatal care and continued through the first year of life. The findings were disseminated through Texas state reports and stakeholder briefings, the Dartmouth Atlas of Health Care,9 and in journal papers.7,8 The study found marked regional and hospital variation in care, including NICU admission rates, number of special care days, and use of imaging services. This variation persisted after adjusting for the rich set of risk factors available in the data. Individual hospitals were not identified in publications.

The specific aim of this study was to assess measures in regard to the degree of interest and value to NICU providers. To do so, we used a mixed methods approach. We first surveyed clinical and administrative NICU leaders of hospitals in Texas and then conducted semi-structured interviews with a subset of respondents to better understand their views of newborn performance measure reporting. We had no a priori hypothesis but considered the possibility that while some might find the information valuable, others might consider it an unwelcomed intrusion in their medical practice.

Methods

Study Design & Sampling

Survey

We used an explanatory sequential design to inform our mixed-method study. Priority was placed on quantitative findings, where the key findings from the survey were used to inform follow-up topics for consideration in semi-structured interviews. The specific aim was to characterize the types of measures that hospital leadership would find most valuable, as well as the desired format and frequency of receiving future measures of NICU care benchmarked against anonymous peer hospitals.

The survey was developed by the study team, which included neonatal, perinatal, survey methods, and qualitative research experts. It was reviewed by external qualitative researchers to inform question order, format, and wording. We used a Texas Health and Human Services Commission hospital database to identify the hospital in our sample. The survey was sent to clinical and administrative NICU leaders of 150 of the 235 Texas hospitals. We excluded 80 hospitals that were accredited by the Texas Health and Human Services Commission as Level I or ‘Well newborn nurseries,’ where specialty care is not generally provided. An additional 5 hospitals were excluded due to unverified contact information. We then identified three contacts at each hospital to receive the survey: hospital CEO or senior leadership, Neonatal Medical Director, and NICU manager/director. The survey instrument included 26 items with a mixture of open and close-ended questions, allowing respondents to add additional comments as necessary. (See supplemental digital content 1; Survey instrument)

Interviews

To enhance the survey findings, we conducted semi-structured telephone interviews with survey respondents as the sample universe. Personalized emails were sent to all Neonatal Medical Directors and NICU managers/directors. While our original target was to interview 15-20 respondents, the beginning of the COVID-19 pandemic coincided with this phase of the project. Seven survey respondents agreed to the interviews. These included a mix of interviewees based on respondent type (i.e., NICU manager/directors and Neonatal Medical Directors), NICU level (i.e. II, III, and IV), and geographic hospital location in Texas (i.e. north, east, south, and west).

(See supplemental digital content 2; Interview guide)

Data Collection and Measures

Survey

In September 2019, survey packets were mailed to all contacts. Included was a copy of the Dartmouth Atlas of Neonatal Intensive Care (https://www.dartmouthatlas.org/atlas-neonatal-intensive-care) (Atlas), which contained a chapter with Texas hospital newborn utilization measures. We asked respondents to review the Texas chapter while completing the survey. Respondents received a small gift card for their efforts. Three additional paper mailings and four personalized emails were sent to each contact between September and December 2019.

The survey asked respondents to rate their interest in the measures presented in the Atlas (i.e. NICU admissions, number of special care days (SCDs), percent of SCDs billed as intensive, head MRI, chest film, abdominal film, and head ultrasound). Respondents selected from a 4-point Likert scale ranging from ‘very interesting,’ ‘somewhat interesting,’ ‘not very interesting,’ and ‘not at all interesting.’ A similar question followed the same structure that asked the degree to which each measure was useful. The questions were repeated with regard to very low birth weight and late preterm newborns.

Respondents were also asked to respond ‘yes’ or ‘no’ to the question: ‘Would a separate report of your hospital’s measures against anonymous peers be useful in planning or improving your hospital’s newborn care?” In a related question, respondents were asked: ‘Do you anticipate any of the information about NICU care provided in the Dartmouth Atlas report will lead to any of the following at your hospital?’ Respondents were able to select multiple responses which included ‘changes in clinical practice patterns,’ ‘changes in billing practices,’ ‘changes in staffing,’ ‘changes in educational programs for physicians and staff,’ ‘changes in educational program for families,’ and ‘other.’

Interviews

Semi-structured interviews were conducted by telephone between April - June 2020 and ranged from 25 to 45 minutes in length. The development of the interview questions and the interview guide was informed by the top survey findings focusing on each of the following key findings: 1. Respondents generally found the report interesting; 2. Some respondents were surprised or uncomfortable by information in the Atlas report; 3. Respondents asked for updated reports with their own hospital compared against anonymous peers; 4. Respondents indicated that the report would aid in clinical educational programs for staff and clinical practice improvement; and 5. Respondents found the NICU admission and special care day measures to be the most interesting and useful measures. Interviewees were also encouraged to offer any additional comments. The interviewer used the interview guide and accompanying materials to guide each of the interviews.

Data Analysis

Survey

Since we did not receive responses from each of the three target respondents for every hospital, we selected one survey from each responding hospital for the hospital-level analyses, prioritizing responses from the Neonatal Medical Director, followed by the NICU manager/director, and finally by the CEO and other senior leadership. This prioritization preferentially selected individuals with firsthand knowledge of newborn patient care and service organization. We also analyzed responses by respondent type, and stratified by hospital characteristics such as NICU level (II vs. III/IV), children’s hospital vs. not, and for profit vs. not-for-profit hospitals. To do so, we linked our survey data to the 2015 Texas American Hospital Association (AHA) survey for 90.4% of the responding and 87% of the non-responding hospitals. Differences of proportions were tested with Chi square tests and difference of means was tested with two sample student t-tests. All analyses were completed using STATA 14 software.

Interviews

The semi-structured interviews were recorded, professionally transcribed, and analyzed thematically using an inductive approach in SQR NVivo software. Two researchers independently coded each interview thematically, before reviewing together. Discrepancies were remedied through discussion until a consensus was reached. Once all interviews were thematically coded, individual themes were analyzed for subsequent sub-themes.

This study was approved by the Dartmouth College Institutional Review Board (IRB).

Results

We received survey responses from at least one contact at 73 of the 146 hospitals (response rate 50%) (Table 1). Four hospitals were excluded due to invalid mailing addresses. Of the 438 contacts who received surveys, 81 responded (18.5% response rate) with more than one response at 8 hospitals.

Table 1.

Characteristics of Texas hospitals included in the survey.

Responding
Hospitals
Non-
Responding
Hospitals
Respondent Characteristics N % N % P-value (1)
Unique Hospitals (N = 146) 73 50.0 73 50.0 NA
Response by contact type
 Hospital CEO 12 16.4 NA NA NA
 Neonatal Medical Director 28 38.4 NA NA NA
 Neonatal Manager/Director 33 45.2 NA NA NA
Hospital Characteristics
NICU Level (2)
 Level II 29 39.7 38 52.1 0.14
 Level III 30 41.1 29 39.7 0.87
 Level IV 14 19.2 6 8.2 0.05
Neonatal/Perinatal fellowship program (3) 4 5.5 1 1.4 0.17
Hospital links to AHA survey (4) 66 90.4 67 87.0 0.77
Hospital Type (4)
 government, non-federal 11 16.7 5 7.5 0.10
 non-government, not for profit 31 47.0 35 52.2 0.54
 investor-owned, for profit 24 36.4 27 40.3 0.64
Children's hospital (4) 6 9.1 2 3.0 0.14
NICU Managed by Contract (4) 3 4.5 4 6 0.71
Mean S.D. Mean S.D. P-value (1)
Total deliveries (4) 2411 2215 2550 2114 0.70
Total births (4) 2419 2173 2568 2127 0.68
Transfers - in (4) 52 157 31 96 0.33
Transfers - out (4)) 18 23 24 38 0.25
Delivered and NICU Admitted (4) 358 426 311 356 0.47
Total bed count (4) 375 231 322 212 0.15
(1)

Differences of proportions tested with Chi square tests; difference of means tested with two sample student t-tests.

(2)

Data from the 2018 Texas Health and Human Services Commission NICU Designations.

(3)

Data from the Public American Academy of Pediatrics.

(4)

Data from the 2015 Texas Hospital Association and the American Hospital Association (AHA) Annual Hospital Survey.

Survey and Interview Respondent Characteristics

As shown in Table 1, responding and non-responding hospital’s shared similar characteristics. The majority of survey respondents were NICU managers/directors (45.2%), closely followed by Neonatal Medical Directors (38.4 %), and finally hospital CEOs and other senior leadership (16.4%). Among responding hospital NICUs, 41.1% were classified as Level III, 39.7% as Level II, and 19.2% as Level IV. 86% of interviewees were Neonatal Medical Directors and 15% were NICU manager/directors. Interviewees represented hospitals from each of the geographic Texas locations (i.e. north, east, south, and west) and accounted for all NICU Levels (Level IV = 57%; Level III = 29%; Level II = 14%).

Individual themes from our data analysis are described below in subheadings.

Survey respondents and interviewees found the Atlas interesting and useful

Table 2 shows the percentage of respondents who found the measures and newborns in the study to be ‘very interesting’ and ‘very useful.’ Five of the seven measures were rated above 50% for ‘very interesting’ and three of the seven were rated as ‘very useful.’ 82.2% of respondents rated the late preterm newborn measures ‘very interesting’ and 72.6% rated them ‘very useful.’ (See supplemental digital content; Tables 4 and 5)

Table 2.

Responses about measures provided in the Dartmouth Atlas of Neonatal Intensive Care

Overall
Hospital
Respondent
(1)
Hospital Leadership
Respondent (2)
NICU Respondent (3)
N = 73 % N = 15 % N = 61 % P-value
(4)
Survey Question
Found measures very interesting (5)
NICU admissions 58 79.5 15 100.0 46 75.4 0.03
Number of Special Care Days (SCD) 47 64.4 11 73.3 39 63.9 0.49
Percent of SCDs billed as intensive 41 56.2 9 60.0 35 57.4 0.85
Head MRI 38 52.1 7 46.7 32 52.5 0.69
Chest Film 33 45.2 6 40.0 28 45.9 0.68
Abdominal film 31 42.5 5 33.3 27 44.3 0.44
Head ultrasound 40 54.8 7 46.7 34 55.7 0.53
Found measures very useful (6)
NICU admissions 46 63.0 10 66.7 38 62.3 0.75
Number of SCDs 38 52.1 8 53.3 32 52.5 0.95
Percent of SCDs billed as intensive 36 49.3 8 53.3 30 49.2 0.77
Head MRI 31 42.5 8 53.3 25 41.0 0.39
Chest Film 31 42.5 6 40.0 27 44.3 0.77
Abdominal film 28 38.4 6 40.0 24 39.3 0.96
Head ultrasound 37 50.7 8 53.3 31 50.8 0.86
Found newborn group very interesting (5)
Very low birth weight measures 53 72.6 14 93.3 41 67.2 0.04
Late preterm measures 60 82.2 12 80.0 50 82.0 0.86
Found newborn group very useful (6)
Very low birth weight measures 47 64.4 10 66.7 39 63.9 0.84
Late preterm measures 53 72.6 9 60.0 46 75.4 0.23
Felt surprised by some information in report 27 37 5 33.3 23 37.7 0.75
Would like a report with own hospital's results 64 87.7 15 100 52 85.2 0.11
Desired frequency of hospital specific report (% denominator = N of previous row)
Quarterly 24 37.5 4 6.3 20 31.3 0.65
Annually 24 37.5 7 10.9 19 29.7 0.26
Bi-annually 16 25 4 6.3 13 20.3 0.66
Report will aid changes in
clinical practices 32 43.8 5 33.3 29 47.5 0.32
billing practices 13 17.8 0 0 13 21.3 0.05
staffing 10 13.7 1 6.7 9 14.8 0.64
educational programs for clinicians 36 49.3 9 60 29 47.5 0.39
educational programs for families 12 16.4 2 13.3 12 19.7 0.57
(1)

One survey per hospital was counted, prioritizing survey responses from the neonatal medical director first, then NICU manager/director, and finally hospital leadership (i.e. hospital CEO or other senior leadership).

(2)

Survey responses from hospital leadership (i.e. hospital CEO or other senior leadership).

(3)

One survey per hospital was counted, prioritizing survey responses from the neonatal medical director first and then NICU manager/director second.

(4)

P-values comparing responses from hospital leadership with NICU responses (i.e. NICU manager/director and neonatal medical director) with Chi square test.

(5)

Responses from the 4-point Likert scale ranged from ‘very interesting,’ ‘somewhat interesting,’ ‘not very interesting,’ and ‘not at all interesting.'

(6)

Responses from the 4-point Likert scale ranged from ‘very useful,’ ‘somewhat useful,’ ‘not very useful,’ and ‘not at all useful.'

In the semi-structured interviews, interviewees characterized the report as a fair depiction of the regional and hospital variation that occurs in newborn care in Texas. One interviewee stated, “You could see variation on us. I liked how it [the Atlas] was analyzed on a city level. I could see the difference between how Fort Worth behaves regarding how they ordered tests and how care's deployed versus in Austin versus in Dallas versus in San Antonio.” Another interviewee effectively summarized this stating, “I think the survey just highlights the fact that different parts of the country do things differently.” Interviewees also elaborated on the usefulness of receiving this information, with one individual commenting, “Data is always very, very helpful to let us modify what we do or change what we do depending on outcomes.”

Survey respondents and interviewees would like to receive performance data about their NICU

87.7% of all survey respondents said they would be in favor of receiving additional reports including 90.9% of Level III/IV NICUs and 100% of children’s hospitals (Table 2) . When asked about the impact of such a report on their NICU, 49.3% of respondents reported that it would help facilitate educational program changes for clinicians and 43.8% said would aid in clinical practice changes.

In the semi-structured interviews, all interviewees indicated additional performance data reports would serve as a mechanism to drive change by identifying opportunities for improvement. One interviewee stated, “…Competition breeds excellence. You want to know how your neighbors are doing and find out what their strengths and weaknesses are, utilizing those techniques to help build your practice or learning from what they're doing, what their best practices are.” Another indicated that a report like this “…would help us prioritize quality improvement projects, help us identify things that we do or trends that we didn't even know about.” Finally, one interviewee commented on the report’s anonymous measure presentation by pointing out that without comparison data, it is difficult to accurately assess the unit’s performance, or the quality of care being delivered, stating, “I think you always want to see where you stand amongst your peers and how you can be better at what you're doing.”

Survey respondent and interviewee’s interest in specific newborn measures

With regards to specific measures, 79.5% of survey respondents rated the NICU admission measure as ‘very interesting’ and 63.0% rated it as ‘very useful.’ Similarly, 64.4% of respondents rated the number of SCDs measure as ‘very interesting’ and 52.1% rated it as ‘very useful’ (Table 2). Respondents rated the imaging measures such as head MRIs, chest and abdominal x-rays less interesting or useful.

In contrast, interviewees were particularly interested in the imaging measures and were in favor of receiving additional measures, citing that it could spark meaningful conversations about warranted and unwarranted care. One respondent stated, “I was fascinated by the regional differences, why there was so much imaging deployed for premature babies in certain parts of at least my state, and then in other parts they weren't imaging to the same degree… And I wondered why some regions might have those selection pressures to order more imaging versus others.” Another respondent asked, “Why is it that it takes 10 times as many chest X rays to take care of a 30-week premature baby in Fort Worth as it does to take care of that same baby in Dallas? Right, so what is that? That could lead to a series of interesting questions and observations. If you could have a nice transparent sharing of why that might be.”

Differences between responses by respondent type and NICU level

The differences in the viewpoint of hospital and NICU level respondents were not statistically significant (p < 0.05) with two exceptions. 100% of hospital leadership rated the NICU admission measure ‘very interesting’ compared to only 75.4% of NICU-level respondents rating it as such (P-value 0.03) (Table 2.) We also found that while 21.3% of NICU-level respondents stated that a report with their own hospital’s performance data could result in billing practice changes, 0% of hospital leadership agreed (P-value 0.05).

In stratified analyses, responses were similar between Level II and Level III/IV hospitals except that higher level NICUs were almost twice as likely to rate the very low birth weight newborn measures as being useful than lower level NICUs (77.3% and 44.8%, P-value 0.01). (Table 3)

Table 3.

Responses stratified by neonatal intensive care unit (NICU) level.

Level II NICU
Respondents (1)
Level III/IV NICU
Respondents (2)
N = 29 % N = 44 % P-value
(3)
Survey Question
Found measures very interesting (4)
 NICU admissions 23 79.3 35 79.6 0.98
 Number of Special Care Days (SCD) 20 69.0 27 61.4 0.51
 Percent of SCDs billed as intensive 17 58.6 24 54.6 0.73
 Head MRI 11 37.9 27 61.4 0.05
 Chest Film 11 37.9 22 50.0 0.31
 Abdominal film 9 31.0 22 50.0 0.11
 Head ultrasound 14 48.3 26 59.1 0.36
Found measures very useful (5)
 NICU admissions 16 55.2 30 68.2 0.26
 Number of SCDs 14 48.3 24 54.6 0.60
 Percent of SCDs billed as intensive 13 44.8 23 52.3 0.53
 Head MRI 11 37.9 20 45.5 0.53
 Chest Film 12 41.4 19 43.2 0.88
 Abdominal film 10 34.5 18 40.9 0.58
 Head ultrasound 12 41.4 25 56.8 0.20
Found newborn group very interesting (4)
 Very low birth weight measures 18 62.1 35 79.6 0.10
 Late preterm measures 23 79.3 37 84.1 0.60
Found newborn group very useful (5)
 Very low birth weight measures 13 44.8 34 77.3 0.01
 Late preterm measures 18 62.1 35 79.6 0.10
Felt surprised by some information in report 8 27.6 19 43.2 0.18
Would like a report with own hospital's results 24 82.8 40 90.9 0.30
Desired frequency of hospital specific report (% denominator = N of previous row)
 Quarterly 7 29.2 17 42.5 0.24
 Annually 9 37.5 15 37.5 0.79
 Bi-annually 8 33.3 8 20.0 0.34
Report will aid changes in
 clinical practices 11 37.9 21 47.7 0.41
 billing practices 3 10.3 10 22.7 0.18
 staffing 4 13.8 6 13.6 0.99
 educational programs for clinicians 20 69.0 16 36.4 < 0.01
 educational programs for families 4 13.8 8 18.2 0.62
(1)

One survey per hospital was counted from Level II NICU hospitals, prioritizing survey responses from the neonatal medical director first, then NICU manager/director, and finally hospital leadership (i.e. hospital CEO or other senior leadership).

(2)

One survey per hospital was counted from Level III/IV hospitals, prioritizing survey responses from the neonatal medical director first and then NICU manager/director second.

(3)

P-values comparing hospital responses with Level II compared with Level III/IV with Chi-square test.

(4)

Responses from the 4-point Likert scale ranged from ‘very interesting,’ ‘somewhat interesting,’ ‘not very interesting,’ and ‘not at all interesting.'

(5)

Responses from the 4-point Likert scale ranged from ‘very useful,’ ‘somewhat useful,’ ‘not very useful,’ and ‘not at all useful.'

Financial Implications of NICU performance data

A key theme of reporting NICU performance data concerned the hospital’s public image as it relates to hospital improving or threatening funding. One interviewee stated, “…if we're very efficient and we have good outcomes, will you support us more? There also could be negative implications in that, ‘Oh this group of hospitals are not doing very well.’ And is the state going to take a look at them and say, ‘Maybe you all should not be offering level four care, level three care?’”

The second theme explores the financial implications of hospital practice styles. One respondent asked, “If length of stay is important for an institution… a longer length of stay is more profitable, then is that going to impact our management? If length of stay is not profitable, is it going to affect safety?” It became clear in our interviews that there was a concern about how care measures could potentially affect the hospital’s financial status in either direction. Similarly, two interviewees voiced concerns about the possibility that contracted NICU management have overly aggressive practice styles and billing practices. One interviewee stated, “Their primary interest is not in being fiscally responsible and trying to save money or expenses for the patients and the hospital.”

Discussion

This study utilizes quantitative and qualitative instruments to assess Texas hospital clinical and administrative NICU leader’s perceptions of newborn performance measures, as well as their interest in receiving future measures of newborn care, particularly NICU admission rates and special care day measures. Respondents were strongly in favor of receiving reports with their hospital’s performance data benchmarked against peer hospitals and stressed the importance of receiving performance measures as a method to focus quality improvement initiatives and track improvement over time. Interviewees stated that in the absence of this information, their NICU would not be able to empirically understand their successes and shortcomings.

Our survey did not address the more sensitive issue of public reporting, which provides consumers, health care systems, and providers with information about the performance of named health care providers.10,11 Although not without criticism12,13,14,15, generally these measurement and reporting efforts can stimulate change10,16 ,17,18 and provide opportunities for greater consumer choice.3,19,11 The degree of provider (i.e. hospitals and clinicians) identification ranges from completely anonymous to full public reporting. In the Dartmouth Atlas report used in this study,9, we were limited to presenting only regional and hospital level analyses, without any provider identification.

Public reporting in the United States began in the late 1980’s when outcomes data for cardiac surgery, were compiled and released to the public.20,21 The New York State Department of Health became the first state-wide program to publish annual report cards for Coronary Artery Bypass Graft (CABG) surgeries at both the hospital and physician level, which resulted in a steep decline in CABG mortality rates, ranging from 33-60% in respective research groups.22 Today, approximately 95% of all cardiac surgery programs participate in the Adult Cardiac Surgery Database, which focuses on standardizing and improving cardiac care through public reporting.21

Although much of the research and application of public reporting is limited to adult populations, some neonatal initiatives have been successfully implemented. The Joint Commission began publicly reporting hospitals with high caesarian birth rates in July 2020,23 which marks an important step in monitoring maternal and newborn care outcomes. Selvaratnam, et. al. demonstrated a statistically significant improvement in Australian newborn outcomes, as well as a decreased rate of stillborn newborns after publicly reporting a quality indicator for identifying newborns born small for gestational age.24 The Vermont Oxford Network provides performance data benchmarked against anonymous peers and observed decreased newborn morbidity and mortality within member NICUs over an 8-year period.25

The Cystic Fibrosis Foundation has some similarities to the Vermont Oxford Network: It has a clinical registry of all patients receiving care in their network of Centers, and reports measures back to provider members, and promotes both research and clinical improvement. The Foundation also reports named Center measures to the public.26

Study Limitations and Generalizability

This study has limitations. First, the survey was completed by respondents reporting their own opinions, which are not necessarily aligned with a hospital leadership consensus viewpoint. Given our sampling strategy and the breadth of respondents’ job responsibilities, our results indicate at least one dominant viewpoint at each hospital. Similarly, respondents may have been more likely to complete our survey if they were either strongly in favor of- or strongly against receiving performance reports. Second, the qualitative interviews were conducted during the Coronavirus pandemic, which may have hindered our recruitment efforts. Similarly, we were unable to interview any hospital leadership, which leaves out an important perspective of newborn care. Finally, the study was limited to Texas hospitals and reported information about only Medicaid-insured newborns. While the generalizability of our findings to other providers is uncertain, Texas is a large state with close to 400,000 births each year27 with a very diverse population.

Conclusion

This study demonstrates that Texas hospital NICU leaders have a strong appetite for receiving performance measures regarding newborn care compared to anonymous peer hospitals.

Supplementary Material

Supplemental Digital Content_Survey
Supplemental Digital Content_Interview Guide
Supplemental Digital Content Table 1
Supplemental Digital Content Table 2

Funding:

This work was supported by the Agency for Healthcare Research and Quality's (AHRQ's) Comparative Health System Performance Initiative under Grant # 1U19HS024075, which studies how health care delivery systems promote evidence-based practices and patient-centered outcomes research in delivering care. The findings and conclusions in this article are those of the author(s) and do not necessarily reflect the views of AHRQ.

Footnotes

Conflict of Interest Disclosures (includes financial disclosures): The authors have no conflicts of interest relevant to this article to disclose.

References

  • 1.Lamb GC, Smith MA, Weeks WB, Queram C. Publicly reported quality-of-care measures influenced Wisconsin physician groups to improve performance. Health Aff (Millwood). 2013;32(3):536–543. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood). 2005;24(4):1150–1160. [DOI] [PubMed] [Google Scholar]
  • 3.Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care. 2009;47(1):1–8. [DOI] [PubMed] [Google Scholar]
  • 4.Edwards EM, Ehret DEY, Soll RF, Horbar JD. Vermont Oxford Network: a worldwide learning community. Transl Pediatr. 2019;8(3):182–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Horbar JD, Soll RF, Edwards WH. The Vermont Oxford Network: a community of practice. Clin Perinatol. 2010;37(1):29–47. [DOI] [PubMed] [Google Scholar]
  • 6.Cystic Fibrosis Foundation. Cystic Fibrosis Foundation Patient Registry 2018 Annual Data Report. 2019. Available at: https://www.cff.org/. Accessed June 27, 2020.
  • 7.Goodman DC, Ganduglia-Cazaban C, Franzini L, Stukel TA, Wassernab JR, Murphy MA, et al. Neonatal Intensive Care Variation in Medicaid-Insured Newborns: A Population-Based Study. J Pediatr. 2019;1:44–51. [DOI] [PubMed] [Google Scholar]
  • 8.Ganduglia Cazaban C, Kim Y, Goodman DC, Avritscher EB, Vogel B, Franzini L. Role of Prices, Utilization, and Health in Explaining Texas Medicaid Newborn Care Spending Variation. Med Care. 2019;57(2):131–137. [DOI] [PubMed] [Google Scholar]
  • 9.Goodman DC, Little G, Harrison W, Moen A, Mowitz ME, Canduglia-Cazaban C et al. The Dartmouth Atlas of Neonatal Intensive Care. The Dartmouth Institute of Health Policy & Clinical Practice, Geisel School of Medicine at Dartmouth; 2019. Available at: https://www.dartmouthatlas.org/atlas-neonatal-intensive-care/. Accessed June 27, 2020. [PubMed] [Google Scholar]
  • 10.James J. Public Reporting on Quality and Costs. Health Aff Health Policy Brief. March 8, 2012. [Google Scholar]
  • 11.Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA. 2005;293(10):1239–1244. [DOI] [PubMed] [Google Scholar]
  • 12.Vukovic V, Parente P, Campanella P, Sulejmani A, Ricciardi W, Specchia ML. Does public reporting influence quality, patient and provider’s perspective, market share and disparities? A review. J Public Health. 2017;27(6):972–978. [DOI] [PubMed] [Google Scholar]
  • 13.Berger ZD, Joy SM, Hutfless S, Bridges JFP. Can public reporting impact patient outcomes and disparities? A systematic review. Patient Educ Couns. 2013;93(3):480–487. [DOI] [PubMed] [Google Scholar]
  • 14.Damberg CL, Hyman D, France J. Do public reports of provider performance make their data and methods available and accessible? Med Care Res Rev. 2014;1(5 Suppl):81S–96S. [DOI] [PubMed] [Google Scholar]
  • 15.Chimonas S, Fortier E, Li DG, Lipitz-Snyderman A.Facts and Fears in Public Reporting: Patients’ Information Needs and Priorities When Selecting a Hospital for Cancer Care. Medical. 2019;39(6):632–641. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Krumholz HM, Normand S-LT. Public reporting of 30-day mortality for patients hospitalized with acute myocardial infarction and heart failure. Circulation. 2008;118(13):1394–1397. [DOI] [PubMed] [Google Scholar]
  • 17.Smith MD, Shannon M. To bring consumers to the table, simplify the review. JAMA Intern Med. 2013;173(14):1358. [DOI] [PubMed] [Google Scholar]
  • 18.Greenhalgh J, Dalkin S, Gibbons E, Wright J, Valderas JM, Meads D, et al. How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis. J Health Serv. 2018;23(1):57–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Haviland AM, Damberg CL, Mathews M, Paddock SM, Elliott MN. Shifting From Passive Quality Reporting to Active Nudging to Influence Consumer Choice of Health Plan. Med Care Res Rev. 2018;1:1077558718798534. [DOI] [PubMed] [Google Scholar]
  • 20.Campanella P, Vukovic V, Parente P, Sulejmani A, Ricciardi W, Specchia ML. The impact of Public Reporting on clinical outcomes: a systematic review and meta-analysis. BMC Health Serv Res. 2016;1:296. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Shahian DM. Public reporting of cardiac surgery performance: introduction. Ann Thorac Surg. 2011;1(3 Suppl):S1. [DOI] [PubMed] [Google Scholar]
  • 22.Hannan EL, Kumar D, Racz M, Siu AL, Chassin MR. New York State’s Cardiac Surgery Reporting System: four years later. Ann Thorac Surg. 1994;58(6):1852–1857. [DOI] [PubMed] [Google Scholar]
  • 23.Baker D. Public Reporting of High Cesarean Rates to Begin in July 2020. The Joint Commission; 2019. [Google Scholar]
  • 24.Selvaratnam RJ, Davey M-A, Anil S, McDonald SJ, Farrell T, Wallace EM. Does public reporting of the detection of fetal growth restriction improve clinical outcomes: a retrospective cohort study. J Obstet. 2020;127(5):581–589. [DOI] [PubMed] [Google Scholar]
  • 25.Horbar JD, Edwards EM, Greenberg LT, Morrow KA, Soll RF, Buss-Frank ME, et al. Variation in Performance of Neonatal Intensive Care Units in the United States. JAMA Pediatr. 2017;171(3). [DOI] [PubMed] [Google Scholar]
  • 26.Cystic Fibrosis Foundation. Available at: https://www.cff.org/. Accessed June 27, 2020.
  • 27.Matthews T, Hamilton BE. Total Fertility Rates by State and Race and Hispanic Origin: United States, 2017. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics National, Vital Statistics System; 2019. Available at: https://www.cdc.gov/nchs/products/index.htm. Accessed June 27, 2020. [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Digital Content_Survey
Supplemental Digital Content_Interview Guide
Supplemental Digital Content Table 1
Supplemental Digital Content Table 2

RESOURCES