Skip to main content
NCBI home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2018 May 24.
Published in final edited form as: Matern Child Health J. 2016 Nov;20(11):2239–2246. doi: 10.1007/s10995-016-2105-y

Importance of Performance Measurement and MCH Epidemiology Leadership to Quality Improvement Initiatives at the National, State and Local Levels

Kristin M Rankin 1, Loretta Gavin 2, John W Moran Jr 3, Charlan D Kroelinger 4, Catherine J Vladutiu 5, David A Goodman 4, William M Sappenfield 6
PMCID: PMC5967243  NIHMSID: NIHMS965468  PMID: 27423235

Abstract

Purpose

In recognition of the importance of performance measurement and MCH epidemiology leadership to quality improvement (QI) efforts, a plenary session dedicated to this topic was presented at the 2014 CityMatCH Leadership and MCH Epidemiology Conference. This paper summarizes the session and provides two applications of performance measurement to QI in MCH.

Description

Performance measures addressing processes of care are ubiquitous in the current health system landscape and the MCH community is increasingly applying QI processes, such as Plan-Do-Study-Act (PDSA) cycles, to improve the effectiveness and efficiency of systems impacting MCH populations. QI is maximally effective when well-defined performance measures are used to monitor change.

Assessment

MCH epidemiologists provide leadership to QI initiatives by identifying population-based outcomes that would benefit from QI, defining and implementing performance measures, assessing and improving data quality and timeliness, reporting variability in measures throughout PDSA cycles, evaluating QI initiative impact, and translating findings to stakeholders. MCH epidemiologists can also ensure that QI initiatives are aligned with MCH priorities at the local, state and federal levels. Two examples of this work, one highlighting use of a contraceptive service performance measure and another describing QI for peripartum hemorrhage prevention, demonstrate MCH epidemiologists’ contributions throughout. Challenges remain in applying QI to complex community and systems-level interventions, including those aimed at improving access to quality care.

Conclusion

MCH epidemiologists provide leadership to QI initiatives by ensuring they are data-informed and supportive of a common MCH agenda, thereby optimizing the potential to improve MCH outcomes.

Keywords: Quality improvement, Performance measurement, MCH epidemiologists, Contraceptive services Peripartum hemorrhage

Purpose

In response to the goals of the Patient Protection and Affordable Care Act (2010) and the recent Title V Block Grant transformation to improve population health through prevention and quality care, the focus in maternal and child health (MCH) on measuring and improving clinical and population health indicators has intensified (Rosenbaum 2011; Lu et al. 2015). Indicators such as Title V performance measures, the Children’s Health Insurance Program Reauthorization Act (CHIPRA) measures (CMS 2015), and other Healthcare Effectiveness Data and Information Set (HEDIS) measures provide standardized methods for monitoring quality, assessing variability across and within states or clinical settings, and identifying targets for improvement (Bethell 2011; CMS 2015; Lu et al. 2015).

Quality Improvement (QI) in public health is defined as a continuous and ongoing effort to achieve measurable improvements in health equity and population health by applying a defined and deliberate process to improve the efficiency, effectiveness, performance, or accountability in processes and services that impact health outcomes (Riley 2010). Developing and reporting clearly-defined performance measures is critical for successfully identifying processes in need of QIand monitoring change over time as improvement strategies are tested.

In recognition of the role of MCH epidemiologists in performance measurement and QI, a plenary session at the 2014 CityMatCH Leadership and MCH Epidemiology Conference was dedicated to discussing the use of performance measures to inform QI initiatives at the local, state and federal levels. The purpose of this article is to summarize the plenary session by providing an overview and two examples of performance measurement and QI in MCH. We will also highlight MCH epidemiologists’ unique contributions to these activities. The article closes with future challenges as the QI framework is applied more frequently to population-based MCH outcomes.

Description

Performance Measures

Performance measurement and QI are embedded within the Public Health Performance Management System (Public Health Foundation 2004, Fig. 1). Performance measures are ubiquitous in the current health system landscape. The National Quality Forum (NQF) has developed a process for applying consensus standards to evaluate and catalog these quality measures. The NQF is a not-for-profit organization whose endorsement is viewed as essential before a performance measure is adopted by health systems, such as CMS or community health centers (NQF 2016a). Performance measures must meet the following five criteria before achieving NQF endorsement (NQF 2016b):

  • Importance to Measure & Report Is evidence-based and linked to a health outcome; represents a national health goal or priority with a demonstrated gap in performance or room for improvement.

  • Scientific Acceptability of Measure Properties Produces reliable and valid results about quality of care.

  • Feasibility Uses readily available data or could be captured without undue burden.

  • Usability and Use Is clear and usable by potential audiences (e.g., consumers, purchasers, providers, policymakers) for accountability and improvement.

  • Comparison to Related or Competing Measures Is deemed distinct from other measures after considering similarities or differences with previously existing measures.

Fig. 1.

Fig. 1

Open in a new tab

Public health performance management system

Whether or not formal NQF endorsement is sought, consideration of these criteria facilitates the creation of clearly-defined measures of performance and outcomes for use in monitoring population health over time, encouraging accountability in health systems, and targeting issues for QI efforts.

Quality Improvement (QI)

QI is based on the tenet that a system left unchanged will only continue producing the same results and that change is necessary for improving health outcomes. The QI process has several steps, including: forming a team of stakeholders; setting aims; establishing measures; selecting change strategies; testing strategies; and finally implementing and spreading successful strategies (Langley et al. 2009). Data are needed throughout the QI process, from identifying and prioritizing opportunities to testing improvement strategies and deciding whether to adopt, adapt or abandon strategies. One popular technique for testing strategies consists of rapid Plan-Do-Study-Act (PDSA) cycles, which are supported by timely reporting of selected performance measures (Fig. 2). Tools such as run or control charts support PDSA cycles by displaying changes in performance measures over time during a QI initiative and highlighting important variations in performance that should be further investigated (Provost and Murray 2011).

Fig. 2.

Fig. 2

Open in a new tab

Plan-Do-Study/Check-Act (PDSA) Cycle

While QI has historically been applied in hospital and clinical settings, public health has more recently engaged in QI efforts, both through adopting a more active role in clinical QI and through applying QI principles to population-based health indicators. In MCH, public health partners have most notably engaged in clinical QI through statewide perinatal quality collaboratives (CDC 2015). At the population level, the application of QI to MCH performance measures has been accelerated by the Collaborative Improvement and Innovation Network (CoIIN) to Reduce Infant Mortality (HRSA 2016). Launched in 2012 in Public Health Regions IV and VI and since expanded nationwide, CoIIN combines the science of collaborative learning with QI processes, including rapid PDSA cycles to design, implement, spread, and scale innovative programmatic and policy change strategies to improve birth outcomes. The Title V Block Grant Transformation also adds a QI lens to MCH practice through a new framework of evidence-based or evidence-informed strategies that state MCH programs will develop, test, and refine in PDSA cycles with the goal of improving national and state performance measures (Lu et al. 2015).

Assessment

Contributions of MCH Epidemiologists to the Application of Performance Measurement in QI

QI encompasses a series of activities similar to the MCH planning cycle (MCH Navigator 2014), only carried out more rapidly and on a smaller scale. MCH epidemiologists have intimate knowledge of the planning cycle and are therefore well-positioned to provide leadership at multiple stages of QI, including:

  • Identifying population-based MCH outcomes for which QI initiatives are needed;

  • Assessing available evidence to identify promising improvement strategies;

  • Selecting or defining performance measures;

  • Choosing valid and reliable data sources for performance measures;

  • Assessing and improving data quality;

  • Designing reports to monitor progress;

  • Measuring variability in an indicator and identifying potential sources of variability;

  • Assessing and minimizing bias in measures

  • Monitoring data for unintended consequences of QI initiatives;

  • Evaluating the population impact of QI initiatives; and

  • Translating results to stakeholders.

More specifically, MCH epidemiologists lend methodological expertise to striking a balance between timeliness and accuracy when selecting a data source, evaluating the impact on data quality of using more timely provisional data and assessing whether the consequent decrease in accuracy is acceptable. They also can identify and address potential sources of bias that may explain observed variation in performance over time or across subgroups. The use of historical (rather than external) comparison groups for most QI initiatives makes results particularly prone to bias, so findings must be appropriately interpreted and translated in light of those biases. Similarly, when stakeholders raise questions about the suitability of applying stratification or risk-adjustment techniques (e.g. to control for payer or patient mix), MCH epidemiologists can estimate the magnitude of confounding and critically evaluate with stakeholders whether accounting for confounding or heterogeneity will support the ultimate goals of the QI initiative.

More broadly, MCH epidemiologists provide leadership to selecting and addressing priorities for their jurisdictions through the Title V Block Grant, and therefore can ensure that QI initiatives are well-aligned with this common MCH agenda and that stakeholders are engaged in the QI process. Overall, MCH epidemiologists have the knowledge and skills to be leaders in performance measurement and QI initiatives to address clinical or population-based health indicators.

Illustrative Examples of Performance Measurement and QI in MCH

Following are two examples illustrating the use of performance measures and QI to address current priorities in MCH and the role of MCH epidemiologists in these efforts. The first describes the development and implementation of a performance measure for effective contraceptive provision and the second describes a QI initiative to prevent peripartum hemorrhage.

Example 1: A Contraceptive Performance Measure for Title X Clinics and Beyond

More than one-half of pregnancies, or over 3 million per year, are unintended (Finer and Zolna 2014), which has important consequences for infant health, maternal health, and the life course trajectory of mothers and children (Gipson et al. 2008; Cheng et al. 2009). Unintended pregnancy is particularly prevalent among the approximately 700,000 teens aged 15–19 years that become pregnant annually (Ventura et al. 2012). Furthermore, only one-half of pregnancies are spaced to optimize maternal and infant outcomes (Gemmill and Lindberg 2013).

Using effective contraceptive methods is recommended for preventing unintended pregnancies and improving pregnancy spacing by the American Congress of Obstetricians and Gynecologists (ACOG), the Centers for Disease Control and Prevention (CDC), and the Office of Population Affairs (OPA) (ACOG 2009; CDC & OPA 2014). Increasing access to the most effective methods of contraception is a promising strategy for expanding women’s choices and improving population health because a woman’s probability of experiencing unintended pregnancy is strongly associated with the type of contraceptive used. Failure rates during typical use range from <1 % over a year for the most effective methods (e.g., male and female sterilization, IUD and implant), to 6–12 % for moderately effective methods (e,g., injectable, pill, patch, ring and diaphragm), and 18–28 % for the least effective methods (e.g. condoms, withdrawal, sponge, rhythm, spermicide) (Trussell 2011).

To inform the delivery of family planning services throughout its national network of approximately 90 Title X Family Planning Program grantees operating 4200 service sites and serving approximately 5 million clients annually (OPA 2016), OPA is developing new clinical performance measures designed for tracking and supporting efforts to increase access to effective methods of contraception. The primary performance measure estimates the percentage of women at risk of unintended pregnancy who are using a most or moderately effective method of contraception. Women at risk of unintended pregnancy are defined as those who are neither pregnant nor seeking pregnancy, are fecund and have ever had sex. The measure represents an intermediate outcome since it quantifies the result of the clinical encounter with respect to contraceptive provision, which should ultimately impact the longer term outcome of unintended pregnancy. It is a marker of quality under the assumption that increased uptake of most or moderately effective contraceptive methods among women at risk of unintended pregnancy represents high (versus low) quality contraceptive services.

Figure 3 demonstrates that performance on this measure among all Title X grantees (aggregated by state) varied widely in 2013, with twenty states or jurisdictions reporting values of 80 % or higher but seventeen states reporting values of 70 % or lower, five with fewer than 60 % of their eligible clients using a most or moderately effective method. These results indicate that there is substantial room for improvement in this measure for some Title X-funded grantees.

Fig. 3.

Fig. 3

Open in a new tab

Percentage of Title X clients 15–44 years who use a most or moderately effective contraceptive method, by state, 2013, Family Planning Annual Report

Expanding this measure’s use beyond women served by the Title X program to women eligible for Medicaid would allow for assessing and improving the quality of contraceptive services and access to the most and moderately effective contraception at a broader population level. Women represented in Title X data have, by definition, accessed family planning services at a Title X clinic, while the broader population of women enrolled in Medicaid represent women who may not have access to Title X clinics or other qualified providers for contraceptive services.

NQF endorsement is currently being sought for this performance measure to allow for its use in monitoring and improving care for Medicaid clients. In partnership with OPA, MCH epidemiologists in states such as Iowa and Louisiana have been instrumental in assessing several of the NQF criteria to support the application for NQF endorsement. They have developed and tested algorithms to assess the feasibility of producing the measure using state-level Title X program data and state-level Medicaid claims data (CMS 2014). The resulting algorithms were then used to develop reports demonstrating gaps in performance across states, geographic regions within states, and local service sites, which established the importance of reporting the measure.

This measure will also be used to monitor and improve the quality of contraceptive access and services, thereby establishing the usability of the measure. For example, the federal Title X program plans to use the measure to identify grantees with low performance on the measure, work with them to assess potential reasons for low performance and identify needs, then provide training and technical assistance to address those needs. At the local level, grantees have planned a similar process with their local clinical sites. Other potential uses of the measure in the future may include benchmarking for all service sites and annual reporting on the effectiveness of strategies employed to improve performance. Finally, the measure can be used to identify targets for QI initiatives, monitor and report the impact of tested improvement strategies, and assess the population impact of scaling up successful strategies.

Example 2: QI to Support Peripartum Hemorrhage Prevention in Florida

In partnership with clinical and other stakeholders, MCH epidemiologists have significantly contributed to a QI project addressing peripartum hemorrhage in Florida. When choosing the state’s first individualized maternal health QI initiative, the Florida Perinatal Quality Collaborative (FPQC) leadership focused their attention on leading causes of pregnancy-related morbidity and mortality that had the potential for improvement. One of the leading causes of pregnancy-related death from 1999 to 2012 was hemorrhage (15.2 %) (Hernandez 2014). As found in other states, many of these deaths were identified to be potentially preventable based on the state’s and others’ mortality review findings (Berg et al. 2005; CMACE 2011; Main et al. 2015; Hernandez 2014). In addition, the severe maternal morbidity (SMM) rate, as defined by the Centers for Disease Control and Prevention (Callaghan et al. 2012), has been increasing in Florida, with blood transfusion as the leading condition (64 % of SMM deliveries in 2011). Although the mortality rates were too small and not collected in a fashion to examine hospital-specific rates, hospital SMM rates varied substantially in the state from <5 to >60 cases per 1000 births in 2011. This information, along with the availability of existing successful state hemorrhage prevention initiatives in California and New York (unpublished data), contributed to FPQC leaders’ decision to choose hemorrhage as the target of the first state-developed initiative.

Rather than develop a completely new initiative, FPQC leaders chose to adapt the already successful and recently updated California Maternal Quality Care Collaborative toolkit (Lyndon et al. 2015), which was based on the state’s maternal mortality review findings and current literature. Florida’s adaptation, substantiated with their own mortality review findings, included nine QI components focused on three prevention areas: (1) readiness, such as establishing hospital protocols and promoting trainings; (2) recognition, such as performing risk assessment upon admission and quantifying blood loss; and (3) response, such as having practice drills and conducting debriefing.

This QI initiative framework directly contributed to the selection, collection and reporting of process and outcome measures. Choosing measures can be a complex process due to the necessary integration of clinical expertise, available evidence, and data experience. MCH epidemiologists contributed to the selection of measures, including structural and process measures such as the presence of a hospital protocol and professional training or practice drills, which were relatively simple to collect. Other measures were more complicated, requiring monthly chart audits on a sample of charts at each hospital, for which MCH epidemiologists developed a sampling plan and data collection protocol. Reports generated from those measures indicated, for example, that the percentage of mothers across all 35 participating hospitals who were risk assessed upon admission increased from 14 to 79 % from pre-December 2014 to April 2015. In addition, three-quarters of participating hospitals reported that at least 75 % of mothers in their hospital were risk-assessed. For the same time period, the percent of mothers delivering vaginally with quantified blood loss increased from 4 to 62 %; half of participating hospitals reported that 80 % or more had quantified blood loss. As QI initiatives such as this one progress, process measures may be adapted or replaced to most effectively address current needs, which can be challenging and dependent on participating hospitals’ willingness and resources to report new complex measures.

Demonstrating changes in outcomes as a result of QI can also be challenging, as it requires selecting measures that most effectively capture intended outcomes and finding an acceptable way to monitor those measures. This is especially true with peripartum hemorrhage. If hospital providers have historically underestimated blood loss and thereby hemorrhage, then this type of initiative may actually increase reporting of hemorrhage and blood product use in the initial period, before a decrease is seen as a result of QI. The FPQC initiative saw an increase in any blood product use per mother during the first 6 months of the initiative and saw a decline in only the last 2 months. In terms of the new nationally recommended measure for mass transfusion—>3 units of any blood products (The Joint Commission 2015)—the initiative has not yet seen a substantial change; however, among participating hospitals with the highest baseline rates, variability appears to be decreasing. Based on personal communications, other states with longer operating initiatives for reducing hemorrhage are observing a decrease in overall blood product use plus an initial reduction in rates of pregnancy-related mortality due to hemorrhage. In Florida, the mortality review team is currently reviewing the pregnancy-related deaths for the time period of the initiative.

As a result of this QI initiative, for the first time state perinatal leaders feel that substantial collective efforts are underway to reduce pregnancy-related mortality, and that they can finally have an impact on the deaths that they have been reviewing for a long time. MCH epidemiologists in Florida have been vital contributors to these collective efforts. Among other activities, they have guided the data-driven process for selecting the initiative, defined and monitored performance measures, and translated results to stakeholders within and external to Florida.

Challenges to Effective QI in MCH

As described in the previous examples, clinical QI has great potential for impacting the health of MCH populations when coordinated centrally for clinical settings within or across states, especially when performance measures are thoughtfully applied and MCH epidemiologists are integrally involved. For strategies implemented outside of a clinical setting, the use of QI is less familiar and more complex, especially when the targeted processes are related to the implementation of evidence-based interventions at a systems or community level. An ongoing challenge for the MCH community is to better understand how QI processes can effectively be applied to population-based interventions, such as cross-sector initiatives to address social determinants of health or interventions to improve access to clinical or preventive care for those not engaged in the healthcare system. While clinical QI efforts may improve the quality of care, quality healthcare that is not accessible or equitably distributed will not result in improved health for all. Without complementary QI initiatives to improve access to high quality care, clinical QI efforts may have the unintended consequence of increasing health disparities. MCH epidemiologists bring this population focus to the process of identifying and implementing QI initiatives to improve MCH outcomes. A complementary challenge is in defining discrete performance measures to sufficiently monitor changes in complex phenomena such as access to care over the course of a population-based QI initiative.

Another significant challenge to effective QI is limited access to high quality and timely data. While recent QI efforts using state vital statistics data have prompted the reporting of more timely, provisional data in participating states, challenges remain. Improving data timeliness requires testing the limits of feasibility for reducing data collection and processing time, while identifying what level of accuracy can acceptably be sacrificed in the process. Concurrently, the quality of critical data elements should be evaluated and, if necessary, enhanced through PDSA cycles testing improvement strategies for data ascertainment. These complementary data QI initiatives have the potential to not only enhance related clinical or population-based QI, but also to improve data systems for surveillance, monitoring and evaluation. MCH epidemiologists are at the nexus of all of these data-focused activities, so can provide leadership to efforts to improve data timeliness and quality and, more broadly, to advocate for data systems and linkages that will best support QI and other MCH initiatives.

Conclusion

Using data to inform QI processes increases the potential to affect change in MCH outcomes at the local, state and national levels. Whether performance is considered in an individual clinical setting or in the larger public health system, selecting, defining, and implementing appropriate performance measures is crucial for effectively monitoring evidence-based/informed processes for improving MCH outcomes. MCH epidemiologists are well poised to provide leadership in identifying MCH issues for QI initiatives, assessing promising improvement strategies, reporting changes over time, evaluating population impact, and translating findings to practice. Most importantly, MCH epidemiologists can ensure that QI efforts are aligned with local, state and federal priorities and related MCH initiatives, thereby optimizing the opportunity for these efforts to collectively improve health outcomes for all MCH populations.

Significance.

What is already known on this subject?

Quality improvement (QI) initiatives, which are informed by standardized performance measures and use methods such as Plan-Do-Study-Act (PDSA) cycles to introduce and test practice changes, are increasingly being applied to maternal and child health (MCH) indicators.

What this study adds?

While QI has recently become a prominent strategy for addressing MCH priorities at the national, state and local levels, MCH partners have not formally discussed the role of performance measurement and MCH epidemiology leadership in QI initiatives. We posit that sound performance measurement leads to success in identifying appropriate MCH targets for QI, assessing promising improvement strategies, reporting changes over time, evaluating population impact, and translating findings to practice. We also suggest that MCH epidemiologists have an important leadership role in applying rigor to data analysis for QI and ensuring that QI efforts are aligned with local, state and federal priorities, thereby optimizing the opportunity to improve MCH outcomes.

Acknowledgments

Funding for the 2014 CityMatCH Leadership and MCH Epidemiology Conference was provided by the Centers for Disease Control and Prevention Division of Reproductive Health and the Maternal and Child Health Bureau of the Health Services and Resources Administration. The authors would like to sincerely thank the other members of the conference planning committee, especially Caroline Stampfel, Russell Kirby, Laurin Kasehagen, and Mary Balluff for their input on this plenary session.

Footnotes

Disclaimer The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention, the Health Resources and Services Administration, the Office of Population Affairs, or the Department of Health and Human Services.

References

  1. American College of Obstetricians and Gynecologists. Increasing use of contraceptive implants and intrauterine devices to reduce unintended pregnancy. Obstetrics and Gynecology. 2009;114(6):1434–1438. doi: 10.1097/AOG.0b013e3181c6f965. [DOI] [PubMed] [Google Scholar]
  2. Berg CJ, Harper MA, Atkinson SM, Bell EA, Brown HL, Hage M, et al. Preventability of pregnancy-related deaths: Results of a state-wide review. Obstetrics and Gynecology. 2005;106:1228–1234. doi: 10.1097/01.AOG.0000187894.71913.e8. [DOI] [PubMed] [Google Scholar]
  3. Bethell CD, Kogan MD, Strickland BB, Schor EL, Robertson J, Newacheck PW. A national and state profile of leading health problems and health care quality for US children: Key insurance disparities and across-state variations. Academic Pediatrics. 2011;11(3):S22–S33. doi: 10.1016/j.acap.2010.08.011. [DOI] [PubMed] [Google Scholar]
  4. Callaghan WM, Creanga AA, Kuklina EV. Severe maternal morbidity among delivery and postpartum hospitalizations in the United States. Obstetrics and Gynecology. 2012;120:1029–1039. doi: 10.1097/aog.0b013e31826d60c5. [DOI] [PubMed] [Google Scholar]
  5. CDC & OPA. Providing quality family planning services: Recommendations of CDC and the US Office of Population Affairs. MMWR Recommendations and Reports. 2014;63(4):1. [PubMed] [Google Scholar]
  6. Center for Medicare and Medicaid Services (CMS) Maternal and Infant Health Initiative. (Producer) Measuring Contraceptive Use in Medicaid and CHIP [Video webinar] 2014 Retrieved from: http://www.medicaid.gov/medicaid-chip-program-information/by-topics/quality-of-care/downloads/contraception-use-webinar.pdf.
  7. Center for Medicare and Medicaid Services (CMS) Center for Medicaid and CHIP Services. Core set of children’s health care quality measures for medicaid and CHIP (Child Core Set): Technical specifications and resource manual for federal fiscal year 2015 reporting. 2015 Retrieved from: http://www.medicaid.gov/medicaid-chip-program-information/by-topics/quality-of-care/adult-health-care-quality-measures.html.
  8. Centers for Disease Control and Prevention (CDC) Perinatal Quality Collaboratives. 2015 Apr 29; Retrieved from: http://www.cdc.gov/reproductivehealth/MaternalInfanthealth/PQC.htm.
  9. Centre for Maternal and Child Enquiries (CMACE) Saving mothers’ lives: Reviewing maternal deaths to make motherhood safer: 2006–2008. The eighth Report on confidential enquiries into maternal deaths in the United Kingdom. British Journal of Obstetrics and Gynaecology. 2011;118(suppl 1):1–203. doi: 10.1111/j.1471-0528.2010.02847.x. [DOI] [PubMed] [Google Scholar]
  10. Cheng D, Schwarz EB, Douglas E, Horon I. Unintended pregnancy and associated maternal preconception, prenatal and postpartum behaviors. Contraception. 2009;79(3):194–198. doi: 10.1016/j.contraception.2008.09.009. [DOI] [PubMed] [Google Scholar]
  11. Finer LB, Zolna MR. Shifts in intended and unintended pregnancies in the United States, 2001–2008. American Journal of Public Health. 2014;104(S1):S43–S48. doi: 10.2105/AJPH.2013.301416. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gemmill A, Lindberg LD. Short interpregnancy intervals in the United States. Obstetrics and Gynecology. 2013;122(1):64. doi: 10.1097/AOG.0b013e3182955e58. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gipson JD, Koenig MA, Hindin MJ. The effects of unintended pregnancy on infant, child, and parental health: A review of the literature. Studies in Family Planning. 2008;39(1):18–38. doi: 10.1111/j.1728-4465.2008.00148.x. [DOI] [PubMed] [Google Scholar]
  14. Health Resources and Services Adminstration (HRSA) Maternal and Child Health Bureau. Collaborative Improvement & Innovation Network to Reduce Infant Mortality. 2016 Retrieved from: http://www.nichq.org/childrens-health/infant-health/coiin-to-reduce-infant-mortality.
  15. Hernandez L. Pregnancy-Associated Mortality Review: Pregnancy-Related Deaths Due to Hemorrhage, 1999–2012. Florida Department of Health; 2014. Retrieved from: http://www.floridahealth.gov/programs-and-services/womens-health/pregnancy/pamr-hemorrhage.pdf. [Google Scholar]
  16. Langley GJ, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide: A practical approach to enhancing organizational performance. 2nd. San Francisco, CA: Jossey-Bass; 2009. [Google Scholar]
  17. Lu MC, Lauver CB, Dykton C, Kogan MD, Lawler MH, Raskin-Ramos L, et al. Transformation of the title V maternal and child health services block grant. Maternal and Child Health Journal. 2015;19(5):927–931. doi: 10.1007/s10995-015-1736-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Lyndon A, Lagrew D, Shields L, Main E, Cape V. Improving health care response to obstetric hemorrhage. (California Maternal Quality Care Collaborative Toolkit to Transform Maternity Care) Developed under contract #11-10006 with the California Department of Public Health; Maternal, Child and Adolescent Health Division; Published by the California Maternal Quality Care Collaborative, 3/17/15 2015 [Google Scholar]
  19. Main EK, McCain CL, Morton Holtby S, Lawton ES. Pregnancy-related mortality in California: Causes, characteristics, and improvement opportunities. Obstetrics and Gynecology. 2015;125:938–947. doi: 10.1097/AOG.0000000000000746. [DOI] [PubMed] [Google Scholar]
  20. MCH Navigator. MCH Training Primer: MCH Training Cycle. 2014 Jan; Retrieved from: http://mchnavigator.org/trainings/MCH-planning-cycle.php.
  21. National Quality Forum (NQF) NQF: About Us. 2016a Retrieved from: http://www.qualityforum.org/story/About_Us.aspx.
  22. National Quality Forum (NQF) What NQF Endoresement Means. 2016b Retrieved from: http://www.qualityforum.org/Measuring_Performance/ABCs/What_NQF_Endorsement_Means.aspx.
  23. Office of Population Affairs (OPA) Title X family planning. 2016 Retrieved from: http://www.hhs.gov/opa/title-x-family-planning.
  24. Provost LP, Murray SK. The health care data guide: Learning from data for improvement. San Francisco, CA: Jossey-Bass; 2011. [Google Scholar]
  25. Public Health Foundation Turning Point. From silos to systems: Using performance management to improve the public’s health. 2004 Retrieved from: http://www.phf.org/resourcestools/Documents/silossystems.pdf.
  26. Riley WJ, Moran JW, Corso LC, Beitsch LM, Bialek R, Cofsky A. Defining quality improvement in public health. Journal of Public Health Management and Practice. 2010;16(1):5–7. doi: 10.1097/PHH.0b013e3181bedb49. [DOI] [PubMed] [Google Scholar]
  27. Rosenbaum S. The patient protection and affordable care act: Implications for public health policy and practice. Public Health Reports. 2011;126(1):130. doi: 10.1177/003335491112600118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. The Joint Commission. Statement clarifies Sentinel Event Policy in severe maternal morbidity cases. Joint Commission Online February. 2015;4:1–2. 2015. [Google Scholar]
  29. Trussell J. Contraceptive failure in the United States. Contraception. 2011;83(5):397–404. doi: 10.1016/j.contraception.2011.01.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Ventura SJ, Curtin SC, Abma JC, Henshaw SK. Estimated pregnancy rates and rates of pregnancy outcomes for the United States, 1990–2008. National vital statistics reports: From the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System. 2012;60(7):1–21. [PubMed] [Google Scholar]

RESOURCES