Evaluation of an Active Surveillance System for Stillbirths in Metropolitan Atlanta

Abstract

Background

In 2005, a pilot project was started at the Centers for Disease Control and Prevention (CDC) to expand an existing birth defects surveillance program, the Metropolitan Atlanta Congenital Defects Program (MACDP), to conduct active surveillance of stillbirth. This pilot project was evaluated using CDC’s current guidelines for evaluating surveillance systems.

Methods

We conducted stakeholder interviews with the staff of MACDP’s stillbirth surveillance system. We reviewed the published literature on stillbirth ascertainment including 4 previous publications about the MACDP stillbirth surveillance system. Using fetal death certificates (FDC) as a second, independent data source, we estimated the total number and prevalence of stillbirths in metropolitan Atlanta using capture-recapture methods, and calculated the sensitivity of the MACDP stillbirth surveillance system.

Results

The MACDP stillbirth surveillance system is useful, flexible, acceptable, and stable. The system’s data quality is improved because it uses multiple sources for case ascertainment. Based on 2006 data, estimated sensitivities of FDCs, MACDP, and both sources combined for identifying a stillbirth were 78.5%, 76.8%, and 95.0%, respectively. The prevalence of stillbirths per 1,000 live births and stillbirths was 8.2 (95% confidence interval [CI]: 7.5-9.0) based on FDC data alone and 9.9 (95% CI: 9.1-10.8) when combined with MACDP data.

Conclusion

Use of MACDP as an additional data source for stillbirth surveillance resulted in higher levels of case ascertainment, better data quality, and a higher estimate of stillbirth prevalence than using FDC data alone. MACDP could be considered as a model to enhance stillbirth surveillance by other active birth defects surveillance programs.

Introduction

Stillbirth, often defined as an intrauterine fetal death after 20 weeks of gestation, affects about 1 in 100-200 deliveries,^1,2 creating significant emotional and psychological distress for patients, families, and clinicians.^3,4 With advances in antepartum care, the rates for late preterm stillbirths (ie, 34-36 weeks of gestation) and term stillbirths (more than 37 weeks of gestation) have declined, but the rate of early stillbirths (20-27 weeks) has remained stable.^5,6 Stillbirths occur with greater frequency among pregnancies complicated by certain risk factors, such as diabetes, obesity, maternal hypertension, and smoking;^1,2,7,8 among non-Hispanic black women compared to non-Hispanic white women in the United States;^1,2,5,6 and in economically deprived communities compared to wealthier communities.^7,9 However, in many cases, depending on the extent and expertise of the postmortem evaluation, the cause of the stillbirth is unknown.¹⁰

Much of our understanding of the variations in the occurrence of stillbirth among different populations and over time is based on data from fetal death certificates (FDCs). However, the usefulness of FDCs as the sole data source for stillbirth surveillance and for epidemiologic studies might be limited.^11,12 Certain information on the FDCs, particularly with respect to contributing causes to fetal death, is incomplete and unreliable.^11-13 Furthermore, although fetal death is a reportable event in the United States, and the National Center for Health Statistics at the Centers for Disease Control and Prevention (CDC) recommends that all fetal deaths of more than 350 grams or, if weight is unknown, more than 20 weeks of gestation be reported, reporting requirements vary across states.¹⁴ The variations in gestational age and birth weight reporting criteria limit the ability to compare data across states.

In 2005, the National Center on Birth Defects and Developmental Disabilities (NCBDDD) at CDC initiated a pilot study to assess the feasibility of leveraging the resources of an existing population-based birth defects surveillance program, the Metropolitan Atlanta Congenital Defects Program (MACDP), to incorporate surveillance of stillbirths with and without birth defects.¹⁵ The goals of the MACDP stillbirth surveillance project are: (1) to assess the feasibility of expanding MACDP to incorporate existing medical record information on all stillbirths among the study population, (2) to monitor and report the occurrence of stillbirth among the study population, (3) to serve as a registry for etiologic studies on causes of stillbirth, and (4) to serve as a resource for education and evaluation of targeted prevention programs. Active ascertainment of stillbirths in this pilot project began with deliveries occurring in 2006. We evaluated the pilot project in 2010, using the CDC guidelines for the evaluation of the surveillance systems¹⁶ as our framework.

Methods: Description of Surveillance System and Methods for Evaluation

Description of the MACDP Stillbirth Surveillance System

MACDP is an ongoing, population-based birth defects surveillance system established in 1967 that actively monitors birth defects among the offspring of women living in the 5 central counties of metropolitan Atlanta at the time of delivery. The program routinely collects data on clinical and demographic characteristics of liveborn and stillborn infants, as well as pregnancy terminations for the presence of structural birth defects. Major structural defects, chromosomal abnormalities, and clinical syndromes diagnosed within 6 years of delivery are included in MACDP.^17,18 MACDP conducts surveillance activities under the authority of the State of Georgia and uses multiple sources for case finding, including discharge summaries from area birth hospitals, prenatal diagnostic clinics, and cytogenetic laboratories, as well as live birth and fetal death records from the state.

Inclusion Criteria and Stillbirth Ascertainment

Pregnancy outcome classification is based on the definitions for live birth, fetal death, and induced termination of pregnancy provided by the 1992 Revision of the Model State Vital Statistics Act and Regulation (Model Law).¹⁹ There is no universally accepted definition of stillbirth that includes the criteria for gestational age or birth weight. For surveillance purposes, the MACDP pilot project defines a stillbirth as any intrauterine fetal death occurring at more than 20 weeks of gestation or more than 350 grams of weight if gestational age is unknown, and the mother must be a resident of the 5-county metropolitan Atlanta area. The assigned gestational age is the age of the fetus at death as determined by the physician and recorded in the medical record. The addresses for the mothers of stillbirth are confirmed through matching with US Postal Service ZIP Codes.¹⁷ If there is uncertainty about a mother’s address, the FDC is used to determine county of residence.

Stillbirths are ascertained from multiple sources. Trained abstractors visit all birthing hospitals and review labor and delivery logs, stillbirth logs, neonatal intensive care logs, and postmortem or pathology logs. Additionally, MACDP routinely acquires a disease index from each hospital to identify deliveries potentially affected by a major birth defect. With the implementation of MACDP’s stillbirth surveillance activities, the disease index has been expanded to include additional codes that might assist in identifying stillbirths. Abstractors also routinely visit several high-risk obstetric providers and maternal-fetal medicine departments to identify pregnancies diagnosed with intrauterine fetal death in these settings. Detailed information is collected for each stillbirth and entered into an electronic data abstraction tool. This information includes demographic, diagnostic, and pregnancy information, as well as a maternal pregnancy history. Abstractors also collect additional information on stillbirth, including pregnancy complications and postmortem examinations from radiographs, autopsies, fetal examinations, and placental histopathology. Information is entered into fingerprint-password access-secured laptops to protect the confidentiality of cases. Birth defects among stillbirths are coded using a modified British Pediatric Association 6-digit code developed specifically for MACDP. All clinical and postmortem evaluation information is reviewed and assessed for completeness and accuracy as potential contributors to fetal death.

Fetal Death Certificates

Fetal deaths are by law reportable events in all states and territories. Although reporting requirements vary by state, most states mandate that fetal deaths at 20 weeks of gestation or longer be reported. In Georgia, however, fetal deaths occurring at any gestational age are to be reported if brought to the attention of a health care provider.¹⁹ FDCs are routinely used by MACDP as a source for case finding.

Methods for Evaluation

We assessed the usefulness of the surveillance system and the following attributes using methods recommended in the CDC’s updated guidelines for evaluating public health surveillance systems.¹⁶

1. Usefulness was assessed by determining the extent to which the system’s outputs (publications of its methods and data) are deemed of value, including measurement of the frequency of their citations in the published literature.

2. Simplicity was assessed by considering the system’s methods of data collection and the level of integration with other systems.

3. Flexibility was assessed by observing how the surveillance system (MACDP) has responded to a new demand.

4. Data quality was assessed based on the completeness and validity of data elements recorded by the system, as previously reported in 2 publications.

5. Acceptability was assessed based on the willingness of persons and organizations to participate in the surveillance activities.

6. Sensitivity, the proportion of “true” stillbirths reported by the surveillance system, was calculated using capture-recapture analysis²⁰ to estimate the true number of stillbirths in metropolitan Atlanta.

7. Representativeness was assessed by comparing the distribution of demographic characteristics among the population in the surveillance system with those of the base population it is designed to represent (metropolitan Atlanta), as previously reported in 2 MACDP publications.

8. Timeliness was assessed by the speed in which data flow through the surveillance system and covered the time from case identification through abstraction, processing, and review to availability of data for use.

9. Stability was assessed by the system’s operational reliability and availability.¹⁶

To gather the information necessary to assess the aforementioned characteristics, we conducted structured stakeholder interviews with the staff of MACDP’s stillbirth surveillance system (including medical records abstractors, clinical reviewers, and managers), and with the director of a stillbirth advocacy organization. We also performed 2 hospital site visits to observe the abstraction of data from medical records, autopsy reports, placenta pathology reports, prenatal records, and other sources reviewed by MACDP abstractors. We reviewed the published literature on stillbirth ascertainment and gathered detailed information on MACDP methods for the capture of stillbirth data. Lastly, we conducted a capture-recapture analysis described below.

Capture-Recapture Analysis

Prevalence estimates from incomplete, independent data sources can be calculated using capture-recapture methods.²⁰ We hypothesized that the ascertainment of stillbirths from both FDCs and MACDP was incomplete. In 2006, because of administrative issues, FDCs were not available as a data source to MACDP, allowing for independency of data sources for that year. We were thus able to apply capture-recapture methods to these independent data sources and estimate the total number of stillbirths in the surveillance population. We also calculated the 95% confidence intervals for stillbirth prevalence and the sensitivity of each data source independently as well as combined.

Results

Usefulness

MACDP stillbirth surveillance system is useful. Four peer-reviewed articles have been published by MACDP on methodologies and best practices of stillbirth surveillance activities.^12,13,15,21 These publications have been cited 14 times as of March 1, 2012. One of these publications indicated that MACDP stillbirth surveillance pilot project data is useful for monitoring the occurrence of stillbirths in the metropolitan Atlanta area, and suggested that active surveillance of stillbirths, building on an existing birth defects surveillance system, might serve as a model for state programs that are considering initiating stillbirth surveillance.¹³ Two of these studies have assessed the utility of both MACDP’s active case identification and FDCs as sources for stillbirth surveillance data.^12,13 In one study, the authors linked stillbirths with birth defects ascertained by MACDP over a 9-year period with FDCs and assessed the value of FDCs in monitoring birth defects among stillbirths. Using MACDP as the gold standard, the authors reported that the sensitivity and positive predictive value of FDCs for selected categories of birth defects ranged from 10% to 70% and 25% to 93% respectively. The values were higher for the more obvious defects such as anencephaly, spina bifida, and cleft lip/palate and less so for Down syndrome, heart defects, and renal agenesis.¹² A second study reported on the potential enhancements to surveillance data collected on stillbirths through linkage of data sources.¹³ A random sample of 125 fetal deaths from 2004 were selected for abstraction using the revised MACDP abstraction protocol for stillbirth surveillance. Among the 102 cases abstracted (23 were excluded when no medical record could be found) and linked to FDCs, there was less missing information for selected variables, such as fetal sex, birth weight, and substance use, when both data sources were combined. Furthermore, 42% of stillbirth cases had no information recorded on the FDCs that indicated a cause of or contributor to death, whereas when data from MACDP were used to assess cause of and contributors to death, only 10 cases could not be classified as to the cause of death. Lastly, this study demonstrated that 3% of the 102 cases that were issued a FDC were in fact live births expiring shortly after delivery, and 13 of the 102 cases were stillborn after medical induction of labor for termination.¹³

Surveillance System Attributes

Simplicity

Because abstractors use multiple sources of information for case finding and data abstraction, active surveillance of stillbirth is not simple. Interpreting, classifying, and reporting information on birth defects and contributors of death for stillbirth entail the need for review of complex information by abstractors. In addition, different types of reporting systems (eg, paper or electronic formats) used by health care facilities contribute to the complexity of the data abstraction process. However, MACDP’s current electronic surveillance data entry module enables information to be entered faster than did the previous paper-based format. In addition, a recent in-house survey of MACDP abstractors on the efficacy of stillbirth surveillance activities indicated that incomplete and vague information in the medical records and incomplete prenatal records were major obstacles in abstracting stillbirth cases (survey data unpublished). Because medical records often are incomplete, abstractors might need to make several visits to a single hospital to capture information from distinct locations within the hospital, such as pathology departments for postmortem information. Although data collection can be complex and time consuming for abstractors, integration of stillbirth surveillance into an existing and established birth defects surveillance system like MACDP is potentially a more simple approach for enhancing data on stillbirths compared to developing and implementing a new surveillance system de novo.

Flexibility

MACDP adapted its methodology to incorporate the ascertainment of stillbirths without birth defects. Specifically, this meant the incorporation of regular visits to 4 birth hospitals in neighboring counties to ascertain potential stillbirths delivered outside of the catchment area. This protocol modification was made following a preliminary evaluation of FDCs for the period 1994-2004, which indicated that approximately 3% of stillbirths occurring among mothers residing in the 5-county surveillance area were delivered in neighboring counties. In addition, MACDP developed and successfully pilot tested a revised data collection tool designed to capture additional clinical and histopathological information, such as maternal health conditions, pregnancy complications, and placental pathology/fetal autopsy findings. This additional information is used to understand potential causes and contributors to fetal death. Also, MACDP abstractors who previously had abstracted only information regarding birth defects were trained to identify and review medical records of fetal deaths to determine inclusion/exclusion criteria. The development and implementation of a modular electronic data management system for MACDP has shown that future modifications to the data collection protocol (eg, the inclusion of additional variables) are feasible.

Data Quality

As previously described, the MACDP method for stillbirth surveillance, which includes information both actively abstracted from medical records and passively ascertained through linkage with FDC data, results in increased case finding and improves the completeness of data collected compared with using either source of data alone.^12,13 However, a major challenge affecting the overall quality of data is the incompleteness of information in medical records and the general lack of postmortem evaluations. The clinical review of stillbirth case records by MACDP staff allows for the accurate coding of birth defects if present, and ensures that all information, if available, is obtained to assess other contributors to fetal death. If a postmortem evaluation was conducted, that information will be included in the MACDP stillbirth case record; typically the FDC will not include this information since the vital record is typically completed prior to the availability of this information and is rarely amended.²² The electronic data abstraction application used by MACDP also has built-in logic checks to identify and control data entry errors. Likewise, data cleaning and management are centralized at MACDP, ensuring a standard and uniform process for data quality control. Furthermore, existing abstractors receive continuous training on topics such as placental pathology and the interpretation of autopsy reports. Such training allows the abstractors to have a better understanding of the importance of the information being collected.

Acceptability

The stillbirth surveillance pilot project has been accepted as an integral part of MACDP. MACDP has a long history of partnerships, having been founded in 1967 as a collaborative effort between CDC, Emory University and the Georgia Mental Health Institute. MACDP is administered by NCBDDD at CDC and has authority to conduct surveillance of birth defects from the Georgia Department of Human Resources on behalf of the Georgia Department of Public Health. Fetal deaths also are a reportable event in Georgia, and in 2005, MACDP requested and was given the authority to begin surveillance of stillbirths in a 5-county area of metropolitan Atlanta. This authority is renewed annually. All data are protected by the Privacy Act of 1974 and by an Assurance of Confidentiality granted by the director of CDC. All of these factors serve to provide a strong level of confidence and trust in the ongoing work of MACDP, including its expansion to active stillbirth surveillance. In fact, the National Birth Defects Prevention Network, a non-profit organization that addresses the issues of birth defects surveillance, research, and prevention under 1 umbrella by maintaining a national network of state and population-based birth defects programs, is planning to develop a chapter in its surveillance guidelines that addresses the integration of stillbirth surveillance into existing birth defects surveillance programs. In addition, hospitals and health care facility staff (eg, physicians, nurses, and administrative staff) have accepted and are cooperative with the MACDP stillbirth activities. Some facilities have even provided space to the MACDP abstractors to complete their work.

Prevalence, Sensitivity, and Predictive Value Positive

Prevalence

Using 2006 data, we applied capture-recapture methods²⁰ to estimate the total number of stillbirths among the surveillance population and the relative contribution of each data source (MACDP vs FDC) to case finding. Based on these methods, an estimated 581 cases of stillbirth occurred in the metropolitan Atlanta area. For each data source alone, MACDP identified 446 cases and FDCs identified 456 cases. MACDP identified 96 cases that did not link to an FDC, while 106 cases identified by FDCs were not captured by MACDP Of these 106 cases, 79 were missed initially, but subsequently abstracted, by MACDP; 17 had medical records that could not be found; and 10 occurred in neighboring counties outside the catchment area. The estimated number of cases missed by both data sources and identified by capture-recapture methods was 29 (see table). Given that there were a total of 55,707 live births in Atlanta in 2006, the prevalence of stillbirths per 1,000 live births and stillbirths was 8.2 (95% Cl: 7.5-9.0) based on FDC data alone, 8.0 (95% Cl: 7.3-8.8) based on MACDP data alone, and 9.9 (95% Cl: 9.1-10.8) based on both data sources combined (FDC and MACDP).

Table.

Distribution of Stillbirths by Source of Identification: Metropolitan Atlanta Congenital Defects Program

		Identification by MACDP
		Yes	No	Total
Identification by FDC	Yes	350	106	456
	No	96	(29)	125
Total		446	135	(581)

Note: The cases missed by both sources (n=29) and the total number of stillbirths (n=581) were calculated with capture-recapture methodology. Missed cases=(106×96)/350=29.

Sensitivity calculations: FDC(456/581=78.5%); MACDP (446/581=76.8%); FDC and MACDP ([350+96+106]/581=95.0%).

MACDP=Metropolitan Atlanta Congenital Defects Program; FDC=fetal death certificate.

Sensitivity and Predictive Value Positive

In the absence of a reference or “gold standard” for stillbirths, the sensitivity and predictive value positive of MACDP stillbirth surveillance system could not be calculated. However, using the results of the capture-recapture analysis, and assuming that the cases identified by both data sources (ie, combining FDCs and MACDP) represent the entire sample of cases of stillbirth in the population (missing cases by both sources [n=29] were not included), then the sensitivity of FDCs alone would be 78.5% and that of MACDP alone would be 76.8%. The estimated sensitivity of MACDP when FDCs are included as a source of case finding increased to 95.0% (see table).

Representativeness

Since the inception of MACDP in 1967, the population of the 5-central counties of metropolitan Atlanta has changed considerably and currently does not demographically represent the entire state of Georgia or the United States. In particular, metropolitan Atlanta has become progressively more urban over time, with an increasingly higher proportion of residents of races other than non-Hispanic white who have both higher median income and educational levels compared with residents of the rest of Georgia.¹⁷ Furthermore, the higher proportion of non-Hispanic black mothers in the 5 counties explains in large part the higher overall prevalence of stillbirths in the 5 counties compared with that of the national average.⁵

On the other hand, a previous MACDP stillbirth publication determined that underreporting of stillbirth to vital records may not necessarily occur randomly with race/ethnicity and autopsy status both associated with a stillbirth being issued an FDC.¹² Therefore, it is likely that improvements in case ascertainment utilizing data from MACDP will enhance the representativeness of surveillance data for population of interest.

Timeliness

There are approximately 400-500 stillbirths occurring annually within the MACDP stillbirth surveillance population. Stillbirths typically are identified by abstractors within 1 month of delivery. The timeliness of ascertainment, however, can be affected by the often delayed (up to 4 months) receipt of autopsy and placental pathology information. The average time of abstraction of a single stillbirth case ranges from 30-60 minutes. After abstraction, MACDP staff with clinical expertise review the case record. Cases needing additional information or clarification, such as those with documentation of an ultrasound but without ultrasound results in the record, are returned to the abstractor for completion. Typically, the ascertainment of this additional information takes 1-2 weeks. All abstraction takes place using MACDP’s electronic surveillance data entry module, enabling data entry to be faster than previous, paper-based format.

Stability

As mentioned previously, MACDP has been in operation since 1967 and is administered by CDC with dedicated staff and support. A full-time medical officer oversees and manages the project, and fellows and trainees regularly are utilized to assist in research and evaluation studies. The incorporation of stillbirth surveillance into MACDP is intended to be an ongoing surveillance activity. The MACDP stillbirth surveillance project is positioned to provide leadership, support, and technical assistance in the future to other state-based programs considering similar expansions.

Discussion

The MACDP stillbirth surveillance pilot project has proven to be useful, flexible, acceptable, and stable. In addition, initial studies have shown that the use of both sources (MACDP linked with FDCs) for surveillance of stillbirths with birth defects have resulted in greater ascertainment of stillbirths, more complete and reliable data,^12-13 and more accurate estimates of stillbirth prevalence. This approach for enhancing stillbirth surveillance is extremely useful. It is built upon an active and ongoing surveillance system with an established methodology, resulting in reduced costs and time compared with the implementation of a new system.

The long history of partnerships and the infrastructure of MACDP has benefited stillbirth surveillance activities. Two planning workshops were held in 2005 to assess the challenges and priorities for conducting stillbirth surveillance as part of existing birth defects monitoring programs.¹⁵ These workshops informed and guided the development of a revised data collection tool for use in stillbirth surveillance and identified additional potential sources for case finding, such as emergency room and pathology departments.¹³ Thus, MACDP has shown the flexibility to adapt to challenges that might arise over time, including new developments in methods and medical technology.

A number of challenges remain for active stillbirth surveillance. A particularly important one is that results of postmortem examinations often are absent from medical records or are incomplete at the time of abstraction. In many cases, postmortem evaluations are not performed for a number of reasons, including parental refusal.²¹ Although the American Congress of Obstetricians and Gynecologists provides recommendations for the postmortem examination of stillbirths,² there is a need for routine implementation of these recommendations in clinical settings. Likewise, there is a need to develop an appropriate classification protocol based on existing clinical and postmortem information for reporting descriptive surveillance data on stillbirths. Once the pilot phase of MACDP’s stillbirth surveillance program is completed, a further evaluation, including an assessment of the system’s cost-effectiveness, will be needed to ensure that the objectives of the system are being met adequately.

MACDP should consider publishing methods and guidelines for stillbirth surveillance which can be useful to other state-based birth defects surveillance programs that might be contemplating similar expansions of their systems to capture stillbirths. MACDP should also engage stakeholders to address stillbirth as an important public health issue through better surveillance, identification of risk factors, and strategies for prevention. Addressing such challenges will be a key step towards the development and implementation of effective public health prevention strategies.

Lessons Learned

We learned through the capture-recapture methods that using MACDP stillbirth data can enhance the completeness of case ascertainment compared to FDC alone. Active surveillance activities require coordination and well-established working relationships with stakeholders. The fact that stillbirth surveillance is built on an existing birth defects surveillance system is a major strength and could be considered as a model for other surveillance systems.