Abstract
According to the World Health Organization (WHO), an estimated 303,000 neonates die within their first month of age every year globally as a result of a birth defect. Neural tube defects, serious birth defects of the brain and spine, are among the most common and severe of these birth defects. Since some low- and middle-income countries lack comprehensive, accurate data documenting the burden of these defects, providing technical assistance to help build birth defects surveillance programs can accelerate the collection of data needed to demonstrate this burden and advance prevention initiatives. We developed a birth defects surveillance toolkit, a technical assistance tool for country staff to help them implement birth defects surveillance. An evaluation of the toolkit with partners in Africa was conducted to assess perceptions of the usefulness, effectiveness, and policy impact of the surveillance toolkit and surveillance-related technical assistance provided to countries thus far. Overall, respondents provided very positive feedback about the toolkit components. Recommendations for improvement included customization to country contexts, such as photos reflective of African babies; surveillance examples from other countries; and consistent use of terms.
Background
Annually, an estimated 300,000 neonates [1] are born around the globe with a devastating neural tube defect. These birth defects of the brain and spine, such as anencephaly and spina bifida, are among the most common and severe birth defects, contributing largely to the 303,000 newborn deaths due to birth defects that occur every year worldwide [2]. Fetuses with anencephaly are either stillborn or die within hours after birth, while those affected with spina bifida require special medical and surgical care. Vast reductions in infant mortality due to infectious and diarrheal diseases have helped to highlight the impact of birth defects on under-5 mortality [3]. Moreover, since 2000, there have been numerous calls for increased attention to the impact and prevention of birth defects, such as neural tube defects, by global leaders. This has included the United Nations Millennium Development Goal 4 which aimed to reduce the under-5 child mortality rate by two-thirds by 2015, the 63rd World Health Assembly resolution on birth defects which encourages countries to prevent birth defects wherever possible, to implement screening programmes, and to provide ongoing support and care to children with birth defects and their families,” and the United Nations 2030 Sustainable Development Goal 3, which encourages countries to “ensure healthy lives and promote well-being for all at all ages” [4–6]. Finally, considering the recently established association between maternal Zika virus infection during pregnancy and birth defects such as microcephaly [7], there may be a heightened awareness of the need for birth defects surveillance.
Although high-quality evidence exists for the role of folic acid in the prevention of occurrence and re-occurrence of neural tube defects [8–9], it is often difficult for low- and middle-income countries to advocate for prevention initiatives without comprehensive, accurate data documenting the burden of these defects and the need for these initiatives [1, 10]. According to a recent publication, only an estimated 40% of World Health Organization (WHO) Member States have published neural tube defect prevalence data [10], and the comprehensiveness and representativeness of these data can vary greatly. Providing technical assistance to low- and middle-income countries to help build birth defects surveillance programs can accelerate the collection of data needed to demonstrate the burden of birth defects and the urgent need to implement effective prevention initiatives, such as fortification of staple foods with folic acid. This paper discusses the evaluation of a comprehensive technical assistance toolkit to help build birth defects surveillance capacity in Africa and the perceived utility of this toolkit by users in Africa.
Provision of Technical Assistance: Birth Defects Surveillance Toolkit
Birth Defects COUNT (Countries and Organizations United for Neural Tube Defects Prevention) is an initiative to reduce morbidity and mortality due to neural tube defects and other birth defects that was launched by the U.S. Centers for Disease Control and Prevention (CDC) in 2010. A key objective is to provide technical assistance to countries seeking to develop or enhance birth defects surveillance by increasing and improving capacity for sustainable systems globally. These surveillance systems produce data that inform decision-making for the implementation and evaluation of prevention efforts. CDC received requests from in-country public health professionals for technical assistance to improve country-level capacity for collecting high quality birth defects data. To help fulfill the objective of Birth Defects COUNT and address these country requests, partners collaborated on the development of a birth defects surveillance toolkit. The international partner group, termed the Birth Defects Surveillance Working Group (BDSWG), was made up of scientists and other technical staff from the partner organizations, and included clinicians, epidemiologists, health communication specialists, and health education specialists.
The toolkit is a technical assistance tool to help in-country public health professionals implement birth defects surveillance. It is important to note that the requests from these professionals were made with the ultimate goal of being able to collect high quality data to highlight the need for effective prevention efforts (e.g., folic acid initiatives). Many low- and middle-income countries lack the comprehensive systems that can identify and monitor the occurrence of birth defects, resulting in an underestimation of the burden of birth defects [1]. Further, because many babies affected with severe birth defects in these countries die before a diagnosis is made, there is a misperception that birth defects do not warrant attention [1]. The development of a toolkit was one way to provide critical technical assistance to aid in the process of collecting valuable data. These data allow public health professionals to ascertain the burden of birth defects and to use this information to get the attention of policy makers to implement and enhance birth defects prevention initiatives in their own countries.
The toolkit is housed on an interactive online portal. This portal can be accessed via CDC’s Birth Defects COUNT webpage (http://www.cdc.gov/ncbddd/birthdefects/surveillancemanual/index.html), and provides links to resources available on the WHO nutrition website. The primary toolkit materials can also be downloaded via the WHO, International Clearinghouse for Birth Defects Surveillance and Research (ICBDSR), and CDC sites. Limited numbers of hard copies of the major components are available; however, the toolkit components were developed to be primarily web-based for countries to easily adapt them to meet their unique needs. The major components are also currently available in English, Spanish, and French.
The toolkit consists of three primary components - a manual, photo atlas, and facilitator’s guide. These tools are designed to provide the user with information needed to develop and implement birth defects surveillance in their counties. In addition, the toolkit contains a resource library, database, and virtual training platform that provide tools and information to help support the development and implementation of birth defects surveillance. Each of these items is described in more detail in the sections that follow.
Birth Defects Surveillance: A Manual for Programme Managers
At the center of the toolkit is a birth defects surveillance manual. This manual, currently available in English, Spanish, and French, has served as the foundation for numerous international birth defects surveillance workshops. The manual is comprised of five chapters providing examples of seven external, observable birth defects - neural tube defects, orofacial clefts, hypospadias, limb defects, gastroschisis, omphalocele, and talipes equinovarus. Because the capacity to capture internal birth defects, such as heart defects, can be more resource- and technology-intensive, the manual highlights those birth defects that can be captured more readily by new or early-stage surveillance systems.
The manual chapters also provide a wide array of guidance – from detailed explanations about the differences between hospital-based and population-based surveillance, to the diagnosis and coding of birth defects, including inclusion criteria, use of International Classification of Diseases (ICD) codes, and the necessary qualifications of personnel for a surveillance program. The manual was highly commended in the public health category of the British Medical Association’s 2015 Medical Book Competition.
Pilot testing of the draft manual was conducted during birth defects workshops held in both South-East Asia and Africa. This approach served as an opportunity to gauge whether the information was well understood by the intended audience (e.g., clinicians, program managers, hospital staff) and whether gaps were present. For example, hypospadias was not one of the defects included in earlier drafts of the manual, but during pilot testing of the manual, many country representatives indicated it was a very common observable birth defect and should be included. In this way, valuable input from countries helped to shape the final product in order to maximize potential for relevance and use.
Birth Defects Surveillance: Atlas of Selected Congenital Anomalies
During pilot testing of the birth defects surveillance manual, country representatives also indicated their need and want for a photo and illustration atlas to clearly show each birth defect presented in the manual, since some of these birth defects had not been recognized before in their hospital settings. A photo atlas would also help reduce the potential for diagnostic error. In order to be responsive to this request, the BDSWG developed an atlas of selected congenital anomalies, which includes both photographs and illustrations along with brief descriptions of each birth defect and its corresponding ICD-10 code and the Royal College of Pediatrics and Child Health (RCPCH) extension for coding of congenital anomalies. The categories of birth defects that were included in the atlas were the same seven external, observable birth defects reflected in the manual.
Birth Defects Surveillance Training: Facilitator’s Guide
As noted earlier, a primary goal of Birth Defects COUNT is to build sustainable birth defects surveillance capacity globally. It was very important to the group, then, to develop a facilitator’s guide that could be used by in-country trainers for their own audiences. The facilitator’s guide is made up of 5 modules that align with the information presented in the manual. The guide is a scripted tool that is meant to be used in conjunction with both the manual and the photo atlas. It provides the facilitator with step-by-step guidance that can be used to facilitate an interactive/hands-on birth defects surveillance training session.
Both the manual and the facilitator’s guide have an accompanying participant workbook developed for workshop participants. The workbook contains exercises that allow the participant to practice the various skills necessary to conduct birth defects surveillance. For example, the workbook contains an exercise that requires the participant to describe and diagnose a birth defect based on an illustration or photograph, and assign the correct ICD-10 code. The workbook can also be used to help facilitate discussion among the workshop moderators and participants, to highlight areas that are not well understood, and to provide additional practice for workshop participants.
Resource Library: PowerPoint Presentations
The toolkit includes a resource library that houses a suite of PowerPoint presentations on major topic areas presented in the manual. These were created by the BDSWG to provide a readymade, accurate, evidence-informed tool that country representatives could use during workshops or when asked to give oral presentations. Country representatives can adapt the presentations to include country-specific issues related to birth defects surveillance to make the presentations more contextually relevant. In order to use some of these presentations during online pre-courses, narrated recordings of many of them were made by BDSWG members.
PreSurv Database
ICBDSR led the development of an Epi Info©-based data collection and management tool known as PreSurv (Prevention and Surveillance of Birth Defects Suite), which is available upon request from ICBDSR. The development of the database tool was informed by existing birth defects data collection tools and provides standard variables needed for birth defects surveillance (a core minimum as well as optional, expanded variables). The aim of this tool was to provide countries with a standardized way to collect, manage, and share birth defects data. Countries using PreSurv could also then compare data within and across countries.
Virtual Training
Finally, in order to maximize the time invested in-country conducting birth defects surveillance trainings, the BDSWG discussed the value of having a virtual pre-course covering the basic surveillance information that would usually take up a day or more of time at an actual training workshop. In this way, workshop moderators and participants could allocate the workshop time to focus on more complex concepts, sharing information, and developing concrete, country-specific action steps. As a result, a virtual birth defects surveillance pre-course was developed and has been used for multiple workshops to date. Following the suggestion of partners in Africa from earlier evaluations [11], the virtual training has not only been used as the starting point of each workshop, but also as an on-line, stand-alone resource, with access granted upon request. This on-line training includes videos, documents, forums, links to external resources, and exams. Since 2014, 144 users have been granted access to the virtual training. The virtual training has proven to be an invaluable tool – the utility of which has been described in another publication [11].
Measuring Impact
An evaluation was conducted to assess the perceived usefulness, effectiveness, and policy impact of the surveillance toolkit and surveillance-related technical assistance provided to countries thus far. The evaluation sought to understand how end-users perceive the toolkit products; to assess how these products have been used to develop birth defects surveillance capacity; to identify which components of the toolkit are most useful in developing birth defects surveillance capacity; and to elicit recommendations for improving the components. This assessment will allow for the improvement of the toolkit as the number of countries using it expands.
Methods
In February and March 2016, eight individuals identified through the Birth Defects COUNT network from five countries in the African region (Tanzania, Kenya, Rwanda, Uganda, and Nigeria) were contacted via email inviting them to participate in evaluation interviews. These individuals were clinicians and program leaders. Their in-country roles included serving as technical experts, surveillance program leaders, and birth defects advocates, and all were involved with implementing components of the Birth Defects COUNT Toolkit in their respective countries. All eight individuals agreed to be interviewed and all five countries were represented in the final sample.
The process was deemed Institutional Review Board exempt. Three interviewers conducted the interviews. Six interviews were conducted via telephone and two interviews were conducted in-person. The nature and objectives of the interview and the process were explained to participants. Verbal consent was obtained from all of the interviewees. Interviewers took notes, and when available, additional note takers were present.
The interview guide covered multiple topics, but the analysis presented here is based on interview notes related to the following interview guide questions:
What was most useful about the birth defects surveillance tool kit? (Probe: How was it useful?)
What was least useful about the birth defects surveillance tool kit? (Probes: Why wasn’t it useful? and How can it be improved?)
- Thinking about the birth defects surveillance manual, atlas, facilitator’s guide, and other resources:
- What have you used and how has the [toolkit item] been used?
- What has been the most useful part of [toolkit item]?
- How could [toolkit item] be more useful?
- Have you distributed [the toolkit component]? If so, to whom?
Each respondent was offered the opportunity to comment about each toolkit component during the interviews. However, progress towards establishing birth defects surveillance capacity varied among the countries, so each participant had not used and thus, could not comment about every component.
MAXQDA software (Release 12.1.0) was used to analyze the notes. Analytical categories, or codes, reflected the evaluation’s purpose. Therefore, content from the notes was grouped into the following categories for each toolkit component: how the component was used to develop in-country surveillance capacity, what was useful about the toolkit component, and recommendations for making the toolkit component more useful for developing surveillance capacity. If a response related to the evaluation purpose was recorded by note takers outside of a particular question, it was categorized appropriately.
Results
Tables 1, 2, and 3 summarize how the toolkit components have been used, what interviewees found useful about the format and content of the toolkit components, and their suggestions for improving the format and content of the toolkit components. Overall, the toolkit components were used for their intended purposes. For example, the manual and atlas were used as references for birth defects surveillance protocol development; the PowerPoints were used as a training material; the virtual trainings were used prior to in-person trainings; and the PreSurv database was used for data collection. Respondents also reported using the toolkit to help enhance in-country surveillance capacity by sharing it with colleagues, including Ministry of Health staff, Field Epidemiology Training Program fellows, health care providers, and program coordinators.
Table 1:
Uses of, Useful Characteristics of, and Suggested Improvements to the Birth Defects COUNT Manual for Programme Managers *
| Uses | Useful Characteristics | Suggested Improvements |
|---|---|---|
|
Content
|
Content
|
Table 2:
Uses of, Useful Characteristics of, and Suggested Improvements to the Birth Defects COUNT Atlas of Selected Congenital Anomalies
| Uses | Useful Characteristics | Suggested Improvements |
|---|---|---|
|
Content
|
Content
|
Table 3:
Uses of, Useful Characteristics of, and Suggested Improvements to the Birth Defects COUNT PowerPoint Presentations, PreSurv Database, and Virtual Training Modules
| Toolkit Component | Uses | Useful Characteristics | Suggested Improvements |
|---|---|---|---|
| PowerPoint Presentations |
|
Content
|
Content
|
| PreSurv |
|
Content
|
Content
|
| Virtual Training |
|
Content
|
Content
|
Interviewees found the content and the format of toolkit components useful in developing birth defects surveillance capacity. Specifically, respondents found illustrations and pictures in the toolkit useful, as well as explanations of ICD coding. Recommendations for improvement included tailoring materials to the country context (i.e. changes to pictures and languages) and making them less technology dependent.
Manual for Programme Managers
Participants from three countries were using the birth defects surveillance manual for surveillance protocol development, and two countries were using it to train teams/staff (Table 1). One respondent mentioned having used the manual as an advocacy tool to raise awareness among decision-makers as to the importance of, and need for, birth defects surveillance. Three respondents from three countries indicated that they felt as if the International Classification of Diseases (ICD-10) coding descriptions included in the manual were its most useful feature. A respondent from a country in the beginning stages of birth defects surveillance capacity development said that the manual’s content was useful to him because it explained how to establish a birth defects surveillance program, the importance of birth defects surveillance, and important details about birth defects surveillance. Improvements to the manual’s content suggested by interviewees included adding information about challenges faced when implementing birth defects surveillance and associated solutions, consistent use of the terms congenital anomalies and birth defects, as well as a list of surveillance Standard Operating Procedures.
Atlas of Selected Congenital Anomalies
One respondent reported using pictures from the atlas during the writing of a proposal about birth defects surveillance activities, while three respondents from two countries reported using the atlas as a summary or reference document (Table 2). Three respondents representing two countries reported that the pictures/visual nature of the atlas was its most useful characteristic. The notes indicated that one respondent described the atlas photographs as “feeling so real.” Another interviewee praised the atlas for being visual, clear, simple, and at the same time detailed. Three respondents from three countries found the ICD coding information to be the most useful component of the atlas. The note taker documented that one respondent described the ICD-10 coding descriptions as the most useful part of the atlas because they are clear and help “de-mystify” the coding. Recommendations for improvement included increasing the types of birth defects included, adding additional case scenarios, and including pictures of African babies.
Facilitator’s Guide
Interviewees’ feedback about the Facilitator’s Guide was sparse. When responses were provided, participants reported not using it or being unfamiliar with it. Therefore, this toolkit component has been excluded from the tables. However, one participant mentioned using the Facilitator’s Guide to train staff, but did not provide feedback about the content or the delivery because they were not the person using the document. This lack of feedback was not unanticipated as the Facilitator’s Guide had not been widely distributed at the time of this evaluation.
PowerPoint Presentations
Three respondents from two countries reported using the PowerPoint slide sets for training/orienting staff. One respondent reported using them to develop additional training slides. The interview notes indicated that one participant found the narration of the slides useful because the content was better conveyed when the presenter’s voice could be heard. Another respondent stated that the most useful characteristic of the PowerPoint slides were that they are that “specific.” Recommendations for improvement included adding literature about the prevalence of birth defects specifically in Africa, making the references clearer, adding more examples related to epidemiology, and increasing the amount of content (Table 3).
PreSurv Database
Use of the PreSurv database collection tool varied. Two respondents from two countries indicated not having used this tool often. The reasons for this were mixed, however. One respondent indicated that their country was already using another data collection tool for birth defects surveillance.
In contrast, another respondent reported that, although PreSurv was modified to accommodate their country’s surveillance capacity, not much time had been spent to practice using it. In countries where birth defects surveillance activities were being implemented, PreSurv was perceived as a tool that helped save resources and manage data. For example, a respondent from one of the two countries implementing surveillance activities said that PreSurv saved them time and money because they did not have to hire a database developer (Table 3). A respondent from another country implementing birth defects surveillance activities said that PreSurv was useful because data did not have to be imported for analyses, and that PreSurv would be useful in preventing data errors. Interviewees recommended that the PreSurv database be improved by including medical terms used in the country on the form and providing a shorter version of the surveillance form.
Virtual Training
Only two respondents indicated extensive use of the virtual training. Limited data were anticipated, as the pre-course has primarily been used by partner groups prior to workshops and has not been promoted as a tool that the country representatives themselves would use. Participants said that the virtual training could be improved by adding more elements that reinforce learning (Table 3). For example, allowing time for questions and making the multiple choice questions more challenging. It was also recommended that content about surveillance and data quality be added to the training.
Recommendations for Overall Improvement
Recommendations for overall improvement focused on customization to country contexts. Specifically, three respondents from two countries requested the addition of photographs or illustrations of African babies in the atlas. Additionally, respondents from two countries mentioned language barriers related to the toolkit. One recommended including country-specific medical terms. Another recommended that the manual be translated into more languages.
Interviewees also had multiple recommendations to make the electronic and physical formats of the toolkit documents more amenable to country contexts. Representatives from two different countries suggested delivery formats for toolkit components that were less reliant on technology. One respondent recommended that the manual, which has a file size of 3.35Mb, be distributed on flash drives or compact discs (CD) because internet access limitations made it difficult to download. Another participant mentioned that making the atlas available in black and white print which could be easily photocopied would allow the document to be more accessible to those without computers.
Recommendations for changing the physical format of the toolkit documents were related to the size of the documents. Specifically, one respondent mentioned that the atlas should be smaller in size so it can fit into a clinician’s pocket. Another recommended that the manual should be printed in a way that reduces the physical bulk of the document.
Conclusions
The evaluation process was very valuable in assessing perceptions about the use and usability of the surveillance toolkit components among the African partner organizations, and will allow for the revision of existing products to maximize their use and relevance. Following this evaluation, many of the recommendations provided by the respondents were used (i.e., including photographs and illustrations of African babies in Toolkit products, translating into additional languages), and will continue to be used, to improve these products.
Development of the birth defects surveillance toolkit has been a lengthy process and is ongoing. Pilot testing, regular reviews and revisions, and the evaluation process outlined in this paper have helped to ensure that the products are relevant to the intended audiences and will be used to develop and launch birth defects surveillance. Translation into additional languages has also been key to maximizing the use of the toolkit products.
In a time when travel is costly and funds are limited, it can be difficult to provide in-person technical assistance upon request. The toolkit serves as one way to increase the reach of technical assistance and continue to provide countries with the guidance they request to be able to implement high-quality birth defects surveillance for their assessment of the effectiveness of their programs to attain the Sustainable Development Goal.
Footnotes
The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
References
- 1.Christianson A, Modell B, Howson C. March of Dimes global report on birth defects: The hidden toll of dying and disabled children. 2006; White Plains: NY. [Google Scholar]
- 2.World Health Organization (WHO). [Internet]. Congenital anomalies: Fact sheet. [cited 2016 Nov 11]. Available from: http://www.who.int/mediacentre/factsheets/fs370/en/
- 3.World Bank. World development report 1993: Investing in health; New York: Oxford University [Google Scholar]
- 4.United Nations (UN). [Internet]. We can end poverty: Millenium development goals and beyond 2015. [cited 2015 May 7]. Available from: http://www.un.org/millenniumgoals/bkgd.shtml
- 5.World Health Organization (WHO). [Internet]. Sixty-third World Health Assembly closes after passing multiple resolutions. [cited 2015 May 7]. Available from: http://www.who.int/mediacentre/news/releases/2010/wha_closes_20100521/en/
- 6.United Nations. [Internet]. Sustainable development goals. [cited 2015 October 10]. Available from: https://sustainabledevelopment.un.org/?menu=1300
- 7.Rasmussen SA, Jamieson DJ, Honein MA, Petersen LR. Zika Virus and Birth Defects — Reviewing the Evidence for Causality. New England Journal of Medicine 2016; 374:1981–1987. [DOI] [PubMed] [Google Scholar]
- 8.CDC grand rounds: Additional opportunities to prevent neural tube defects with folic acid fortification. Morbidity and Mortality Weekly Report 2010;59(31);980–984. [PubMed] [Google Scholar]
- 9.CDC. Updated estimates of neural tube defects prevented by mandatory folic acid fortification – United States, 1995–2011. Morbidity and Mortality Weekly Report 2015;64(01):1–5. [PMC free article] [PubMed] [Google Scholar]
- 10.Zaganjor I, Sekkarie A, Tsang BL, et al. Describing the prevalence of neural tube defects worldwide: A systematic literature review. PLoS One 2016. Apr 11;11(4):e0151586. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Flores A, Valencia D, Sekkarie A, et al. Building capacity for birth defects surveillance in Africa: Implementation of an intermediate birth defects surveillance workshop. Journal of Global Health Perspectives 2015;Edition 1. [PMC free article] [PubMed] [Google Scholar]