Highlights
-
•
Global and regional action plans call for improved immunization data quality.
-
•
Mexico and Peru created information systems to improve data quality.
-
•
Mexico’s immunization registry failed, but the country is reworking its system.
-
•
Peru’s system has improved data quality, while addressing social priorities.
-
•
Funding and capacity-building affect sustainability of immunization registries.
Keywords: Electronic immunization registries, Immunization information systems, Immunization data, National immunization programs, Global vaccine action plan
Abstract
Introduction
The Global Vaccine Action Plan and the Regional Immunization Action Plan of the Americas call for countries to improve immunization data quality. Immunization information systems, particularly electronic immunization registries (EIRs), can help to facilitate program management and increase coverage. However, little is known about efforts to develop and implement such systems in low- and middle-income countries. We present the experiences of Mexico and Peru in implementing EIRs.
Methods
We conducted case studies of an EIR in Mexico and of a population registry in Peru. Information was gathered from technical documents, stakeholder focus groups, site visits, and semi-structured interviews of national stakeholders. We organized findings into narratives that emphasized challenges and lessons learned.
Results
Mexico built one of the world’s first EIRs, incorporating novel features such as local-level tracking of patients; however, insufficient resources and poor data registration practices led to the system’s discontinuation. Peru created an information system to improve affiliation to social programs, including the immunization program and quality of demographic data. Mexico’s experience highlights lessons in failed sustainability of an EIR and a laudable effort to reform a country’s information system. Peru’s demonstrates that attempts to improve health and other data may strengthen health systems, including immunization data. Major challenges in information system implementation and sustainability in Peru and Mexico related to funding, clear governance structures, and resistance among health workers.
Discussion
These case studies reinforce the need for countries to ensure adequate funding, plans for sustainability, and health worker capacity-building activities before implementing EIRs. They also suggest new approaches to implementation, including economic incentives for sub-national administrative levels and opportunities to link efforts to improve immunization data with other health and political priorities. More information on best practices is needed to ensure the successful adoption and sustainability of immunization registries in low- and middle-income countries.
1. Introduction
High-quality immunization data facilitate the management, financial planning, and vaccine forecasting capacities of national immunization programs (NIPs)1 [1], [2]. A prerequisite for quality data is an information system that monitors vaccine administration and facilitates the aggregation and analysis of coverage information.
Since 2002, the Pan American Health Organization’s (PAHO) Technical Advisory Group (TAG) on Vaccine-preventable Diseases for the Region of the Americas has issued recommendations for countries to improve the quality of their immunization data and information systems [3], [4], [5], [6], [7], [8]. In addition to implementing plans of action for improving data quality, PAHO has provided technical assistance to countries developing or upgrading electronic immunization registries (EIRs) [6], [9], [10]. In 2009, PAHO’s TAG recommended that countries document their experiences with electronic immunization registries [5]. Additionally, the Global Vaccine Action Plan 2011–2020 (GVAP) and the Regional Immunization Action Plan 2015–2020 (RIAP) in the Americas reinforced the importance of improving immunization data quality [11], [12].
As confidential, computerized, population-based registries that track administered vaccine doses, EIRs serve important functions at all levels [6]. Locally, registries enable health workers to identify persons due for vaccination and to monitor defaulters [13], [14], [15]. Sub-nationally and nationally, EIRs facilitate coverage monitoring by vaccine, dose, age, and other variables [6], [13], [16]. EIRs also facilitate analysis of vaccine safety data, support research initiatives, promote better understanding of vaccine refusals, and inform the design of coverage interventions and outbreak investigations [13], [17].
In the last decade, countries in the Americas have transformed their immunization information systems [16]. As of 2017, 132 countries and territories have introduced EIRs and eight3 are creating or implementing these systems [18]. Recent developments include incorporating EIRs into larger health information systems and linking registries to mobile technologies [16], [19]. In 2014, Danovaro-Holliday et al. identified lessons learned on EIR implementation. These included the need for permanent financial and human resources and the value of making EIRs useful for health workers to ensure long-term sustainability of the systems [16].
Despite the importance of strengthening data quality to the GVAP, there are few published reports on efforts to implement EIRs in low- and middle-income countries [10], [20], [21]. In this article, we present the experiences of Mexico and Peru in implementing person-based information systems. We describe how these systems developed as well as the challenges and lessons learned in their implementation. Our findings should be useful to countries seeking to improve immunization data quality and to those developing or strengthening EIRs.
2. Methodology
In 2015 and 2016, we conducted case studies of immunization information systems in Mexico and Peru. These countries were chosen based on their distinct experiences and potential value to other nations. We chose Mexico to document the experience of one of the largest and most diverse countries in pioneering one of the first EIRs in Latin America and the Caribbean. Additionally, we wanted to highlight lessons learned from Mexico’s attempts to improve data quality after discovering that its system was not producing accurate coverage levels [22]. We chose Peru to document the county’s unique approach of developing a children population registry not only for the NPI but also for social programs, such as a civil registry. In this respect, we wanted to study how EIRs might be connected to other systems and facilitate inter-ministerial coordination. Both cases were evaluated as part of PAHO’s Improving Data Quality for Immunization (IDQi) project [23].
Used commonly in the social sciences, case studies are qualitative evaluations of individuals, institutions, or events that highlight objective data (e.g., facts and key actions) and provide practical information (e.g., reasons for a program’s success) to audiences in similar situations [24], [25]. By distinguishing a subject from its contexts, case studies offer comprehensive explanations of events or developments [24]. In these case studies, the subjects are Mexico and Peru’s EIRs, while the contexts are the countries’ health systems, economic situations, and political structures.
We gathered information by reviewing national documents on vaccination and information systems, holding focus groups of national stakeholders, conducting semi-structured interviews of individuals, and making visits to sites where immunization data are recorded and analyzed. Initial interviews with stakeholders and a review of documentary evidence helped us to develop interview questions related to the structure and evolution of the immunization information system and to challenges with and lessons learned from implementation. In Peru, we conducted twenty semi-structured interviews and two site visits, one in the regional health network in Cusco (Red de Salud Norte) and another in the municipality of Calca in Cusco. During site visits, we met with immunization officials, municipal authorities, statisticians, managers of information systems, and officials responsible for other health programs (e.g., nutrition). In Mexico, one focus group session was held, with participation of officials from key national agencies, including the NIP, General Directorate of Health Information (DGIS), and National Center for Childhood and Adolescent Health (CENSIA).
We analyzed data by reviewing background documents and transcripts of semi-structured interviews. Information was coded with short phrases, and codes were grouped to identify the main factors affecting information systems in Mexico and Peru [26], [27]. To make results more understandable, we organized major findings into challenges and lessons learned.
3. Results
3.1. Mexico
3.1.1. Health system and National immunization program
In 1973, Mexico established its NIP, which, in 1991, became the Universal Vaccination Program (PVU)4 [28]. At the national level, the Secretary of Health manages the PVU, with additional oversight from the Social Security Institute (IMSS) and the Institute of Social Security and Services for State Workers (ISSSTE). At the local and state levels, health services and the IMSS, ISSSTE, and other public agencies implement vaccination activities and partner with health institutions to offer immunization services. National and state immunization technical advisory groups collaborate with the PVU to standardize schedules and ensure vaccine availability [29].
3.1.2. Immunization information system
In 1991, Mexico established PROVAC, a computerized program to record immunization data in all of the country’s health facilities, including private facilities [28] (Table 1). In 1995, the PVU began organizing immunization records in PROVAC by vaccine type. At this time, the Secretary of Health added nutrition indicators (weight and height), and, in 1999, a unique identifier was added to reduce the number of duplicate entries. In 2000, officials separated administrative records from national census data and started using only PROVAC to determine numerators to calculate coverage rates, using population projections as denominators.
Table 1.
Description of PROVAC, immunization information system in Mexico.
| Description: Established in 1991, PROVAC served as the electronic immunization registry of Mexico until 2013. Though not online, PROVAC was a computer-based system that registered the immunization status of children and pregnant women. The registry included each federal entity’s data and was maintained until 2013 | |||||
|---|---|---|---|---|---|
| Objectives | Information included | Flow of information | Government entities involved | Technical support | Legal basis |
|
Demographics:
Health data:
|
|
|
|
|
Though official coverage rates were high during this time, there were concerns that the levels might be falsely elevated. During the system’s 18-year history, Mexico developed different versions of PROVAC both because the program was open-source and because changes were required to include eight vaccines added to the PVU from 1991 to 2008.
In 2013, Mexico stopped using PROVAC due to coverage discrepancies. The National Population Council (CONAPO) had completed a data reconciliation with information from the National Institute of Statistics and Geography, which showed that PROVAC’s denominators were underestimated and that coverages were thus overestimated. Coverages estimated at near 100% dropped to around 80% (e.g., 99% DTP3 for 2012 vs. 83% DTP3 for 2013) [30], [31].
3.1.3. Implementation challenges
PROVAC faced many challenges, including the accelerated inclusion of new vaccines and insufficient resources and information technology personnel devoted to the system (Table 2). Furthermore, interviewed officials at different levels of the Mexican health system indicated that the major cause of PROVAC’s discontinuation was poor data recording practices.
Table 2.
Challenges and lessons learned, PROVAC, immunization information system in Mexico.
| Challenges | |
| Governance and sustainability | *Lack of central funding for staffing and system maintenance |
| Human resources | *Difficulty securing and retaining information technology staff |
| Processes | *No evaluation comparing the system’s estimates to those from other data sources, eventually resulting in a large coverage adjustment and more work in the long-term *Insufficient ability to adapt to different users' information needs and to changes in national circumstances |
| Tools | *Open-source and open-access program resulting in multiple versions of the same program, in which different institutions modified the software according to their preferences |
| Data use and data quality | *Lack of a culture supporting high quality data practices at all levels |
| Lessons learned | |
| Governance and sustainability | *Activities empowering community members can promote program sustainability *Value of involving the private sector in all stages of implementation *Empowerment and involvement of local governments *An eHealth strategy allows for a sustainable framework for developing the country's health system |
| Human resources | *Promote efficient staffing *Upon implementing a new system, as done after PROVAC, the country should expect resistance on the part medical units responsible for data registration. One solution is to carry out progressive trainings and educational activities in these units. |
| Processes | *Improved ability to quantify target population |
| Tools | *Use of single information system may result in time-savings |
| Data use and data quality | *In calculating coverages, the sources of the numerators and denominators must be determined to anticipate possible causes of underestimates or overestimates. Mexican officials did not realize that the numerator in the PROVAC system was based on doses distributed doses, rather than administered doses. This resulted in overestimated coverage. *Complete systematic evaluations of data quality as part of the routine program |
PROVAC’s open coding exposed it to errors. Between and within regions, users and administrators created different versions of the system, resulting in inconsistencies and the potential for data manipulation. Complicating matters were errors in numerators and denominators. Denominators were never validated against CONAPO’s data, and some numerators had been based on distributed rather than administered doses. This resulted partly from PROVAC’s design. Although health workers linked administered doses to patients at the local level, the system was never online, meaning coverage information at higher levels was based on aggregated data and prone to substitute estimations (i.e., doses distributed rather than doses administered).
3.1.4. Lessons learned and next steps
In 2013, Mexico acknowledged the poor quality of vaccination coverage estimates and began to revamp its information system (Table 2). While developing a new EIR, the country returned to using administrative doses to calculate coverage, started to modernize its informatics system, and began to revise local and regional population estimates. These efforts have been globally recognized, and other countries have been encouraged to follow Mexico’s example of transparency and accountability [31].
Nevertheless, the transition has been difficult due to challenges in coordinating public and private providers of immunization services and because of Mexico’s fragmented health system with multiple providers and health insurance mechanisms [32], [33]. Some health workers resent returning to recording administered doses on paper, and since the system is no longer individualized, it may be more difficult to identify undervaccinated children. To address these challenges, the government has held trainings for health workers on why changes to the EIR occurred and on how to use the new system.
3.2. Peru
3.2.1. Health system and the National immunization program
Peru established its NIP in 1979. Coverages have generally been high (DTP3 88–95% between 2012 and 2016) [34]. In the early 2000s, Peru began decentralizing its government, and the NIP was integrated into the Ministry of Health’s (MoH) Comprehensive Child Health Care program [35], [36]. Renamed the National Immunization Strategy (ESNI) in 2004, the immunization program offers 15 free vaccines [35]. National agencies involved in immunization include the Ministry of Economics and Finance (MEF); the Ministry of Development and Social Inclusion (MIDIS), which oversees efforts to reduce health inequalities; and the National Registry of Identification and Civil Status (RENIEC), which is charged with issuing each Peruvian a National Identification Document (NID) [37], [38].
3.2.2. The Padrón Nominal and its impact on immunization data
In 2012, Peru created a census database, the Padrón Nominal (PN), of children aged <6 years to improve affiliation to social programs and the quality of health and demographic data (Table 3).
Table 3.
Description of the Padrón Nominal, immunization information system in Peru.
| Description: Established in 2012, the Padrón Nominal is a census database that allows online registration of children aged <6 years. The system operates in approximately 2500 health facilities. Health workers use the system to verify children’s identities and to update and validate their immunization records | |||||
|---|---|---|---|---|---|
| Objectives | Information included | Flow of information | Government entities involved | Technical support | Legal basis |
| 1. Provide State programs data for planning and budgeting 2. Identify gaps in insurance and access to health and education services 3. Provide an updated, standardized registry of children aged <6 years at the district level 4. Identify children without national identification documents (NIDs), so that they can be included in civil registries 5. Outfit regional governments with a tool to manage interventions to improve health of children aged <6 years |
36 variables in 5 categories: 1. Service site (e.g., health center code) 2. Identification (e.g., birthdate and NID) 3. Participation in educational and social programs (e.g., school code) 4. Relation to head of household (e.g., father’s last name) 5. Mother’s identification and poverty level (e.g., mother’s native language) |
-Municipal officials enter children’s data in online system -Denominator (i.e., children aged <6 years) updated via birth certificates issued in hospital and through civil registries and review of NIDs. Municipal and civil registry officials search for new children to include in the system -System updated on daily basis |
-Ministry of Health -Ministry of Economics and Finance -National Registry of Identification and Civil Status (RENIEC) -Ministry of Development and Social Inclusion |
-National Registry of Identification and Civil Status (RENIEC) is charged with administering the system, including granting access to users and troubleshooting problems -Ministries of Education and Health review processes and technical information as needed |
-Law 29,332 (2013): creates “Incentives Plan to Improve Municipal Management” Supreme Decree 002–2013-EF: Approves procedures to achieve goals and assign resources for “Incentives Plan to Improve Municipal Management” Resolution 389 (2017): Approves the Padrón Nominal at the district level for children aged <6 years |
Three major political issues led to the PN’s creation. The first was the longstanding problem of citizens lacking representation in civil registries [38], [39]. In 2010, an estimated 4.7 million Peruvians (16% of the population), 4.6 million of whom were aged <18 years, lacked NIDs, and could not fully access social services or exercise their constitutional rights [39]. Second, growing pressure to reduce economic and health disparities underscored the need to develop systems to detect low vaccination coverages [40]. Finally, the MoH and MEF argued that a database that permitted monitoring of individuals would improve the performance of ESNI and health programs.
Developing the PN required Peru to evaluate its birth registration practices. Since 2004, the ESNI had used a paper-based registry linked to the MoH’s health and target data from census projections and, starting in 2016, had used birth registration systems to calculate immunization coverage [36]. Previously, the MoH required health workers in delivery rooms to record all live births in a computer database [41]. Although this practice should have resulted in the creation of birth certificates and subsequent generation of NIDs, the system missed many children [39].
To capture children without documentation, the MoH, MEF, and MIDIS incentivized municipalities to identify all children aged <6 years in their areas through collaboration with health facilities and civil registration offices. The MEF required local governments to capture all users of immunization services in registries as a condition of financing the PN [42]. Similarly, MIDIS incentivized municipalities to implement the PN through its “Sello Muncipal” program, which provided recognition and financial incentives for municipalities for completing various health indicators, including implementation of the PN. In response, local governments employed different strategies for data collection. Some conducted house-to-house searches; others, particularly large municipalities, requested that health facilities generate lists of all persons aged <6 years in their jurisdictions.
From 2013 to 2016, these incentives led to widespread use of the Padrón. The PN started collecting immunization data and soon expanded to nutrition and other health programs. According to local and national officials, the system’s advantages include saved time through the use of a single data entry point for different interventions, better information for planning and decision-making (e.g., more accurate vaccine forecasting), generation of NIDs for children not recorded in the civil registry, and improved identification of target populations for various health interventions. In 2016, MEF decided to stop funding incentives to use the PN, believing that the system had been sufficiently established in the municipalities. However, in 2017, funding resumed, and many municipalities continue to use the system today.
3.2.3. Implementation challenges
Challenges in implementing the PN include reconciling discrepancies in denominator data (PN’s population figures vs. census data), difficulties connecting the system to nominal registries, and lack of funding from the MEF in 2016 (Table 4). The Region of Cusco illustrates the PN’s achievements and challenges (Fig. 1).
Table 4.
Challenges and lessons learned, Padrón Nominal, immunization information system in Peru.
| Challenges | |
| Governance and sustainability | *Lack of economic incentives for implementation in 2016 *Change of regional and local governments *Lack of involvement of the private sector *Occasional lack of support from mayors |
| Human resources | *Health worker perception that more time was spent using the system than serving patients *High labor burden, resulting in incorrect data registration practices *Low health worker payments and high turnover |
| Processes | *Need to more effectively communicate the PN’s advantages to users, national authorities, and the community |
| Tools | *Existence of different formats *Lack of connection between the PN with other information systems of the different government entities involved--specifically, the ESNI did not have the PN connected to its immunization information system |
| Data use and data quality | *Insufficient use of information from the PN by some users and involved programs |
| Lessons learned | |
| Governance and sustainability | *Empowerment and involvement of local governments *Strengthened cooperation among government agencies in different sectors *Economic incentives serve as good motivation for implementation, but if they are unavailable, municipalities may not continue using the system *Activities empowering community members can promote program sustainability *Value of involving the private sector in all stages of implementation *Coordination between government workers at different levels and in different sectors (i.e., both health and municipal public employees) |
| Human resources | *Education and training sessions are necessary to counter expected health worker resistance |
| Processes | *The system facilitates more realistic budgeting practices *Continual feedback on the implementation of the PN was key, and different indicators for monitoring should be established |
| Tools | *Feasible to link population-based immunization information systems with the nominal registry of Peru |
| Data use and data quality | *There is multisectoral interest in programs like PN, since their data are useful in different sectors |
Fig. 1.
Regional case study, Cusco, Peru.
3.2.4. Lessons learned and next steps
Due to the aforementioned challenges, the PN remains incompletely implemented. Even so, the system’s capacity to identify children lacking health or social interventions and to generate useful data for decision-making suggests that it may become a permanent policy tool (Table 4). The PN has promoted better cooperation among government agencies—MoH (vaccines), MEF (public financing), MIDIS (social programs), and RENIEC (civil registration)—and is an excellent example of a multi-purpose information system. Other lessons learned in implementation include the value of economic incentives, the effectiveness of linking immunization advocacy to larger political movements (e.g., lack of civil registration), and the importance of trainings for health workers and community members. As officials in Cusco indicated, the sustainability of the PN ultimately requires the community’s investment. By educating the public on the benefits of the Padrón, including access to health services, civil registration, and social services, parents may come to request the system and increase pressure on the government to sustain the program.
4. Discussion
Extending the benefits of immunization to all people requires reliable and timely immunization data [1]. Mexico and Peru are two examples of national initiatives to improve data quality. Having created one of the world’s first EIRs in 1991, Mexico ultimately devoted insufficient resources and attention to PROVAC, resulting in poor data registration practices and the system’s discontinuation. In 2012, Peru created the Padrón Nominal, which incorporated some of PROVAC’s best features, including non-vaccine indicators and local-level tracking of patients. Mexico’s experience shows both failed implementation and a laudable effort to reform its data system. Peru’s demonstrates that efforts to improve immunization data may strengthen health systems, increase the uptake of other health interventions, and address political and social priorities.
Compared to other aspects of NIPs in low- and middle-income countries, EIRs may be understudied and underappreciated. While sustainable funding and health worker training are key determinants of EIR sustainability, relatively little is known about how low- and middle-income countries can sustain the systems [14]. Countries likely recognize the importance of key components of an enabling environment—i.e., funding, capacity building, and a trained workforce—but struggle with implementation due to limited resources and competing priorities. Based on Mexico and Peru’s experiences, we wish to highlight four lessons relevant to sustainable implementation of EIRs.
First, rationales for establishing or strengthening EIRs may extend beyond immunization to other health and political priorities. As immunization integrates into other health services, governments are more likely to support proposals for multi-purpose interventions. The PN’s creation in Peru, supported by at least four governmental agencies, exemplifies this point. Second, EIR sustainability requires the investment of various government agencies, and investment might be increased if each agency understands what it stands to gain. In Peru, the MEF sought better data for planning purposes, the MoH sought higher coverages, and the MIDIS sought more equitable health outcomes. In designing EIR proposals and sustainability plans, health officials should thus consider how the system will benefit all stakeholders. Third, predictable funding is necessary, and countries lacking funding should likely delay implementation until resources are available and ideally protected by long-term strategies, with clear governance structures and budgets secured by legislation. Countries may consider innovative funding models, such as the economic incentives used in Peru. These models may facilitate implementation and prove cost-effective if they improve service quality.
Finally, EIR sustainability depends on the system’s quality and adaptability. Recommendations exist for variables, such as the child’s age, and for guiding principles, such as entering data as close to vaccination as possible in terms of time and place [16]. To these, we might add that program developers and national officials must balance pressures to modify the EIR for user preference with the need to maintain a uniform system across the country, particularly as Mexico’s system’s open-source coding contributed to its ultimate downfall. Countries should have information technology personnel devoted solely to the management and updating of the EIR.
Several limitations should be acknowledged. Case studies offer in-depth explanations of complex events with multiple causes but are criticized as lacking rigor and producing non-generalizable results [24]. Although we tried to use thorough qualitative methods, different investigators may have reached different conclusions. In evaluating the applicability of our findings to immunization data quality improvement efforts in other countries, national officials should consider their countries’ health, political, and social factors. Additionally, the PN, though promising, is an ongoing project whose ultimate effectiveness is unknown. Finally, while there is considerable evidence associating EIRs and eHealth initiatives with improved immunization coverage levels, these impact studies are from developed countries [13], [43], [44]. Impact studies of EIRs in low- and middle-income countries are needed to better understand these interventions.
To deliver the benefits of vaccination to all persons, countries in the Americas and other WHO regions must continue to improve their immunization information systems. Mexico and Peru’s experiences should be helpful to other countries seeking to use EIRs. More information on best practices and lessons learned is needed to ensure the successful implementation of EIRs in low- and middle-income countries.
Declaration of competing interests
The authors have no competing interest to declare.
Funding
Grant # OPP1084485 (2013), Bill and Melinda Gates Foundation.
Acknowledgments
Acknowledgments
The authors thank the national officials and health workers in Mexico and Peru who participated in these case studies and who work to ensure that all children in their countries have access to life-saving vaccines. We also thank Elsy M. Dumit, MPH, and Robin Mowson, MA, MPH, CPH, for their helpful reviews of this manuscript. Lastly, we appreciate the financial support from the Bill and Melinda Gates Foundation that made this work possible.
Contributors
MC, AGFG, MCDH, and MVG conceived of and designed the study. FAED led country site visits, with input from MC, MVG, and MCDH, and provided initial data analysis. FAED, SPT, AGFG, and MC participated in the initial data analysis. SPT drafted the manuscript with initial revisions from MCHD, MVG, MC, and AGFG and subsequent feedback from all authors. MG, VC, KJPR, and RA participated in the project and provided key inputs from their countries, validating that all national information was correct and complete. All authors have reviewed and approved the final version of the manuscript. All authors confirm they meet the ICMJE criteria for authorship.
Footnotes
National immunization programs (NIPs); Pan American Health Organization (PAHO); Technical Advisory Group (TAG); electronic immunization registries (EIRs); Global Vaccine Action Plan (GVAP); the Regional Immunization Action Plan (RIAP); Improving Data Quality for Immunization (IDQi); General Directorate of Health Information (DGIS), and National Center for Childhood and Adolescent Health (CENSIA); Universal Vaccination Program (PVU); Social Security Institute of Mexico (IMSS); Institute of Social Security and Services for State Workers of Mexico (ISSSTE); National Population Council of Mexico (CONAPO); Ministry of Health (MoH); Ministry of Economics and Finance of Peru (MEF); Ministry of Development and Social Inclusion of Peru (MIDIS); National Registry of Identification and Civil Status of Peru (RENIEC); National Identification Document (NID); and Padrón Nominal (PN).
Argentina, Belize, Brazil, Chile, Colombia, Costa Rica, Grenada, Guatemala, Honduras, Panama, Uruguay, almost every state in the United States, and some Canadian provinces.
Dominican Republic, Ecuador, Peru, Venezuela, and some Caribbean islands (Turks and Caicos, Montserrat, Saint Kitts and Nevis, and Saint Vincent and the Grenadines).
All acronyms of public agencies in Mexico and Peru are provided in Spanish (e.g., PVU for Programa Universal de Vacunación).
References
- 1.Bosch-Capblanch X., Ronveaux O., Doyle V., Remedios V., Bchir A. Accuracy and quality of immunization information systems in forty-one low income countries. Trop Med Int Health. 2009;14:2–10. doi: 10.1111/j.1365-3156.2008.02181.x. [DOI] [PubMed] [Google Scholar]
- 2.Ronveaux O., Rickert D., Hadler S., Groom H., Lloyd J., Bchir A. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull World Health Organ. 2005;83:503–510. [PMC free article] [PubMed] [Google Scholar]
- 3.Pan American Health Organization. XV Meeting of technical advisory group on vaccine-preventable diseases. Washington D.C.: 2002. <http://www1.paho.org/hq/dmdocuments/2009/tag15_2002_conclusions_Eng.pdf> [accessed August 7, 2018].
- 4.Pan American Health Organization. XVII Meeting of the technical advisory committee on vaccine-preventable diseases. Guatemala City: 2006. <http://www1.paho.org/hq/dmdocuments/2009/tag17_2006_FinalReport_Eng.pdf> [accessed August 7, 2018].
- 5.Pan American Health Organization. XVIII meeting of the technical advisory group on vaccine-preventable diseases. San Jose, Costa Rica: 2009. <http://www1.paho.org/hq/dmdocuments/2010/tag18_2009_Final%20Report_Eng.pdf> [accessed August 7, 2018].
- 6.Pan American Health Organization. XIX Meeting of technical advisory group on vaccine-preventable diseases. Washington DC: 2011. <https://www.paho.org/hq/dmdocuments/2011/vaccination-tag19-2011-FinalReport-Eng.pdf> [accessed August 7, 2018].
- 7.Pan American Health Organization. XXI Technical Advisory group meeting on vaccine-preventable diseases. Quito, Ecuador: 2013. <https://www.paho.org/hq/dmdocuments/2013/vaccine-preventable-diseases-tag21-2013-FinalReport-Eng.pdf> [accessed August 7, 2018].
- 8.Pan American Health Organization. XXII technical advisory group meeting on vaccine-preventable diseases. Washington D.C.: 2014. <https://www.paho.org/hq/dmdocuments/2014/vaccine-preventable-diseases-tag22-2014-FinalReport-Eng.pdf> [accessed August 7, 2018].
- 9.Dietz V., Venczel L., Izurieta H., Stroh G., Zell E.R., Monterroso E. Assessing and monitoring vaccination coverage levels: lessons from the Americas. Rev Panam Salud Publica. 2004;16:432–442. doi: 10.1590/s1020-49892004001200013. [DOI] [PubMed] [Google Scholar]
- 10.Ronveaux O., Arrieta F., Curto S., Laurani H., Danovaro-Holliday M.C. Assessment of the quality of immunization data produced by the national individual registration system in Uruguay, 2006. Rev Panam Salud Publica. 2009;26:153–160. doi: 10.1590/s1020-49892009000800008. [DOI] [PubMed] [Google Scholar]
- 11.Global Vaccine Action Plan Decade of vaccine collaboration. Vaccine. 2013;31(Suppl 2):B5–B31. doi: 10.1016/j.vaccine.2013.02.015. [DOI] [PubMed] [Google Scholar]
- 12.54th Directing Council Pan American Health Organization. Regional immunization action plan (RIAP) 2015. <https://www.paho.org/hq/index.php?option=com_content&view=article&id=13101&Itemid=42296&lang=en> [accessed August 7, 2018].
- 13.Groom H., Hopkins D.P., Pabst L.J., Murphy Morgan J., Patel M., Calonge N. Immunization information systems to increase vaccination rates. J Public Heal Manage Pract. 2015;21:227–248. doi: 10.1097/PHH.0000000000000069. [DOI] [PubMed] [Google Scholar]
- 14.Kempe A., Beaty B.L., Steiner J.F., Pearson K.A., Lowery N.E., Daley M.F. The regional immunization registry as a public health tool for improving clinical practice and guiding immunization delivery policy. Am J Public Health. 2004;94:967–972. doi: 10.2105/ajph.94.6.967. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kempe A., O ’leary S.T., Shoup J.A., Stokley S., Lockhart S., Furniss A. Parental choice of recall method for HPV vaccination: a pragmatic trial. Pediatrics. 2016 Mar;137(3):e20152857. doi: 10.1542/peds.2015-2857. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Danovaro-Holliday M.C., Ortiz C., Cochi S., Ruiz-Matus C. Electronic immunization registries in Latin America: progress and lessons learned. Rev Panam Salud Publica. 2014;35:453–457. [PubMed] [Google Scholar]
- 17.Fu L.Y., Weissman M., McLaren R., Thomas C., Campbell J., Mbafor J. Improving the quality of immunization delivery to an at-risk population: a comprehensive approach. Pediatrics. 2012;129:e496–e503. doi: 10.1542/peds.2010-3610. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Pan American Health Organization. Final report of the XXIV technical advisory group (TAG) meeting on vaccine-preventable diseases of the Pan American Health Organization. Panama City, Panama: 2017. <https://www.paho.org/hq/index.php?option=com_content&view=article&id=1862%3Areports-technical-advisory-group-vaccine-preventable-diseases&catid=1549%3Ainformation-products&Itemid=39430&lang=en> [accessed August 7, 2018].
- 19.Blaya J.A., Fraser H.S.F., Holt B. E-Health technologies show promise in developing countries. Health Aff. 2010;29:244–251. doi: 10.1377/hlthaff.2009.0894. [DOI] [PubMed] [Google Scholar]
- 20.Strategic advisory group of experts on immunization. 2016 midterm review of the global vaccine vaccine action plan; 2016. <http://www.who.int/immunization/sage/meetings/2016/october/1_Draft_GVAP_Assessment_report_2016_for_Yellow_Book_28_Sep_2016.pdf> [accessed August 7, 2018].
- 21.BID Initiative. PATH: BID Initiative n.d. <https://www.path.org/projects/bid-initiative.php> [accessed August 7, 2018].
- 22.Santos J.I. La vacunación en México en el marco de las “décadas de las vacunas”: logros y desafíos. Gaceta Médica de México. 2014;150:180–188. [PubMed] [Google Scholar]
- 23.Danovaro-Holliday, MC. A productive 2016 on immunization monitoring and data quality. Global immunization news. January 2017. <http://www.who.int/immunization/GIN_January_2017.pdf> [accessed August 7, 2018].
- 24.Zainal Z. Case study as a research method. J Kemanus Bil. 2007;9 [Google Scholar]
- 25.Neale P. Preparing a case study: a guide for designing and conducting a case study for evaluation input; 2006. <http://www2.pathfinder.org/site/DocServer/m_e_tool_series_case_study.pdf> [accessed August 7, 2018].
- 26.Liamputtong P. Qualitative data analysis: conceptual and practical considerations. Health Promot J Austr. 2009;20:133–139. doi: 10.1071/he09133. [DOI] [PubMed] [Google Scholar]
- 27.Ulin PR. Investigación aplicada en salud pública: Métodos cualitativos. Washington DC: Organización Panamericana de la Salud; 2006. <http://iris.paho.org/xmlui/bitstream/handle/123456789/729/9275316147.pdf> [accessed August 7, 2018].
- 28.Santos J.I. El Programa Nacional de Vacunación: orgullo de México. Rev Fac Med UNAM. 2002;45:142–153. [Google Scholar]
- 29.Díaz-Ortega J.L., Ferreira-Guerrero E., Trejo-Baldivia B., Téllez-Rojo M., Ferreyra-Reyes L., Hernández-Serrato M. Cobertura de vacunación en niños y adolescentes en México: esquema completo, incompleto y no vacunación. Salud Publica Mex. 2013;55:S289–S299. [PubMed] [Google Scholar]
- 30.Pan American Health Organization. PAHO WHO | Data and Statistics (IM) n.d. <http://www2.paho.org/hq/index.php?option=com_content&view=article&id=2043&Itemid=2032&lang=en> [accessed August 7, 2018].
- 31.Strategic advisory group of experts on immunization. 2015 Assessment report of the global vaccine action plan strategic advisory group of experts on immunization; 2015. <http://www.who.int/immunization/global_vaccine_action_plan/SAGE_GVAP_Assessment_Report_2015_EN.pdf> [accessed August 7, 2018].
- 32.Gomez-Dantés O., Sesma S., Becerril V.M., Knaul F., Arreola H., Frenk J. Sistema de Salud de México. Salud Publica Mex. 2011;53:S220–S232. [PubMed] [Google Scholar]
- 33.Unpublished report. Evaluación Internacional del Programa de Vacunación Universal de México: Informe Final; 2014.
- 34.Pan American Health Organization. Country Profiles (IM) n.d. <http://www.paho.org/hq/index.php?option=com_content&view=article&id=2577%3A2010-country-profiles-im&catid=956%3Amain-page&Itemid=2065&lang=en> [accessed August 7, 2018].
- 35.Alcalde-Rabanal J.E., Lazo-González O., Nigenda G. The health system of Peru. Salud Publica Mex. 2011;53(Suppl 2):s243–s254. [PubMed] [Google Scholar]
- 36.Ministerio de Salud, Organización Panamericana de la Salud. Evaluación internacional de la estrategia sanitaria nacional de inmunizaciones del Perú 2014. Lima: Ministerio de Salud, Organización Panamericana de la Salud; 2014.
- 37.Ministerio de Desarrollo e Inclusión Social. Ministerio de Desarrollo e Inclusión Social: ¿Quiénes somos? n.d. <http://www.midis.gob.pe/index.php/es/nuestra-institucion/sobre-midis/quienes-somos> [accessed August 7, 2018].
- 38.Reuben W, Carbonari F. Identification as a national priority: the unique case of Peru. Center for global development. vol. 4164000416. Working paper 454, May 2017. <https://www.cgdev.org/sites/default/files/identification-national-priority-unique-case-peru.pdf> [accessed August 7, 2018].
- 39.Registro nacional de identificación y estado civil. Plan Nacional Perú Contra la Indocumentación 2011-2015. January 2012. Lima, Peru. <http://www2.congreso.gob.pe/sicr/cendocbib/con4_uibd.nsf/9CB85613535A717905257C050060EC79/$FILE/plan-nacional-2011-2015.pdf> [accessed August 7, 2018].
- 40.Ministerio de la Mujer y Poblaciones Vulnerables. Plan Nacional de Acción por la Infancia y la Adolescencia 2012 – 2021. Lima, Peru. <https://www.mimp.gob.pe/webs/mimp/pnaia/pdf/Documento_PNAIA.pdf> [accessed August 7, 2018].
- 41.Curioso W.H. eSalud en Perú: implementación de políticas para el fortalecimiento de los sistemas de información en salud. Rev Panam Salud Pública. 2014;35:437–441. [PubMed] [Google Scholar]
- 42.Pan American Health Organization. Immunization Toolkit – Data Monitoring and Report in. <http://www.paho.org/immunization/toolkit/monitoring-reporting.html> [accessed August 7, 2018].
- 43.Papania M., Rodewald L., Baughman A., Dini E., Dietz V., Bernier R. For better immunisation coverage, measure coverage better. Lancet. 2006;367:965–966. doi: 10.1016/S0140-6736(06)68403-1. [DOI] [PubMed] [Google Scholar]
- 44.Dumit E.M., Novillo-Ortiz D., Contreras M., Velandia M., Danovaro-Holliday M.C. The use of eHealth with immunizations: an overview of systematic reviews. Vaccine. 2018 doi: 10.1016/j.vaccine.2018.06.076. pii: S0264-410X(18)30931-9. [DOI] [PubMed] [Google Scholar]