Abstract
Background: The World Health Organization (WHO) proposed two-dose measles vaccination coverage of at least 95% of the population and percentages of measles immunity in the population of 85%−95% in order to achieve measles elimination in Europe. The objectives of this study were: (1) to determine the measles vaccination coverage required to establish herd immunity against measles viruses with basic reproduction numbers (Ro) ranging from 6 to 60, and (2) to assess whether the objectives proposed by the WHO are sufficient to establish herd immunity against measles viruses. Methods: The herd immunity effects of the recommended objectives were assessed by considering the prevalence of protected individuals required to establish herd immunity against measles viruses with Ro values ranging from 6 to 60. Results: The study found that percentages of two-dose measles vaccination coverage from 88% to 100% could establish herd immunity against measles viruses with Ro from 6 to 19, assuming 95% measles vaccination effectiveness. The study found that the objective of 95% for two-dose measles vaccination coverage proposed by the WHO would not be sufficient to establish herd immunity against measles viruses with Ro ≥ 10, assuming 95% measles vaccination effectiveness. By contrast, a 97% measles vaccination coverage objective was sufficient to establish herd immunity against measles viruses, with Ro values from 6 to 13. Measles immunity levels recommended in individuals aged 1−4 years (≥85%) and 5−9 years (≥90%) might not be sufficient to establish herd immunity against most measles viruses, while those recommended in individuals aged 10 or more years (≥95%) could be sufficient to establish herd immunity against measles viruses with Ro values from 6 to 20. Conclusion: To meet the goal of measles elimination in Europe, it is necessary to achieve percentages of two-dose measles vaccination coverage of at least 97%, and measles immunity levels in children aged 1−9 years of at least 95%.
Keywords: measles, vaccination, immunity, measles vaccination coverage objectives, population measles immunity objectives, measles elimination, measles prevention, herd immunity
1. Introduction
Measles is a highly contagious infectious disease associated with outbreaks, hospitalizations, and deaths in Europe, as well as in other regions of the world, despite highly effective vaccines being available. The main strategy to achieve measles elimination is based on high percentages of routine measles vaccination during childhood. In 2015, the European Region of the World Health Organization (WHO) renewed their commitment to the elimination of measles by the year 2020 [1]. In the WHO European region, the mean vaccination coverage for the first dose of the measles vaccine increased from 76.8% in 1980 to 92%−94% since 2003, and the mean vaccination coverage for the second dose of the measles vaccine increased from 68% in 1995 to 91%−92% since 2004 [2]. The mean vaccination coverage with two doses of the measles vaccine increased from 58.6% in 1995 to 84%−88% since 2004 [2]. Measles vaccines have decreased the incidence and mortality from measles in Europe, but measles cases and outbreaks are still occurring.
The elimination of measles from Europe is feasible because humans are the only reservoir for measles, effective vaccines are available, highly sensitive, and specific diagnostic tests are available, and endemic measles transmission has been interrupted in America [3,4].
Unfortunately, measles cases and outbreaks increased from 2015 to 2019 [4,5,6,7,8]. In 2015, a total of 9010 measles cases, 4259 hospitalizations, and 2 deaths due to measles were reported by European countries to the WHO’s centralized information system for infectious diseases (CISID) [5,6,7]. In 2019, 16,485 measles cases, 9824 hospitalizations, and 8 deaths due to measles were reported by European countries to the CISID [5,6,7]. In 2018, 12,352 measles cases and 34 deaths due to measles were reported by countries of the European Union to the European Center for Disease Prevention and Control (ECDC) [8]. In 2019, 13,207 measles cases and 10 deaths due to measles were reported to the ECDD [9].
In the European Union in 2019, 28% of cases occurred among children under five years, 17% occurred among children 6–14 years old, and 55% occurred among individuals aged 15 or more years—71% of cases were unvaccinated and 18% had received one dose of the measles vaccine [9]. The overall incidence of measles was above the elimination target (one case per million population) in 29 (97%) countries of the European Union in 2019 [9].
Several factors may explain why measles continues to persist in Europe, including low measles vaccination coverage with two doses of the measles vaccine, low anti-measles immunity levels in areas and population groups (immunity gaps), mobility of individuals with measles across Europe, and loss of public confidence in vaccines [2,10,11]. Countries from the eastern part of Europe conducted supplementary vaccination activities to vaccinate population cohorts that were susceptible to measles, but they were not enough to achieve the vaccination coverage required to block measles transmission in Europe [11]. A recent study found that low percentages of two-dose measles vaccination coverage during 2015−2017 could be one of the factors explaining the persistence of measles during 2017−2018 in Europe [12].
The strategy plan proposed by the World Health Organization to achieve measles elimination in Europe by the year 2020 was based on four main measures [4,9,13]:
Achieve and maintain routine measles vaccination coverage with two doses of the measles-mumps-rubella (MMR) vaccine ≥95%.
Supplementary immunization activities to population groups at risk for measles and to individuals susceptible to measles.
Intensive epidemiological surveillance.
Rigorous outbreak control.
The objective of this strategy was to protect vaccinated individuals and to achieve and maintain a proportion (or prevalence) of protected individuals sufficient to establish the necessary herd immunity to block measles transmission in the community. For this reason, the strategic framework for the elimination of measles in the European Region proposed the following objectives for measles immunity in terms of proportion (or prevalence) of protected individuals: ≥85% in children aged 1–4 years, ≥90% in individuals aged 5–9 years, and ≥95% in individuals aged 10–14 years, 15–19 years, and ≥20 years [4] (Table 1). The objectives in terms of the prevalence of susceptible individuals were: ≥15% in children aged 1–4 years, ≥10% in individuals aged 5–9 years, and ≥5% in individuals aged 10−14 years, 15–19 years, and ≥20 years [14].
Table 1.
Objectives for two-dose measles vaccination coverage during childhood and for the proportion of protected individuals in different age groups proposed by the World Health Organization (WHO) and the objectives proposed in this study.
| Objectives for Measles Vaccination Coverage and Measles Immunity | Proposed by the WHO [4,13,14] | Proposed in This Study |
|---|---|---|
| Two-dose measles vaccination coverage during childhood | ≥95% | ≥97% |
| Proportion of individuals protected against measles (% susceptible) | ||
| Aged 1–4 years | ≥85% (≥15%) | ≥95% (≥5%) |
| Aged 5–9 years | ≥90% (≥10%) | ≥95% (≥5%) |
| Aged 10–14 years | ≥95% (≥5%) | ≥95% (≥5%) |
| Aged 15–19 years | ≥95% (≥5%) | ≥95% (≥5%) |
| Aged ≥ 20 years | ≥95% (≥5%) | ≥95% (≥5%) |
The age groups selected by the WHO correspond to preschool children (1–4 years), primary school children (5–9 years), secondary school children (10–14 years), and adults (≥15 years). Anti-measles immunity protection (or susceptibility) in different age groups can be assessed by means of developing serological surveys in representative samples of the population [15,16]. The WHO assumed that recommended anti-measles immunity levels were sufficient to establish the herd immunity required to block measles transmission in the community [14,17].
In Europe, routine measles vaccination is based on two doses of the MMR vaccine. Children receive their initial measles vaccine when they are 12–15 months old and the second dose when they are 3–15 years old [18]. Measles vaccination is based on two doses of the vaccine to provide protection to children that did not respond to their first dose (primary vaccination failure) and to reduce waning vaccine-induced immunity (secondary vaccination failure) [19].
The WHO and the ECDC recommend supplementary vaccination activities to close immunity gaps in adolescents and adults who have missed vaccination opportunities in the past [4,9]. Supplementary vaccination activities to vaccinate population cohorts that were susceptible to measles have been developed in countries from the eastern part of Europe [11], but screening and vaccination programs have not been implemented at either the national or regional level within Europe [20,21].
Sustained high vaccination coverage with two doses of the vaccine is the key preventive measure to successfully eliminating measles from Europe [4,9,21]. European countries have developed routine measles vaccination programs using one dose of vaccine since 1980, and routine measles vaccination using two doses of vaccine since 1995 [2]. Successful measles vaccination programs provide both direct protective effects among vaccinated individuals and indirect protective effects from herd immunity among unvaccinated individuals [22].
The establishment of herd immunity in the population depends on both the measles vaccination coverage and the following factors: (1) the effectiveness of the measles vaccination, (2) the duration of vaccine-induced immunity, (3) the average number of measles cases generated by a measles case in a totally susceptible population (basic reproductive number), and (4) measles immunity levels in unvaccinated population groups. The objectives of this study were: (1) To determine the measles vaccination coverage and measles seroprevalence required to establish herd immunity against measles viruses, and (2) to assess whether the objectives proposed by the WHO for routine measles vaccination coverage and population anti-measles immunity are sufficient to establish herd immunity against different measles viruses.
2. Materials and Methods
2.1. Critical Measles Vaccination Coverage Associated with Herd Immunity against Measles Viruses
In this study, the herd immunity thresholds in terms of critical vaccination coverage (Vc) and the critical prevalence of positive anti-measles serologic results (pc) were determined for different measles viruses and different values of measles vaccination effectiveness. Herd immunity against measles is defined as the indirect protection of susceptible individuals brought about by the presence of individuals with measles immunity in the population. The generation of measles epidemics depends on the average number of individuals directly infected (secondary cases) by one infectious case during the entire infectious period, when the infectious agent has entered a totally susceptible population [23]. This number is called the basic reproductive number (Ro), and epidemics occur when Ro is higher than 1. Anderson and May found values of Ro for measles viruses ranging from 12 to 18 in a review carried out in 1991 [23], but a recent review of studies assessing the Ro values found Ro values ranging from 6 to 45 in Europe, and from 6 to 60 in different countries of the world [24].
The chain of infection is blocked for infectious diseases transmitted person-to-person when the prevalence of protected individuals (I) is higher than a disease-specific critical proportion or prevalence of protected individuals (I > Ic), defined as the herd immunity threshold [23]. The critical prevalence of protected individuals (Ic) required to establish herd immunity against measles in a completely susceptible population can be determined from: Ic = 1 − (1/Ro). This method is based on the following assumptions: (1) a homogeneous mixing of persons within the population, and (2) a homogeneous distribution of protected individuals within the population [23,25].
Measles vaccination programs can reduce measles transmission by means of generating a prevalence of vaccine-induced protected individuals (Iv) because they reduce the prevalence of susceptible individuals in the population. A lower prevalence of susceptible individuals reduces the basic reproductive number from Ro to the effective basic reproductive number (R) depending on the prevalence of vaccine-induced protected individuals generated by the vaccination program: R = Ro (1 − Iv). When R is lower than 1, measles transmission is blocked in the community and measles epidemics are prevented. In this situation, imported infections could not reestablish endemic transmission in countries without the circulation of measles viruses. Measles vaccination programs establish herd immunity in the target measles vaccination population when the prevalence of vaccine-induced protected persons is higher than the critical prevalence associated with herd immunity (Iv > Ic) [22,25].
The critical measles vaccination coverage associated with herd immunity (Vc) in the target measles vaccination population was determined from the critical prevalence of protected individuals associated with herd immunity (Ic) and the effectiveness of the measles vaccination (E): Vc = Ic/E. The critical vaccination coverage associated with herd immunity generates a prevalence of vaccine-induced individuals equal to the critical prevalence of protected individuals (Iv = Ic) for a given measles vaccination effectiveness. The prevalence of vaccine-induced protected individuals in the target measles vaccination population can be determined using the formula: Iv = (V1× E1)+ (V2× E2). In this formula, V1 and V2 are the percentages of vaccination coverage with one and two doses of the measles vaccine respectively, and E1 and E2 are the effectiveness in preventing measles cases with one and two doses of the measles vaccine, respectively. In this study, the critical measles vaccination coverage associated with herd immunity was determined for measles viruses with values of Ro ranging from 6 to 60, and for values of measles vaccination effectiveness ranging from 87.5% to 100%. The range of values considered for measles vaccination effectiveness can cover the values obtained for one and two doses of the measles vaccine [26,27,28].
Uzicanin and Zimmerman [26] reviewed the results of studies assessing measles vaccination effectiveness published during 1960−2010, obtaining a 94% vaccination effectiveness for two doses of the measles vaccine (compared with no vaccination), with an interquartile range from 88% to 98%. Marin et al. [27] obtained the values of effectiveness for two doses of the measles vaccine in preventing secondary cases of measles during an outbreak of 95%, with a 95% confidence interval from 82% to 98%. The effectiveness for one dose of the measles vaccine obtained in both studies was 92% [26,27]. The overall routine vaccination coverage and the overall measles vaccination effectiveness depended on the percentage of individuals vaccinated with one and two doses of the measles vaccine. Nevertheless, in Europe, 85.3% of individuals vaccinated during 2017–2019 had received two doses of the measles vaccine, and 86% of measles vaccination effectiveness was generated by two-dose vaccination coverage [9].
Individuals vaccinated with one dose of the measles vaccine who have anti-measles serological levels lower than the protective level (positive anti-measles serological result) are not protected against measles. The lack of measles protection in vaccinated individuals can be explained by primary and secondary vaccination failures. Primary vaccination failure is associated with lack of adequate immune response in vaccinated individuals. Secondary vaccination failure is associated with waning vaccine-induced immunity. Measles vaccination with two doses of the vaccine is associated with a higher effectiveness than vaccination with one dose of the vaccine because it reduces the primary and secondary vaccination failures associated with the first dose of the measles vaccine [19].
The objective of measles vaccination programs can be to achieve a specific effective basic reproductive number (R), or a maximum permissible value of R. The R value must be lower than 1 for achieving measles elimination, but the selection of a specific R value is a policy decision that depends on several factors, including the measles morbidity tolerated, the degree of secondary spread from infected cases tolerated, and the resources available [14]. The critical prevalence of protected individuals associated with a specific effective basic reproductive number, R, can be determined using the formula: Ic’ = 1 − (R/Ro). The critical vaccination coverage required to achieve a specific R value can be determined using the formula: Vc = Ic’/E. In this study, the critical vaccination coverage (Vc) required to achieve effective basic reproductive numbers of 0.7 and 0.5 was determined for measles viruses with Ro values ranging from 6 to 60, assuming 95% for two-dose measles vaccination effectiveness.
2.2. Critical Measles Seroprevalence Associated with Herd Immunity against Measles Viruses
Anti-measles immunity levels in terms of the prevalence of positive serologic results found in seroprevalence surveys carried out in representative samples of the population were associated with the establishment of herd immunity in the population when they were higher than the critical prevalence of positive measles serologic results (p > pc). In this study, the critical prevalence of positive measles serologic results associated with herd immunity was determined using the following formula: pc = Ic Se + (1 − Ic) (1 − Sp) [20,25]. In this formula, Ic is the critical prevalence of individuals protected against measles associated with herd immunity, and Se and Sp are the sensitivity and specificity of measles serological tests. Values of 97% were assumed for sensitivity and specificity of measles serological tests [20,25]. An alternative formula to estimate the critical prevalence is: pc = Ic Se/PPV [15]. In this formula, PPV is the predictive value of a positive test result. Herd immunity can be considered established when the prevalence of positive measles serologic results obtained in seroprevalence surveys is higher than the critical prevalence associated with herd immunity (p > pc) [15,20,25].
2.3. Herd Immunity Assessment of the Objectives for Measles Vaccination Coverage and Measles Immunity Levels Proposed by the WHO European Region
Herd immunity levels achieved with the recommended two-dose measles vaccination coverage of ≥95% were assessed by comparing the prevalence of vaccine-induced measles-protected individuals (Iv) achieved with this vaccination coverage, with the critical prevalence of protected individuals associated with herd immunity (Ic). The prevalence of protected individuals achieved with 95% two-dose measles vaccination coverage was determined using the formula: Iv = V × E. Herd immunity against measles viruses with Ro values ranging from 6 to 60 was considered established when the prevalence of vaccine-induced protected individuals was higher than the critical prevalence (Iv > Ic).
The European Region of the WHO recommended the following objectives for the proportion of protected individuals (Iwho): ≥85% in children aged 1–4 years, ≥90% in individuals aged 5–9 years, and ≥95% in individuals aged 10–14 years, 15–19 years, and ≥20 years [11,14]. These objectives were designed in order to achieve an effective R = 0.7 against measles viruses, with Ro = 11, assuming a heterogeneous mixing model [14,17]. Based on homogeneous mixing of the population, the critical prevalence of protected individuals required to establish herd immunity against measles viruses with Ro = 11 was 90.9% (9.1% susceptible), and the critical prevalence required to achieve an effective R of 0.7 against measles viruses with Ro = 11 was 93.6% (6.4% susceptible). Therefore, the objective should be 93.6% of protected individuals or 6.4% of susceptible individuals in all age groups, assuming homogeneous mixing.
The heterogeneous model used by the WHO incorporated variable contact rates among school-aged children [14,17]. The WHO considered that, based on heterogeneous mixing, a ≥95% prevalence of protected individuals (5% susceptible) in individuals aged 10 or more years), allowed lower levels of measles protection or higher levels of susceptibility in children aged 1–4 years (≥ 85%) and children aged 5–9 years (≥90%) [14,17]. In this study, the herd immunity effects derived from achieving the objectives for measles immunity proposed by the WHO were assessed by comparing the proposed prevalence of protected individuals with the critical prevalence of protected individuals-associated herd immunity (R = 1) for measles viruses with Ro values ranging from 6 to 60. Herd immunity was considered established when the prevalence of protected individuals proposed by the WHO was higher than the critical prevalence associated with herd immunity (Iwho > Ic).
The prevalence of positive measles serologic results associated with the measles immunity objectives proposed by the WHO (pwho) were determined using the following formula: pwho = Iwho Se + (1 − Iwho) (1 − Sp). In this formula, Iwho is the measles immunity objective, and Se and Sp are the sensitivity and specificity of measles serological tests. Values of 97% were assumed for the sensitivity and specificity of the measles serological tests [20,25]. Herd immunity against measles viruses was considered established when the prevalence of positive measles results associated with the recommended measles immunity levels was higher than the critical prevalence (pwho > pc).
3. Results
3.1. Measles Vaccination Coverage Associated with Herd Immunity against Measles Viruses
Table 2 presents the herd immunity thresholds in terms of the critical prevalence of protected individuals (Ic), critical vaccination coverage (Vc), and critical prevalence of positive serologic anti-measles results (pc) for measles viruses with basic reproductive numbers (Ro) ranging from 6 to 60. Herd immunity can be considered established when the prevalence of protected individuals is higher than the critical prevalence (I > Ic), when the vaccination coverage is higher than the critical coverage (V > Vc), and when the prevalence of positive serologic anti-measles results is higher than the critical prevalence (p > pc). The critical prevalence of protected individuals (Ic) associated with herd immunity ranged from 83.3% for measles viruses with Ro = 6, to 98.3% for measles viruses with Ro = 60.
Table 2.
Herd immunity thresholds in terms of the critical prevalence of protected individuals (Ic), critical vaccination coverage (Vc), the percentage of susceptible individuals in vaccinated populations, and the critical prevalence of positive anti-measles serologic results (pc) for measles viruses with basic reproductive numbers (Ro) ranging from 6 to 60.
| Measles VirusRo | Critical Prevalence of Protected Individuals a Ic (%) | Measles Vaccination | Critical Prevalence of Positive Serologic Results d pc (%) | |
|---|---|---|---|---|
| Critical Vaccination Coverage b Vc (%) | Susceptible Individuals c (%) | |||
| 6 | 83.3 | 87.7 | 16.7 | 81.3 |
| 7 | 85.7 | 90.2 | 14.3 | 83.6 |
| 8 | 87.5 | 92.1 | 12.5 | 85.3 |
| 9 | 88.9 | 93.6 | 11.1 | 86.6 |
| 10 | 90.0 | 94.7 | 10.0 | 87.6 |
| 11 | 90.9 | 95.7 | 9.1 | 88.5 |
| 12 | 91.7 | 96.5 | 8.3 | 89.2 |
| 13 | 92.3 | 97.2 | 7.7 | 89.8 |
| 14 | 92.9 | 97.7 | 7.1 | 90.3 |
| 15 | 93.3 | 98.2 | 6.7 | 90.7 |
| 16 | 93.8 | 98.7 | 6.2 | 91.1 |
| 17 | 94.1 | 99.1 | 5.9 | 91.5 |
| 18 | 94.4 | 99.4 | 5.6 | 91.8 |
| 19 | 94.7 | 99.7 | 5.3 | 92.1 |
| 20 | 95.0 | 100 | 5 | 92.3 |
| 21 | 95.2 | − | 5 | 92.5 |
| 22 | 95.5 | − | 5 | 92.7 |
| 23 | 95.7 | − | 5 | 92.9 |
| 24 | 95.8 | − | 5 | 93.1 |
| 25 | 96.0 | − | 5 | 93.2 |
| 30 | 96.7 | − | 5 | 93.9 |
| 40 | 97.5 | − | 5 | 94.7 |
| 50 | 98.0 | − | 5 | 95.1 |
| 60 | 98.3 | − | 5 | 95.4 |
a Ic = 1 − (1/Ro), b Vc = Ic/E. Assuming 95% effectiveness (E) of 95% for measles vaccination. c Percentage of susceptible individuals = 100 − (Vc E), d pc = Ic Se + [(1 − Ic) (1 − Sp)]. Assuming 97% sensitivity (Se) and 97% specificity (Sp) for measles serological tests.
Table 2 shows that the percentages of two-dose measles vaccination coverage from 88% to 100% could establish herd immunity against measles viruses with a Ro ranging from 6 to 19, assuming 95% measles vaccination effectiveness. Measles vaccination programs can establish herd immunity against measles viruses with Ro = 6 by achieving a vaccination coverage higher than the critical coverage of 87.7%, and against measles viruses with Ro = 19 by achieving a vaccination coverage higher than the critical coverage of 99.7%. Nevertheless, measles vaccination programs could not establish herd immunity against measles viruses with Ro ≥ 20, assuming 95% vaccination effectiveness, because the 95% prevalence of vaccine-induced protected individuals associated with 100% vaccination coverage is lower than the critical prevalence of protected individuals required to establish herd immunity.
The critical measles vaccination coverage (Vc) associated with an effective basic reproductive number, R, of 0.7 ranged from 93% for measles viruses with Ro = 6 to 100% for measles viruses with Ro = 14 (Table 3). Vaccination programs could not establish herd immunity against measles viruses with Ro > 14, assuming 95% vaccination effectiveness, because the prevalence of vaccine-induced protected individuals achieved with 100% coverage (95%) is lower than the prevalence of protected individuals required to establish herd immunity against these viruses. The critical vaccination coverage associated with an effective basic reproductive number, R, of 0.5 ranged from 96.5% for measles viruses with Ro = 6 to 100% for measles viruses with Ro = 10 (Table 3). Vaccination programs could not establish herd immunity against measles viruses with Ro > 10, assuming 95% vaccination effectiveness, because the prevalence of vaccine-induced protected individuals achieved with 100% coverage (95%) is lower than the prevalence of protected individuals required to establish herd immunity against these viruses.
Table 3.
The critical prevalence of protected individuals (Ic), critical vaccination coverage (Vc), and critical prevalence of positive anti-measles serologic results (pc) associated with an effective basic reproductive number, R, of 0.7 and 0.5, for measles viruses with basic reproductive numbers (Ro) ranging from 6 to 60.
| Measles Virus Ro | Herd Immunity Thresholds Ic, Vc, and pc Associated with an Effective Basic Reproductive Number, R a, of 0.7 and 0.5 | |||||
|---|---|---|---|---|---|---|
| R = 0.7 | R = 0.5 | |||||
| Ic (%) b | Vc (%) c | pc (%) d | Ic (%) b | Vc (%) c | pc (%) d | |
| 6 | 88.3 | 93.0 | 86.0 | 91.7 | 96.5 | 89.2 |
| 7 | 90.0 | 94.7 | 87.6 | 92.9 | 97.7 | 90.3 |
| 8 | 91.3 | 96.1 | 88.8 | 93.8 | 98.7 | 91.1 |
| 9 | 92.2 | 97.1 | 89.7 | 94.4 | 99.4 | 91.8 |
| 10 | 93.0 | 97.9 | 90.4 | 95.0 | 100 | 92.3 |
| 11 | 93.6 | 98.6 | 91.0 | 95.5 | − | 92.7 |
| 12 | 94.2 | 99.1 | 91.5 | 95.8 | − | 93.1 |
| 13 | 94.6 | 99.6 | 91.9 | 96.2 | − | 93.4 |
| 14 | 95.0 | 100 | 92.3 | 96.4 | − | 93.6 |
| 15 | 95.3 | − | 92.6 | 96.7 | − | 93.9 |
| 16 | 95.6 | − | 92.9 | 96.9 | − | 94.1 |
| 17 | 95.9 | − | 93.1 | 97.1 | − | 94.2 |
| 18 | 96.1 | − | 93.3 | 97.2 | − | 94.4 |
| 19 | 96.3 | − | 93.5 | 97.4 | − | 94.5 |
| 20 | 96.5 | − | 93.7 | 97.5 | − | 94.7 |
| 21 | 96.7 | − | 93.9 | 97.6 | − | 94.8 |
| 22 | 96.8 | − | 94.0 | 97.7 | − | 94.9 |
| 23 | 97.0 | − | 94.1 | 97.8 | − | 95.0 |
| 24 | 97.1 | − | 94.3 | 97.9 | − | 95.0 |
| 25 | 97.2 | − | 94.4 | 98.0 | − | 95.1 |
| 30 | 97.7 | − | 94.8 | 98.3 | − | 95.4 |
| 40 | 98.3 | − | 95.4 | 98.8 | − | 95.8 |
| 50 | 98.6 | − | 95.7 | 99.0 | − | 96.1 |
| 60 | 98.8 | − | 95.9 | 99.2 | − | 96.2 |
| Mean | 95.3 | 99.0 | 92.5 | 96.6 | 99.7 | 93.8 |
a The effective basic reproductive number (R) is the reduced Ro number due to vaccine-induced protected individuals (Iv). R = Ro (1 − Iv), b Ic = 1 − (R/Ro), c Vc = Ic/E. Assuming 95% effectiveness (E) for measles vaccination, d pc = Ic Se + [(1 − Ic) (1 − Sp)]. Assuming 97% sensitivity (Se) and specificity (Sp).
Figure 1 presents the critical vaccination coverage associated with herd immunity against measles viruses with Ro values ranging from 6 to 60, for values of measles vaccination effectiveness ranging from 87.5% to 100%. Figure 1 can be used to estimate the vaccination coverage required to establish herd immunity for different combinations of Ro values for measles viruses and measles vaccination effectiveness. When the effectiveness of the measles vaccination was 90%, herd immunity could be established against measles viruses with Ro < 9, but the vaccination coverage required to establish herd immunity was >95% for measles viruses with Ro values from 7 to 9. When the vaccination effectiveness was 95%, herd immunity could be established against measles viruses with Ro ≤ 20, but the vaccination coverage required to establish herd immunity was >95% for measles viruses with Ro from 10 to 20.
Figure 1.
The vaccination coverage (%) required to establish herd immunity for measles viruses with reproductive numbers (Ro) from 6 to 60 and the effectiveness of measles vaccination from 87.7% to 100%. The objectives of measles vaccination coverage with two doses of vaccine of 95% (proposed by the WHO European Region at the national level), and 90% are indicated by horizontal dashed lines.
With a 95% vaccination effectiveness, herd immunity could not be established against measles viruses with high Ro ≥ 21 because in this situation, the prevalence of vaccine-induced measles protection with 100% vaccination coverage is lower than the critical prevalence necessary to establish herd immunity (Iv < Ic). When the vaccination effectiveness was 97%, herd immunity could be established against all measles viruses ranging from 6 to 33, but the vaccination coverage required to establish herd immunity was >95% for measles viruses with Ro from 13 to 33. With a 97% vaccination effectiveness, herd immunity could not be established against measles viruses with high Ro ≥ 34 because in this situation, the prevalence of vaccine-induced measles protection with 100% vaccination coverage is lower than the critical prevalence necessary to establish herd immunity (Iv < Ic).
3.2. Measles Seroprevalence Associated with Herd Immunity against Measles Viruses
The critical prevalence of positive serologic results (pc) that should be obtained in a serological study when the effective basic reproductive number, R, is 1, ranged from 81.3% for measles viruses with Ro = 6 to 95.4% for measles viruses with Ro = 60 (Table 2). The prevalence of positive serologic results (pc) required to establish herd immunity must be higher (at least 0.1% more) than the values obtained for R = 1 because herd immunity can be considered to be established when R is lower than 1. The critical prevalence of positive serologic results that should be obtained when the effective reproductive number is 0.7 ranged from 86% for measles viruses with Ro = 6 to 95.9% for measles viruses with Ro = 60 (Table 3). The critical prevalence of positive serologic results that should be obtained when the effective reproductive number is 0.5 ranged from 89.2% for measles viruses with Ro = 6 to 96.2% for measles viruses with Ro = 60 (Table 3). For measles viruses with Ro = 18, herd immunity can be considered established when the vaccination coverage is 99.5% and the prevalence of positive serologic results is 91.9%.
3.3. Herd Immunity Assessment of the Objectives for Measles Vaccination Coverage and Measles Immunity Levels Proposed by the WHO European Region
Figure 1 shows that the objective of 95% for two-dose measles vaccination coverage proposed by the WHO was sufficient to establish herd immunity against measles viruses with Ro values from 6 to 9, but it was not sufficient to establish herd immunity against measles viruses with Ro ≥ 10, assuming a 95% vaccination effectiveness. When the measles vaccination effectiveness increased to 97%, a 95% measles vaccination coverage was sufficient to establish herd immunity against measles viruses with Ro values from 6 to 13. When the measles vaccination effectiveness decreased to 93%, a 95% measles vaccination coverage was sufficient to establish herd immunity only against measles viruses with Ro values from 6 to 8.
Figure 1 shows that changing the objective for measles vaccination coverage from 95% to 97% could establish herd immunity against measles viruses with Ro values from 6 to 13, assuming 95% measles vaccination effectiveness. When the measles vaccination effectiveness increased to 97%, 97% measles vaccination coverage was shown to be sufficient to establish herd immunity against measles with Ro values from 6 to 18. When the measles vaccination effectiveness decreased to 93%, 97% measles vaccination coverage was sufficient to established herd immunity only against measles virus with Ro values from 6 to 10.
Table 2 shows that the objectives for measles immunity proposed by the WHO in individuals aged 1–4 years (≥85%) and 5–9 years (≥90%) were not sufficient to block the transmission of most measles viruses. By contrast, the objectives proposed in individuals aged 10–14 years, 15–19 years, and ≥20 years (≥95%) were sufficient to block the transmission of measles viruses with Ro values ranging from 6 to 20. In addition, the minimum measles immunity levels proposed in individuals aged 1–4 years (85%) and 5–9 years (90%) were not consistent with the objective for routine two-dose measles vaccination coverage of at least 95%. A 95% vaccination coverage for routine two-dose measles vaccination, given at 12–15 months and 3–15 years, must generate a prevalence of vaccine-induced measles protection of 90.2% in individuals aged 1–4 years and 5–9 years, assuming a measles vaccination effectiveness of 95%. Consequently, I suggest an increase in the objectives for measles immunity in individuals aged 1–4 years and 5–9 years to at least 95% (Table 1).
4. Discussion
The main objectives of measles vaccination programs developed by public health departments are to protect vaccinated individuals against measles and to achieve sufficient vaccination coverage to establish the necessary herd immunity to prevent measles transmission in the population. This study found that success in preventing measles transmission and achieving measles elimination depended on the basic reproductive number (Ro) of the measles viruses and measles vaccination effectiveness. The WHO proposed to achieve and maintain percentages of measles vaccination coverage with two doses of measles vaccines equal to or higher than 95%, as well as to achieve specific anti-measles immunity levels in different age groups. Nevertheless, this study found that the objectives proposed by the WHO for two-dose measles vaccination coverage and measles immunity were not sufficient to achieve measles elimination in Europe.
The study determined the minimum vaccination coverage required to establish herd immunity against measles viruses with Ro values ranging from 6 to 60. High percentages of two-dose measles vaccination coverage are necessary to establish herd immunity and block measles transmission [20,22,25]. The study found that the minimum two-dose measles vaccination coverage recommended by the WHO (95%) was not sufficient to establish herd immunity against measles viruses with Ro values equal to or higher than 10, assuming a 95% vaccination effectiveness. In addition, the recommended two-dose vaccination coverage was not sufficient to establish the herd immunity required to achieve an R value of 0.7 against measles viruses with Ro values equal to or higher than 8, and it was not sufficient to establish the herd immunity required to achieve an R value of 0.5 against measles virus with Ro values equal to or higher than 6. Consequently, I suggest an increase in the recommended two-dose vaccination coverage to ≥97% (Table 1).
The study determined the minimum proportion or prevalence of protected individuals required to establish herd immunity against measles viruses with Ro values ranging from 6 to 60. High percentages of measles immunity are necessary to establish herd immunity and block measles transmission [17,20,29]. The study found that the minimum proportion or prevalence of protected individuals recommended by the WHO in children aged 1–4 (85%) and individuals aged 5–9 years (90%) were not sufficient to establish herd immunity against most measles viruses. By contrast, the prevalence of protected individuals recommended in individuals aged ≥10 years (95%) was sufficient to establish herd immunity against measles viruses. Consequently, I suggest an increase in the recommended prevalence of measles protection to at least to 95% in individuals aged 1–4 and 5–9 years (Table 1).
The recommendations proposed in this study for measles immunity in different age groups were determined for measles viruses with Ro values from 6 to 60 assuming a homogeneous mixing of the population. The objectives proposed by the WHO in children aged 1–4 and 5–9 years were designed for measles viruses with an Ro value equal to 11 using a heterogeneous mixing model. These criteria allowed a lower prevalence of measles protection in children aged 1–4 years (≥85%) and individuals aged 5–9 years (≥90%) [17,30].
The objectives for measles immunity should be increased to at least 95% in individuals aged 1–4 and 5–9 years for two reasons. First, the WHO’s recommendations for measles immunity were determined assuming a Ro value of 11 for measles viruses, but measles viruses can be associated with values of Ro higher than 11. Second, the minimum measles immunity levels proposed in individuals aged 1–4 years (85%) and 5–9 years (90%) were lower than the prevalence of vaccine-induced measles protection of 90.2% generated by 95% two-dose measles vaccination coverage, assuming a measles vaccination effectiveness of 95%.
This study has several limitations. First, the herd immunity assessment of the objectives for two-dose measles vaccination coverage and measles immunity levels proposed by the WHO was carried out based on the following assumptions: (1) homogeneous mixing of individuals within the population, and (2) homogeneous distribution of protected individuals within the population. The approach used in this study is based on the following reasons: (1) the mixing pattern and contact rates among individuals is not known, and (2) assuming a heterogeneous mixing can reduce the recommended two-dose vaccination coverage and measles immunity in population groups without a consistent justification. The heterogeneous mixing model reduces the vaccination coverage required to establish herd immunity in age groups with low measles transmission rates (1–9 years), when compared to the coverage based on homogeneous mixing. Nevertheless, individuals aged 1–9 years are the target population for routine measles vaccination and have the highest risk for measles complications [4,9,21]. For this reason, the herd immunity assessment based on the homogeneous mixing model can be considered a conservative approach.
The second limitation is that the herd immunity assessment for different percentages of two-dose measles vaccination coverage was carried out assuming vaccination effectiveness ranging from 87.5% to 100%. The establishment of herd immunity would be more difficult for values of measles vaccination effectiveness lower than 87.5%; however, it is possible to assume that 87.5% is the lower effectiveness for two-dose measles vaccination programs [26,27,28].
The results obtained in this study indicated that it is necessary to increase the recommended objectives for measles vaccination coverage with two-dose coverage in order to achieve measles elimination from Europe from ≥95% to ≥97%. The current objectives for measles vaccination coverage recommended by the WHO for routine measles vaccination could be sufficient for achieving measles elimination, however, with intensive supplementary measles vaccination activities based on mass catch-up and follow-up vaccination campaigns. Supplementary mass catch-up vaccinations should be given to children, adolescents, and adults who have not received the first or second dose of measles vaccine or who have lost their vaccination records [4]. Catch-up vaccination activities are usually implemented for a limited time. The measles prevention strategy based on high routine measles vaccination of children and intensive supplementary vaccination activities was implemented by the Pan American Health Organization (PAHO) in 1980–1990, and it succeeded in interrupting the transmission of measles in America [4,31]. The objective of the PAHO was to achieve at least 95% coverage with the MMR vaccine for children aged 12 months and to implement intensive supplementary vaccination activities to ensure the establishment of herd immunity in all countries.
The intensive supplementary vaccination activities consisted of one mass campaign among adolescents and adults, catch-up campaigns in children aged 1–14 years, and follow-up campaigns in children aged < 5 years. In Europe, the objective for measles vaccination coverage with two-dose coverage should be increased from ≥95% to ≥97% as intensive supplementary vaccination activities or screening and vaccination programs are not developed. In addition, measles prevention based on routine and supplementary vaccination activities requires continuous intensive catch-up vaccination activities to maintain high percentages of vaccination coverage [32].
An alternative strategy to increase measles immunity in the population can be based on the administration of a third dose of the measles vaccine to young adults (15–35 years). Two vaccination programs could be developed: (1) vaccination of susceptible young adults detected by pre-vaccination screening (catch-up of susceptible), and (2) vaccination of all young adults, regardless of their vaccination and immunity status (catch-up or routine vaccination) [20]. Nevertheless, increasing routine two-dose measles vaccination to ≥97% has a higher priority than vaccinating young adults with a third dose of measles vaccine for two reasons. First, vaccinating young adults with a third dose of measles vaccine will not increase measles protection in the target vaccination population (1–9 years). Second, the strategy based on a third dose of measles vaccine in young adults is more complex and costly than increasing two-dose measles vaccination coverage to ≥97%.
Routine measles vaccination coverage with two doses of the vaccine could be increased to 97% by implementing interventions to enhance access to vaccination services, interventions to increase demand for vaccines, interventions to increase provider vaccination activities, and interventions to increase vaccine confidence [33,34,35]. Specific regulations on vaccination requirements can increase measles vaccination coverage, but this strategy should add to, more than replace, other strategies to reach and maintain high percentages of measles vaccination coverage [36]. In the European Union, measles vaccination (the MMR vaccine) was mandatory in 28% of the countries in 2010 [37]. Advanced vaccination programs, based on nominal vaccination registries and health information technology, can implement interventions to increase measles vaccination coverage in a more efficient way than less advanced vaccination programs [38,39].
5. Conclusions
This study found that the objectives for two-dose measles vaccination coverage and measles immunity levels in children recommend by the WHO European Region were not sufficient to establish herd immunity against measles viruses with a Ro equal to or higher than 10. To meet the goal of measles elimination in Europe, it is necessary to achieve percentages of two-dose measles vaccination coverage of at least 97%, and measles immunity levels in children aged 1–9 years of at least 95%.
Funding
The work did not receive any funding.
Conflicts of Interest
The author has no financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter discussed in the manuscript.
References
- 1.World Health Organization (WHO) Global Measles and Rubella Strategic Plan: 2012–2020. WHO; Geneva, Switzerland: 2012. [Google Scholar]
- 2.Plans-Rubió P. Why does measles persist in Europe. Eur. J. Clin. Microbiol. Infect. Dis. 2017;36:1899–1906. doi: 10.1007/s10096-017-3011-y. [DOI] [PubMed] [Google Scholar]
- 3.Moss W.J., Strebel P. Biological feasibility of measles eradication. J. Infect. Dis. 2011;204(Suppl.1):S47–S53. doi: 10.1093/infdis/jir065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.World Health Organization (WHO) Strategic Plan. for Measles and Congenital Rubella Infection in the WHO European Region. WHO Regional Office for Europe; Copenhagen, Denmark: 2003. [Google Scholar]
- 5.WHO Centralized Information System for nfectious Diseases (CISID) [(accessed on 7 April 2020)]; Measles-Number Ofcases, 2015–2019. Available online: http://data.euro.who.int/cisid/?TabID=523870.
- 6.WHO Centralized Information System for Infectious Diseases (CISID) [(accessed on 7 April 2020)]; Measles-Number of hospitalizations, 2015–2019. Available online: http://data.euro.who.int/cisid/?TabID=523869.
- 7.WHO Centralized Information System for Infectious Diseases (CISID) [(accessed on 7 April 2020)]; Measles-Number of deaths, 2015–2019. Available online: http://data.euro.who.int/cisid/?TabID=523871.
- 8.European Centre for Disease Prevention and Control (ECDC) Monthly Measles and Rubella Monitoring, February 2018. ECDC; Stockholm, Sweden: 2018. [(accessed on 10 April 2020)]. Available online: https://www.ecdc.europa.eu/sites/default/files/documents/measles-rubella-monthly-monitoring-report-february-2019.pdf. [Google Scholar]
- 9.European Centre for Disease Prevention and Control (ECDC) Monthly Measles and Rubella Monitoring Report, February 2019. ECDC; Stockholm, Sweden: 2019. [(accessed on 10 April 2020)]. Available online: https://www.ecdc.europa.eu/sites/default/files/documents/measles-rubella-monthly-report-february-202.pdf. [Google Scholar]
- 10.Moss W.J., Griffin D.E. Global measles elimination. Nature. 2006:4. doi: 10.1038/nrmicro1550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Steffens I., Martin R., Lopalco P.L. Spotlight on measles 2010: Measles elimination in Europe—A new commitment to meet the goal by 2015. [(accessed on 15 January 2020)];Euro Surveill. 2010 15:19749. Available online: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19749. [PubMed] [Google Scholar]
- 12.Plans-Rubió P. Low percentages of measles vaccination coverage with two doses of vaccine and low herd immunity levels explain measles incidence and persistence of measles in the European Union in 2017–2018. Eur. J. Clin. Microbiol. Infect. Dis. 2019;8:1719–1729. doi: 10.1007/s10096-019-03604-0. [DOI] [PubMed] [Google Scholar]
- 13.World Health Organization Regional Office for Europe . Eliminating Measles and Rubella: Framework for the Verification Process in the WHO European Region. WHO Regional Office for Europe; Copenhagen, Denmark: 2014. [(accessed on 21 January 2020)]. Available online: http://www.euro.who.int/__data/assets/pdf_file/0009/247356/Eliminating-measles-and-rubella-Framewok-for-the-verification-process-in-the-WHO-European-Region.pdf. [Google Scholar]
- 14.WHO . A Strategic Framework for the Elimination of Measles in the European Region. WHO; Copenhagen, Denmark: 1999. [Google Scholar]
- 15.Plans P. Prevalence of antibodies associated with herd immunity: A new indicator to evaluate the establishment of herd immunity and to decide immunization strategies. Med. Dec. Mak. 2010;30:438–443. doi: 10.1177/0272989X09353453. [DOI] [PubMed] [Google Scholar]
- 16.Hazlina Y., Marlindawati M.A., Shamsuddin K. Serological assessment of the establishment of herd immunity against measles in a health district in Malaysia. BMC Inf. Dis. 2016;16:740. doi: 10.1186/s12879-016-2069-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Gay N.J. The theory of measles elimination: Implications for the design of elimination strategies. J. Infect. Dis. 2004;189(Suppl.1):S27–S35. doi: 10.1086/381592. [DOI] [PubMed] [Google Scholar]
- 18.European Centre for Disease Prevention and Control (ECDC) Measles: Recommended Vaccination. [(accessed on 21 January 2020)]; Available online: https://vaccine-schedule.ecdc.europa.eu/Scheduler/ByDisease?SelectedDiseaseId=8&SelectedCountryIdByDisease=-1.
- 19.Rosenthal S. R., Clements C. J. Two-dose measles vaccination schedule. Bull. Who. 1993;71:421–428. [PMC free article] [PubMed] [Google Scholar]
- 20.Plans P. New preventive strategy to eliminate measles, mumps and rubella from Europe based on the serological assessment of herd immunity levels in the population. Eur. J. Clin. Microbiol. Infect. Dis. 2013;32:961–996. doi: 10.1007/s10096-013-1836-6. [DOI] [PubMed] [Google Scholar]
- 21.Muscat M., Bang H., Wohlfahrt J., Glismann S., Mølbak K. EUVAC.NET Group. Measles in Europe: An epidemiological assessment. Lancet. 2009;373:383–389. doi: 10.1016/S0140-6736(08)61849-8. [DOI] [PubMed] [Google Scholar]
- 22.Anderson R.M. The concept of herd immunity and the design of community-based immunization programmes. Vaccine. 1992;10:928–935. doi: 10.1016/0264-410X(92)90327-G. [DOI] [PubMed] [Google Scholar]
- 23.Anderson R. M., May R.M. Infectious Diseases of Humans: Dynamics and Control. University Press; Oxford, UK: 1991. [Google Scholar]
- 24.Guerra F., Bolotin S., Lim G., Heffernan J., Deeks S.L., Li Y., Crowcroft N.S. The basic reproduction number (Ro) of measles: A systematic review. Lancet Infect. Dis. 2017:e420–e428. doi: 10.1016/S1473-3099(17)30307-9. [DOI] [PubMed] [Google Scholar]
- 25.Plans-Rubió P. Evaluation of the establishment of herd immunity in the population by means of serological survey sand vaccination coverage. [(accessed on 1 March 2020)];Hum. Vaccin. Immunother. 2012 8:184–188. doi: 10.4161/hv.18444. Available online: http://www.tandfonline.com/doi/pdf/10.4161/hv.18444. [DOI] [PubMed] [Google Scholar]
- 26.Uzicanin A., Zimmerman L. Field effectiveness of live attenuated measles-containing vaccines: A review of published literature. J. Infect. Dis. 2011;204(Suppl.1):S135–S147. doi: 10.1093/infdis/jir102. [DOI] [PubMed] [Google Scholar]
- 27.Marin M., Nguyen H.Q., Langidrik J.R., Edwards R., Briand K., Papania M.J., Seward J.F., LeBaron C.W. Measles transmission and vaccine effectiveness during a large outbreak on a densely populated island: Implications for vaccination policy. Clin. Infect. Dis. 2006;42:315–319. doi: 10.1086/498902. [DOI] [PubMed] [Google Scholar]
- 28.Demicheli V., Rivetti A., Debalini M.G., Di Pietrantonj C. Vaccines for measles, mumps and rubella in children. Cochrane Database Syst. Rev. 2012:CD004407. doi: 10.1002/14651858.CD004407.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Plans P., Torner N., Godoy P., Jané M. Lack of herd immunity against measles in individuals aged <35 years could explain re-emergence of measles in Catalonia (Spain) [(accessed on 14 January 2020)];Int. J. Infect. Dis. 2014 18:81–83. doi: 10.1016/j.ijid.2013.09.015. Available online: http://download.journals.elsevierhealth.com/pdfs/journals/12019712/PIIS120197121300307X.pdf. [DOI] [PubMed] [Google Scholar]
- 30.Gay N.J., Hesketh L.M., Morgan-Capner P., Miller E. Interpretation of serological surveillance data for measles using mathematica lmodels: Implications for vaccine strategy. Epidemiol. Infect. 1995;115:139–156. doi: 10.1017/S0950268800058209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.De Quadros C.A., Hersh B.S., Nogueira A.C., Carrasco P.A., daSilveira C.M. Measles eradication: Experience in theAmericas. Bull. Who. 1998;76(Suppl. 2):47–52. [PMC free article] [PubMed] [Google Scholar]
- 32.Mokdad A.H., Gagnier M.C., Colson K.E., Dansereau E., Zuniga-Brenes P., Rios-Zertuche D., Haakenstad A., Johanns C.K., Palmisano E.B., Hernandez B., et al. Missed opportunities for measles, mumps, and rubella (MMR) immunization in Mesoamerica: Potential impact on coverage and days at risk. PLoS ONE. 2015;10:e0139680. doi: 10.1371/journal.pone.0139680. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Wilder-Smith A.B., Qureshi K. Resurgence of measles in Europe: A systematic review on parental attitudes and beliefs of measles vaccine. J. Epidemiol. Glob. Health. 2020;10:46–58. doi: 10.2991/jegh.k.191117.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Frew P.M., Lutz C.S. Interventions to increase pediatric vaccine uptake: An overview of recent findings. Hum. Vaccin. Immunother. 2017;13:2503–2511. doi: 10.1080/21645515.2017.1367069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Mac Donald N., Butler R., Dubé E. Addressing barriers to vaccine acceptance: An overview. Hum. Vaccin. Immunother. 2018;14:218–224. doi: 10.1080/21645515.2017.1394533. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Salmon D.A., Teret S.P., Macintyre C.R., Salisbury D., Burgess M.A., Halsey N.A. Compulsory vaccination and conscientious or philosophical exemptions: Past, present, and future. Lancet. 2006;367:436–442. doi: 10.1016/S0140-6736(06)68144-0. [DOI] [PubMed] [Google Scholar]
- 37.Haverkate M., D’Ancona F., Giambi C., Lopalco P.L., Cozza V., Appelgren E. Mandatory and recommended vaccination in the EU, Iceland and Norway: Results of the VENICE 2010 survey on the ways of implementing national vaccination programmes. Euro Surveill. [(accessed on 2 March 2020)];Euro Surveill. 2010 17:20183. doi: 10.2807/ese.17.22.20183-en. Available online: http://www.eurosurveillance.org/ViewArticle.aspx?Articleld=20183. [DOI] [PubMed] [Google Scholar]
- 38.Szilagyi P.G., Schaffer S., Shone L., Barth R., Humiston S.G., Sandler M., Rodewald L.E. Reducing geographical, racial, and ethnic disparities in childhood immunization rates by using reminder/recall interventions in urban primary care practices. Pediatrics. 2002;110:e58. doi: 10.1542/peds.110.5.e58. [DOI] [PubMed] [Google Scholar]
- 39.Stockwell M.S., Fiks A.G. Utilizing health information technology to improve vaccine communication and coverage. Hum. Vaccin. Immunother. 2013;9:1802–1811. doi: 10.4161/hv.25031. [DOI] [PMC free article] [PubMed] [Google Scholar]