Vaccine preventable disease incidence as a complement to vaccine efficacy for setting vaccine policy

Abstract

Traditionally, vaccines have been evaluated in clinical trials that establish vaccine efficacy (VE) against etiology-confirmed disease outcomes, a measure important for licensure. Yet, VE does not reflect a vaccine’s public health impact because it does not account for relative disease incidence. An additional measure that more directly establishes a vaccine’s public health value is the vaccine preventable disease incidence (VPDI), which is the incidence of disease preventable by vaccine in a given context. We describe how VE and VPDI can vary, sometimes in inverse directions, across disease outcomes and vaccinated populations. We provide examples of how VPDI can be used to reveal the relative public health impact of vaccines in developing countries, which can be masked by focus on VE alone. We recommend that VPDI be incorporated along with VE into the analytic plans of vaccine trials, as well as decisions by funders, ministries of health, and regulatory authorities.

1. Background

Traditionally, vaccines have been evaluated in clinical trials that establish vaccine efficacy (VE) against specific etiology-confirmed disease outcomes. In this sense, VE is a characteristic of the vaccine and thus an important measure for licensure. Yet, VE may not reflect the public health impact of a vaccine because it does not account for the background incidence of the disease in the absence of vaccine. A complementary measure that more directly establishes a vaccine’s public health impact is the vaccine preventable disease incidence (VPDI) (also known by the equivalent term vaccine attributable rate reduction), a measure recently described [1–3].

VPDI measures the difference in the incidence of any particular outcome between an unvaccinated and a vaccinated population. VPDI does not depend on knowing the incidence of etiologically confirmed disease, and originally was used for clinical syndromes where etiology was difficult to determine or where laboratory confirmation might substantially underestimate true disease burden. Examples include the VPDI of non-bacteremic pneumonia following Hib [4] or pneumococcal conjugate vaccine (PCV) [5], all cause diarrhea following rotavirus vaccine [6–8], and outpatient infant respiratory illness following maternal influenza vaccine [9].

When calculating VPDI, it may be the case that high VPDI occurs despite a relatively low VE. This relationship is seen both within an individual vaccine, when looking at different outcomes or populations, and when comparing several vaccines with each other. In this paper, we set out to describe this relationship between VE and VPDI, and show how VPDI can complement VE as a basis for assessing the public health utility of a vaccine and making vaccine-related policy decisions, particularly in countries with a high burden of morbidity and mortality.

2. Definitions and limitations

Mathematically, VE = 1 − (I_v/I_c) where I_v and I_c = incidences of an outcome in the vaccinated and control groups, respectively. This is equivalent to (I_c − I_v)/I_c, which can be recognized as the formulation for an epidemiologic concept that has been alternatively called rate fraction, etiologic fraction, and attributable fraction [10,11]. Most often this expression describes the fraction of cases in which a risk factor contributes as a cause of disease; in the case of a vaccine, the expression describes the fraction of cases prevented by the vaccine. Some authors have distinguished the excess fraction from the etiologic fraction, the former being the fraction of cases (as an all or none outcome) caused by an exposure among all exposed persons with the outcome while the latter is the fraction of cases caused or accelerated by an exposure [10,11]. Likewise, vaccines might prevent or delay etiology-specific disease; however, most vaccine trials have follow-up periods too short to distinguish the two and thus VE can be considered to encompass prevented and – if it occurs within the timeframe of the trial – delayed disease.

VPDI, in contrast to VE, is not a fraction, but an incidence. Using a randomized clinical trial design, VPDI is calculated as I_c − I_v, which equates to the numerator in the VE formula. A mathematically equivalent formulation is I_c × ((I_c − I_v)/I_c), which reduces to I_c × VE. This latter formulation emphasizes that VPDI encompasses both VE and the background incidence of the disease syndrome in question. For the incidences used to calculate VPDI the numerator population is part of the denominator, since vaccine clinical trials begin enrollment at the receipt of first vaccination (whether intervention, control, or placebo vaccine) and assess outcomes only among the vaccinated [10]. As is apparent, VPDI is an incidence difference, which has also been called a rate reduction [10]. The latter term has some appeal since the concepts presented here can be applied to clinical trials of non-vaccine interventions [12]. However, as a tool for advocacy and policy within the field of vaccinology, we support the use of the specific term VPDI, just as VE is used in vaccinology for the broader term etiologic fraction.

VPDI provides an overall assessment of a vaccine’s public health value in a population during the period of evaluation. As such, the application of VPDI has some limitations. It cannot address the degree to which competing risks exist, for example, if a decrease in one organism leads to an increase in disease from another. VPDI provides information only for the measured disease outcome while vaccine may prevent unexpected and unmeasured outcomes that influence the vaccine’s overall public health value. Similar to VE, VPDI cannot address changes in vaccine impact outside the period of observation, for example, if a vaccine-induced decrease in exposure and natural immunity during the study period leads to increased disease risk after the period of study follow-up. Similarly, within the period of study follow-up, VPDI cannot distinguish prevention of disease from a delay in occurrence [11]; in principle, this could be addressed through ever-finer age stratification but in practicality study power may limit this approach. Lastly, VPDI conflates individual and population effects, i.e., direct and herd protection. Consequently, in an individually randomized trial, substantial indirect effects may reduce observed VPDI to zero (and make VE undefined) despite substantial vaccine-induced disease reduction. This is a strong argument for conducting cluster-randomized trials of vaccines with clusters large enough to maintain infection risk. Within a cluster-randomized trial, VPDI will include reduction in disease incidence resulting from direct protection of vaccinees who had an adequate immune response plus indirect protection among vaccinees who did not respond to vaccine.

3. VPDI against different outcomes with the same vaccine

Most vaccine licensure in the past has depended on a vaccine achieving a high VE against the most specific disease outcome, namely – etiologically confirmed disease. Examples are Hib vaccine against Hib meningitis and pneumococcal conjugate vaccine (PCV) against vaccine-type invasive pneumococcal disease. Within etiologically confirmed disease outcomes, regulators have focused on those outcomes for which high VEs are found; for example for rotavirus vaccines, the outcomes of focus were rotavirus-specific severe disease, hospitalization, and death rather than all confirmed disease. Yet, even when limited to severe outcomes, most vaccines prevent additional episodes of severe disease that is not etiologically confirmed.

This occurs because some pathogens, and possibly most pathogens, cause clinical disease not accounted for by traditional, accepted diagnostic tools used at the point of contact with the health care system. For example, Hib vaccine and PCV prevent a substantial amount of pneumonia, but most of the preventable pneumonias are non-bacteremic; thus etiologic confirmation would require evaluations such as lung puncture or trans-tracheal aspiration which may be considered unethical or impractical in the context of a clinical trial. Influenza viruses may constitute part of the causal pathway of some bacterial pneumonias, yet no longer be present, and thus undetectable, by the time a patient presents to a health care facility with pneumonia. Cholera outbreaks occur quickly and can affect thousands of people in an area, overwhelming health care systems and preventing etiologic confirmation for the majority of cases. Antibiotic administration before collection of clinical specimens may alter diagnostic test results for many bacterial diseases. In each of these cases, a vaccine probe study [1] and calculation of VPDI may provide a better assessment of vaccine public health utility than VE against etiologically confirmed disease alone.

VPDI assesses the amount of disease prevented when considering both VE and baseline disease incidence. For Hib, pneumococcal, and rotavirus vaccines, the outcome that had the lowest statistically significant VE had a higher VPDI than the outcome with the highest VE in ratios of 99, 8.9 and 5.8, respectively (Table 1) [4,5,7]. The reason for this finding is that nonspecific disease syndromes, such as pneumonia and severe gastroenteritis, simultaneously include non-confirmed, vaccine-target etiologies (leading to higher sensitivity for all vaccine preventable disease and thus higher VPDI), as well as non-vaccine-target etiologies (leading to lower specificity for all vaccine preventable disease and thus lower VE). Syndromic disease may or may not be equivalent to etiologically confirmed disease with regard to severity. As an example, we are not aware of evidence that bacteremic compared to non-bacteremic Hib or pneumococcal pneumonia has a worse outcome given the same clinical presentation, since hypoxia, rather than bacteremia, is the primary risk factor for death [13] (for studies comparing bacteremic vs. non-bacteremic pneumonia, diagnosis of the latter would require invasive procedures, which are not commonly done).

Table 1.

Relationship between vaccine efficacy (VE) and vaccine preventable disease incidence (VPDI) in selected vaccine clinical trials.

Vaccine trial	Outcome with highest VE				Outcome with lowest, statistically significant VE
	Outcome	Incidence controlsa	VE	VPDIa	Outcome	Incidence controlsa	VE	VPDIa
Indonesia Hib vaccine (4)	Culture-confirmed Hib meningitis	0.19	86%	0.16	Clinical pneumonia	395	4.0%	15.8
Gambia PCV(5)	Vaccine-serotype IPDb	2.5	77%	1.9	Clinical pneumonia	249	7%	17.0
Kenya rotavirus vaccine (7)	Severe rotavirus gastroenteritis attending health facility (1st year)	40	83%	33	Gastroenteritis with severe dehydration reported at home (1st year)	550	34%	190

^aAll incidences and VPDIs are given as per 1000 person-years.

^bIPD, invasive pneumococcal disease.

4. VPDI for the same outcome and vaccine in different settings

VPDI will vary by setting as disease burden varies. Similarly, VE against clinical syndromes will depend on the burden of other etiologies for the same syndrome; for example PCV VE against clinical pneumonia will be lower during peak respiratory syncytial virus (RSV) burden years and higher during low burden years.

Less readily appreciated is that VE (or effectiveness) against etiologically confirmed disease also varies. For example, OPV VE as measured by neutralizing antibody has varied from 36% to 99% for serotype 1, with a generally lower level in developing and south Asian countries, leading to a requirement for multiple immunizations to develop protection [14]. This may result from factors external to vaccinees (such as force of infection) and thus lead to different results during vaccine efficacy vs. effectiveness trials; however, it also may result from factors intrinsic to vaccinees (such as differences in the intestinal microbiome and impaired gut mucosal immunity in poorer populations) and thus would affect results from both vaccine efficacy and effectiveness studies. Rotavirus vaccine provides another example of this phenomenon. Two multi-site trials conducted in developing countries found lower VE in the sites with poorer populations and presumably worse sanitation [6,8]. In both trials, the site with the lower VE (Malawi 50% vs. South Africa 77%; Bangladesh 43% vs. Vietnam 64%) reported a higher VPDI (per 1000 child-years: Malawi 67 cases vs. South Africa 42 cases; Bangladesh 42 cases vs. Vietnam 20 cases), because of higher disease incidence in settings with lower VE.

5. VPDI for severe disease among different vaccines

PCV and Hib conjugate vaccines have been licensed and introduced globally based on high VE, as well as studies documenting rapid reduction in etiologically confirmed disease and cost-effectiveness. Consequently, it is instructive to use VPDI to compare these vaccines – which have documented cost-effectiveness and are recommended for global use – with other vaccines that have been considered less successful based on a relatively low VE.

We reviewed pivotal studies for several new vaccines that the GAVI Alliance funds or is considering funding, including vaccines against pneumococcus (PCV) [5], Hib [4], malaria (RTS,S) [15,16], rotavirus [6,8], and cholera [17] to illustrate the general principles associated with using VPDI as a disease burden measure. Where investigators did not report VPDI, we calculated it as the incidence in the control group minus incidence in the vaccinated group. Additionally, we included only studies from less developed settings.

For the current analysis, we focused on severe disease, as defined by the study investigators; the exception was the cholera trial, which reported only data for all diarrheal disease whether severe or not. Within the category of severe disease, we present VPDI for the outcome yielding the highest value, that is, the most sensitive outcome; for example, etiology non-specific outcomes and all severe disease regardless of hospitalization were preferred over etiology specific or hospitalized disease. To maximize the VPDI estimate, we used per protocol analyses, and where available VPDI against all disease episodes rather than first episode. We used trial results for children only, even if reported for older persons; this issue was relevant only for the cholera vaccine trial. PCV and Hib conjugate vaccines prevent primarily non-bacteremic pneumonia but also a substantial amount of meningitis; consequently, for these vaccines we combined VPDI for meningitis outcomes (using the principles just described) and pneumonia outcomes to create a composite VPDI for severe disease. We think this approach justified because VPDI should reflect the overall public health value of a vaccine.

Table 2 describes trial characteristics, while the supplemental Table provides data derived from the studies. Fig. 1a presents the traditional measure used by regulatory agencies to assess the appropriateness of licensure, namely VE against disease that was etiologically confirmed (and vaccine serotype specific for PCV). PCV, Hib conjugate, and rotavirus vaccine in South Africa and Vietnam all demonstrated 70% or greater VE on some measure (Vietnam had 73% rotavirus VE among children during the first year of life and 64% during the entire two year study period), compared to 50% or less for rotavirus vaccine in Malawi and Bangladesh, as well as for malaria and cholera vaccines.

Table 2.

Characteristics of seven studies from nine study sites included in measurement of VPDI shown in Fig. 1.

Trial (reference)	Vaccine	Vaccine schedule	Follow-up period	Episodes	Outcome
Rotavirus Africa (Malawi, South Africa) (6)	Rotarix (human, live, attenuated, monovalent G1P8 vaccine)	2 or 3 doses at age 6, 10, 14 weeks	2 weeks after last dose to age 1 year	All episodes	Severe rotavirus gastroenteritis defined by Vesikari score
Rotavirus Asia (Bangladesh, Vietnam) (8)	Rotateq (human-bovine reassortment pentavalent)	3 doses at 6, 10,14 weeks	14 days after dose 3 up to 24 months	All episodes	Severe rotavirus gastroenteritis defined by Vesikari score
Pneumococcus The Gambia (5)	9-valent pneumococcal conjugate vaccine (1, 4, 5, 6B, 9V, 14, 18C, 19F, 23F)	3 doses separated by at least 25 days, age 6–51 weeks at initiation	2 weeks after first dose to illness, withdrawal, death, age 30 months, or study completion	First episode	Severe all cause pneumonia + confirmed pneumococcal meningitis
Haemophilus influenzae type b (Hib) Indonesia (4)	Hib conjugate vaccine (PRP-T)	3 doses a scheduled age of 6,10,14 weeks delivered through routine immunization program	2 weeks after first dose until death, age 2 years, or study completion	All episodes	Severe all cause pneumonia + either meningitis hospitalization or outpatient seizure
Malaria multi-center older children (15)	RTS,S, adjuvanted anti-circumsporozoite vaccine	3 doses at 1 month intervals, age 5–17 months at initiation	Age first vaccination to 12 months after dose 3	All episodes	Any parasitemia + one or more markers of severe malaria
Malaria multi-center young infants (16)	RTS,S, adjuvanted anti-circumsporozoite vaccine	3 doses at 1 month intervals age 6 to 12 weeks at initiation	Age first vaccination to 12 months after dose 3	All episodes	Any parasitemia + one or more markers of severe malaria
Cholera India (17)	Shanchol: oral, killed, whole-cell vaccine with O1 El Tor Inaba, O1 classical Ogawa, O1 classical Inaba, and O139 V. cholerae	2 doses at 1 month intervals, age 1–4 years at initiation	14 to 730 days after first vaccination	First episodes	Confirmed cholera (3+ stools in 24 h, or <3 stools with dehydration, plus V. cholerae identification in stool)

Fig. 1.

(a) Vaccine efficacy against various severe outcomes from randomized controlled trials of vaccines (Table 2). (b) Vaccine preventable disease incidence (VPDI) against various severe outcomes from randomized controlled trials of vaccines. Fig. 1a and b report data from the same nine trial sites with Fig. 1a in descending order of VE and Fig. 1b in descending order of VPDI. Green bars indicate >60% VE and red cross-hatched bars 50% or less VE.

Supplementary Table S1 related to this article can be found, in the online version, at http://dx.doi.org/10.1016/j.vaccine.2014.04.019.

When considering VPDI against severe disease syndromes, rather than VE against etiologically confirmed disease, the assessment of success for the vaccines with relatively low VE changes dramatically (Fig. 1b). Rotavirus vaccine in Malawi had the highest VPDI, with 6700 cases of hospitalization prevented per 100,000 child-years of follow-up. RTS,S malaria vaccine led to VPDI for severe malaria of 2300 and 900 per 100,000 child-years among older and younger infants, respectively, despite VEs of 47% and 38%.

When anchoring one vaccine to another to assess its public health value using VPDI, several other issues need be considered. The varying duration of immunity is a critical issue. RTS,S immunity may fade over two to three years despite ongoing risk; by contrast, immunity to Hib conjugate vaccine when given with a booster after infancy lasts throughout the period of greatest risk, as does rotavirus vaccine despite waning immunity, while cholera VE remains at 65% through 5 years of follow-up [18]. While we reviewed severe disease impact for different vaccines, some vaccines prevent not only acute disease but also severe sequelae such as the severe cognitive, neurologic and physical disabilities that result from pneumococcal and Hib meningitis, as well as cerebral malaria. By contrast, rotavirus and cholera cause little in the way of long-term sequelae. All of the vaccines, except RTS,S, have been shown to provide substantial indirect effects, with these effects sometimes exceeding the benefit from direct protection [18–20]. Long-term vaccine effects on microbial ecology must be considered, such as serotype replacement following PCV, which can blunt the total impact on all disease. Lastly, the ages at which disease occurs will influence the overall burden of preventable disease, through either direct programs targeting older persons or indirect effects. For example, rotavirus vaccine prevents disease almost entirely during the first 2 years of life, and Hib vaccine during the first 5 years. By contrast, pneumococcal disease, malaria and cholera are not limited to young children, and so vaccines can potentially prevent disease throughout life.

6. Discussion

We provide several examples of how VE, the traditional barometer of vaccine performance, often fails to predict the public health impact of a vaccine on disease burden. This concept is particularly important when assessing vaccines with relatively low VE. In these circumstances, we propose that VPDI against severe clinical disease syndromes must be considered as well and assessed in relation to vaccines that are considered good public health investments. This measure provides a direct assessment of the preventable incidence of the most relevant public health outcome for policy-makers. It incorporates not only disease incidence, but also VE.

VPDI is calculable in theory against any outcome rather than depending on etiologically confirmed disease, whether this is pneumonia, meningitis, gastroenteritis, or severe febrile illness. Where prevention of less severe disease plays an important role in decision-making, VPDI can be calculated for relevant outcomes. These features allow for a more thorough estimate of a vaccine’s value, as well as more relevant comparisons. For example, ministries of health likely have more interest in the ability of rotavirus vaccine to prevent an absolute number of hospitalizations for acute gastroenteritis than a certain proportion of confirmed rotavirus disease. It should be noted, however, that measuring impact on less specific outcomes (e.g., hospitalizations, outpatient illness) will require larger sample sizes and more resources.

We present data from rigorously performed clinical trials that performed analyses of children who received one or more study vaccine doses. However, other designs could incorporate the concept of VPDI. For example, in a cluster-randomized trial, one could estimate VPDI as the difference in outcome incidences between residents of control and intervention clusters regardless of vaccine status; this type of analysis would incorporate indirect protection among vaccine failures as well as unvaccinated persons. One could also estimate VPDI as the difference in outcome incidences between residents of the same community before and after vaccine introduction; such an analysis would need to control for potentially confounding factors that may occur over time.

VPDI calculated in one setting may or may not apply to other settings. Some evidence exists that Hib pneumonia VPDI incidences remain relatively stable across various settings [21]. While data are sparse for other diseases, one could predict that epidemic diseases (e.g., serogroup A meningococcal disease in the African meningitis belt, cholera), diseases with extensive year-to-year variability (e.g., influenza, cholera) and diseases for which the vaccine target changes rapidly (e.g., influenza) will have highly variable VPDIs depending on setting. In these cases, local data or data from multi-year studies may be necessary. However, the same issues exist, and possibly to a greater degree, when measuring vaccine efficacy or effectiveness against non-specific outcomes. For example, if the incidence of clinical meningitis due to Hib remains constant over time while clinical meningitis incidences for other etiologies (viruses, meningococcus) change over time, then Hib VE against clinical meningitis will vary while VPDI remains constant.

Since the public health value of one vaccine does not alter the value of a different vaccine, our goal was not to place vaccines in competition with each other but rather to provide an additional perspective for decision-making. As discussed above, many other important issues must be included in assessing the usefulness of a particular vaccine. Hib and PCV vaccines were early on regarded as valuable tools in developing countries due to their high VE for invasive disease and proven cost-effectiveness, even before data were available showing impact on all cause pneumonia and – for PCV – mortality [22–24]; consequently, these were some of the first new vaccines supported by the GAVI Alliance. Subsequently, WHO and GAVI supported introduction of rotavirus vaccines, despite their lower VE, due in part to recognition of their VPDI. If licensed, future decisions to introduce RTS,S vaccine also likely would benefit from discussions about its VPDI, rather than exclusively VE.

Most of our arguments apply to post-licensure assessment of a vaccine’s public health impact. However, it is worth raising the possibility that regulatory agencies consider VPDI along with VE and safety in making licensure decisions, particularly where VE against etiologically confirmed disease is considered low. We know of no a priori prohibition against use of such a measure by the US Food and Drug Administration. In principle, regulatory agencies have preferred VE because of its perceived stability across settings, but this has not proven to be the case for some vaccines, as noted above.

We have several recommendations regarding the utilization of VPDI. First, we propose that Phase III or IV vaccine trials should include VPDI as a specific outcome in the analytic plan and investigators should present these data in the primary report of study results. Second, global funders, such as the GAVI Alliance, should use VPDI in considering vaccines for support, such as during the prioritization process for the every fifth year update of the Vaccine Investment Strategy. Third, National Immunization Technical Advisory Groups and national decision-makers should use VPDIs as a primary tool for evaluating the value of new vaccines. Fourth, relatively low VE in the context of high VPDI may merit use of a vaccine in spite of low VE, while indicating an opportunity to have even higher impact with an improved vaccine; in fact, we would argue that improvement of vaccines that have both low VE and high VPDI should be a research priority. By contrast, high VE against etiologically confirmed disease with relatively lower VPDI suggests that efforts should focus on expanding disease prevention qualities of future vaccines, such as adding more serotypes or extending duration of protection. Finally, vaccine producers and investigators should engage regulatory authorities to assess whether a measure of absolute impact such as VPDI could have a role in licensure decisions.