Use of vaccines as probes to define disease burden

Abstract

Vaccine probe studies have emerged in the past 15 years as a useful way to characterise disease. By contrast, traditional studies of vaccines focus on defining the vaccine effectiveness or efficacy. The underlying basis for the vaccine probe approach is that the difference in disease burden between vaccinated and unvaccinated individuals can be ascribed to the vaccine-specific pathogen. Vaccine probe studies can increase understanding of a vaccine’s public health value. For instance, even when a vaccine has a seemingly low efficacy, a high baseline disease incidence can lead to a large vaccine-preventable disease burden and thus that population-based vaccine introduction would be justified. So far, vaccines have been used as probes to characterise disease syndromes caused by Haemophilus influenzae type b, pneumococcus, rotavirus, and early infant influenza. However, vaccine probe studies have enormous potential and could be used more widely in epidemiology, for example, to define the vaccine-preventable burden of malaria, typhoid, paediatric influenza, and dengue, and to identify causal interactions between different pathogens.

Introduction

Traditionally, vaccine studies have focused on characterising the vaccine’s efficacy. By contrast, vaccine probe studies characterise the disease. A vaccine probe study is a randomised clinical trial of a vaccine of known efficacy; the difference in the incidence of disease between vaccinated and unvaccinated people represents the vaccine-preventable disease incidence (VPDI). Vaccine probe studies also estimate the aetiological fraction (ie, the proportion of cases of a disease syndrome caused by the pathogen), which is often difficult to define through other study designs. Vaccines have already been used as probes to characterise disease syndromes caused by Haemophilus influenzae type b (Hib), pneumococcus, influenza, and rotavirus. However, vaccine probe studies have much potential and could be used more widely in epidemiology, for example, to define the vaccine-preventable burden of malaria, and to clarify causal interactions between pathogens. This Review outlines the origin of these studies, provides examples of successful vaccine probe studies, and offers potential directions for future vaccine probe studies.

The origin of vaccine probe studies

Only in the past 15 years has the potential of vaccine trials to estimate disease burden been understood. The first trials to be interpreted in this way were ones assessing the effectiveness of conjugate vaccines against Hib.^1,2 In the 1990s, the role of H influenzae in causing pneumonia was acknowledged, but the magnitude and type-specific burden remained unknown.³ A vaccine trial in Gambian infants⁴ showed an efficacy of 95% against culture-confirmed invasive Hib disease and 21% against all-cause radiologically confirmed pneumonia.² The vaccine’s efficacy against pneumonia was much greater than expected and, together, these results supported the first introduction of Hib vaccine into the Expanded Program on Immunisation (EPI) in Africa in 1997. In Asia, however, most health ministries did not consider the incidence of paediatric Hib meningitis (usually <0.1 per 1000 person-years) derived from prospective surveillance to be sufficient to justify introduction of Hib vaccine.⁵ By contrast, most Hib pneumonia episodes in Asia are non-bacteraemic and therefore invisible to normal clinical surveillance methods. This inability to easily diagnose Hib pneumonia generated an impasse whereby the investment case for Hib vaccine in Asia was contingent on an epidemiological parameter that could not easily be estimated.

The solution was to use the vaccine itself as a probe to reveal how much pneumonia could be prevented by Hib vaccine.^6,7 The focus of this new trial was not vaccine efficacy, which was already well established, but the difference in pneumonia incidence between vaccinated and unvaccinated populations. This study, which took place in Lombok, Indonesia, was the first prospectively designed vaccine probe study. Subsequently, this approach of designing, or interpreting, vaccine studies from the perspective of disease effect has been used for several other vaccines.

Insights from vaccine probes

Vaccine probe studies yield three distinct insights into the epidemiology of vaccine-preventable diseases. First, they can estimate the absolute burden of disease incidence that is preventable by a vaccine. Separate vaccine-preventable burden estimates can be calculated for different disease manifestations. For example, for pneumococcus and Hib, these include invasive disease, purulent meningitis, radiologically confirmed pneumonia, clinical pneumonia, otitis media, and death. The effect of a vaccine can also be measured against health-system endpoints such as health-care visits or drug use. Second, vaccine probe studies can measure the contribution of a specific pathogen to a broad clinical syndrome. For example, a vaccine could be used to estimate the proportion of cases of clinically defined pneumonia or culture-negative diarrhoea that are attributable to the pathogen targeted by vaccination. Third, vaccines can be used to investigate the causal chain in disease pathogenesis. For example, if an infectious disease is hypothesised to cause a specific form of cancer, a trial of an effective vaccine against the infection could test the hypothesis.⁸

The epidemiological methods of vaccine probe studies do not differ from those of vaccine efficacy or effectiveness studies. Like vaccine efficacy studies, vaccine probe studies use randomised controlled trials and non-randomised designs,⁹ which measure disease incidence before and after vaccine introduction.^10–13 In view of the similarity in methods, the probe approach can be incorporated into the design of a vaccine efficacy study (eg, the Indonesian Hib vaccine study) or it can be applied retrospectively or prospectively to studies designed primarily to measure vaccine efficacy.

How vaccine probe studies work

Three outcomes can be defined in a vaccine probe study: vaccine effectiveness or efficacy (VE); aetiological fraction; and VPDI. Assuming a controlled cohort study design, VE is defined as 1 minus (incidence in the vaccinated divided by incidence in the unvaccinated) and can be calculated for any disease syndrome for which incidence in vaccinated and unvaccinated populations is known. Aetiological fraction is defined as the syndrome-specific VE divided by the VE for aetiologically confirmed cases. VPDI is calculated as incidence in the unvaccinated population minus the incidence in the vaccinated population, which is mathematically equivalent to the product of VE and incidence in the unvaccinated population. Therefore, VPDI is a combined measure of the ability of the vaccine to prevent disease and the preexisting disease burden.

An important notion in understanding how vaccine probe studies work is the specificity of clinical outcomes for the pathogen targeted by the vaccine. In the figure, the outcome that is most specific for the pneumococcus is blood culture-confirmed disease, but this represents only a small proportion of all pneumococcal pneumonia episodes because most bacterial pneumonias are non-bacteraemic.^2,6,14,15 The proportion of pneumonia syndromes attributable to pneumococcus varies. Pneumococcus is assumed to cause all cases of culture-confirmed pneumococcal pneumonia; vaccine probe study results suggest that pneumococcus causes a substantial proportion of radiologically confirmed pneumonia, a smaller proportion of severe, non-radiologically confirmed pneumonia, and a smaller proportion still of non-severe pneumonia. The most specific syndrome for pneumococcal disease (ie, culture-confirmed pneumonia) is used to estimate vaccine efficacy and by regulatory agencies for licensure on the assumption that vaccine efficacy remains relatively stable across populations. The remaining syndromes that are more sensitive for pneumococcal pneumonia provide more information about the overall preventable disease burden and hence the public health usefulness of a vaccine.

Figure. Proportions of different categories of pneumonia attributable to pneumococcus (left) and the effect of pneumococcal vaccine (right).

On the left, for the more specific outcomes towards the top of the triangle, vaccine efficacy is higher, whereas for the more sensitive outcomes at the base of the triangle, the vaccine preventable disease incidence is higher, as is the public health utility of vaccine.

For a vaccine that is efficacious against pneumococcal disease, the measurable vaccine efficacy against highly specific outcomes such as culture-confirmed pneumonia will be high but it will be low against less specific outcomes (figure). Table 1 shows the numerical consequences of this difference and the results of a hypothetical probe study of a pneumococcal vaccine against pneumonia. The VE against aetiologically confirmed disease is 90%, suggesting the vaccine provides good individual protection—a regulatory requirement for licensure of vaccines designed to provide direct protection. Despite the high VE in this case, however, the VPDI for this outcome is only 0.45 cases per 1000 person-years. By contrast, the VPDI for all pneumonia is 4.5 per 1000 person-years, providing evidence for public health decision makers to introduce the vaccine, despite a VE for this outcome of only 9%. No minimum threshold of VPDI exists for decision makers to introduce vaccine; in addition to VPDI, this decision will depend on local priorities and additional concerns such as disease severity, long-term sequelae, duration of immunity, indirect protection provided to non-vaccinated groups, and vaccine cost.

Table 1.

Hypothetical example of vaccine probe results for a pneumonia-causing pathogen

	Incidence in unvaccinated people (IU)	Incidence in vaccinated people (IV)	Calculated vaccine effectiveness [VE = 1−(IV/IU)]	Calculated aetiological fraction [AF = VE/VEc*]	Vaccine-preventable disease incidence (VPDI = IU − IV)
Aetiologically confirmed disease	0.5	0.05	90%	100%	0.45
Pneumonia with alveolar infiltrate	2.0	1.28	36%	40%	0.72
Clinical severe pneumonia	8.0	6.2	23%	25%	1.8
All pneumonia	50.0	45.5	9%	10%	4.5

^*VEc=vaccine effectiveness for aetiologically confirmed cases. All incidence calculations are per 1000 person-years. Calculation of the aetiological fraction relied on the assumption that the vaccine worked equally well against all measured clinical outcomes caused by the pathogen.

To calculate a sample size in a prospectively designed probe study, some estimations of the VE and the incidence of the outcome are required. For example, in the Lombok Hib study,⁶ although radiological pneumonia occurred at a lower incidence than all WHO-defined pneumonia, the expected VE for radiological pneumonia (20%) was higher than for all pneumonia (5%), requiring estimated sample sizes of 636 cases versus 7132 cases, respectively, to record a significant reduction in the vaccinated group. Thus, the size and cost of a probe study is affected by the primary endpoint of interest.

Examples of vaccine probe studies

Hib

Tables 2, 3, and 4 show the results of three large randomised controlled trials from The Gambia, Chile, and Indonesia. The Gambian study⁴ was a vaccine effectiveness study against pneumonia outcomes that laid much of the groundwork for later probe studies by assessing effectiveness against clinical syndromes. The Chile study¹⁶ was reinterpreted after implementation as a probe study. The Indonesian trial⁶ was the first study prospectively designed and interpreted as a probe study. For all three trials, the results showed that VE increased as the disease outcome became more specific for the aetiology prevented. In The Gambia and Chile, vaccine prevented more than 20% of radiologically confirmed pneumonia (or pneumonia with consolidation), whereas little prevention of this outcome was recorded in Indonesia. When comparing VPDI in Chile and Indonesia (The Gambia study did not report VPDI), the former found a maximum preventable incidence of 2.5 per 1000 person-years for a composite clinical endpoint (conslidation, effusion, bronchial breath sounds, or increased erythrocyte sedimentation) whereas the Indonesian study found a maximum preventable incidence of 15.8 per 1000 person-years for clinical pneumonia, more than six-fold higher than in Chile.

Table 2.

Haemophilus influenzae type b vaccine effectiveness against non-microbiological pneumonia outcomes in The Gambia^2,4 in children from first vaccination to 5–36 months after first vaccination

	Number vaccinated	Number of controls	VE (%)
Cough with fast breathing or chest-wall indrawing	873	913	4.4%
Lower chest-wall indrawing	436	466	6.5%
Any obvious radiological infiltrate	198	251	21%*
Radiological consolidation or effusion	88	115	25%*

VE=Vaccine effectiveness. Most of the cases were admitted to hospital. Vaccine-preventable disease incidence was not calculated in the published papers for this study.

^*VE has 95% CIs that exclude 0.

Table 3.

Haemophilus influenzae type b vaccine-preventable disease incidence for children admitted to hospital with pneumonia in Chile^16,17

	Control incidence*	VE (%)	VPDI*†
Radiological consolidation, effusion, bronchial breath sounds, or raised ESR	9.7	26%‡	2.5
Radiological consolidation or effusion	5.0	22%	1.1

VE=Vaccine effectiveness. VPDI=vaccine-preventable disease incidence. ESR=erythrocyte sedimentation rate.

^*All incidences are per 1000 person-years of follow-up.

^†VPDI is calculated as vaccine effectiveness multiplied by control group incidence, and is shown per 1000 person-years of follow-up in children from age of first vaccination to age 2 years.

^‡VE has 95% CIs that exclude 0.

Table 4.

Hib vaccine-preventable disease incidence for pneumonia and meningitis in Indonesia⁶

	Control incidence*	VE (%)	VPDI*†
Pneumonia outcomes
Clinical pneumonia	395	4.0%‡	15.8
Hospital admission or clinical assessment of severe pneumonia	54.6	4.8%	2.6
Hospital admission for severe pneumonia	45.2	3.8%	1.7
Radiological consolidation or pleural effusion with WHO criteria18	8.9	−4.9%	−0.43
Meningitis outcomes
Hospital admission for meningitis or clinic assessment for seizures	7.0	22%‡	1.6
Hospital admission for meningitis	5.4	16%	0.87
Probable bacterial or confirmed Hib	0.86	55%‡	0.47
Microbiologically confirmed Hib	0.19	86%‡	0.16

Hib=Haemophilus influenzae type b. VE=Vaccine effectiveness. VPDI=vaccine-preventable disease incidence.

^*All incidences are per 1000 person-years of follow-up.

^†VPDI is calculated as the vaccine effectiveness multiplied by the control group incidence, and is shown per 1000 person-years of follow-up in children from age of first vaccination to age 2 years.

^‡VE has 95% CIs that exclude 0.

Findings of the studies also showed geographical variations in the epidemiology of Hib pneumonia. In The Gambia, most cases of vaccine-preventable pneumonia had severe symptoms, whereas the opposite was true in Indonesia, where few cases of vaccine-preventable Hib pneumonia had radiological abnormalities (such as lobar infiltrate or pleural effusion). These findings could point to earlier use of antibiotics, or to faster identification and referral of pneumonia cases.

For meningitis, the Indonesian trial showed a high VE but a low overall burden of aetiologically confirmed Hib disease. However, the vaccine probe study design showed a substantial burden of Hib meningitis that was not detected with standard laboratory techniques. The VPDI for meningitis admission or clinic assessment for seizures was 1.6 per 1000 person-years (95% CI 0.42–2.73).

Streptococcus pneumoniae

Pneumococcus is similar to Hib in that the largest burden of disease is non-bacteraemic pneumonia.^{14,15,19–21} Studies of pneumococcal vaccine also corroborate the finding that as the pathogen specificity of the disease outcome decreases, the VE falls, but the VPDI rises (table 5). Of five studies^{14,15,19,22,23,25} that used a WHO-standardised radiological definition of obvious lobar pneumonia or pleural effusion,¹⁸ and also met the criteria for inclusion in a Cochrane review meta-analysis, the summary VE was 27% (95% CI 15–36%, intention-to-treat analysis of HIV-negative children <24-months-old).^{14,15,19,22–25} The range of VEs (16–35%) in these studies was narrow. VPDI, on the other hand, varied widely by site, and was proportional to the baseline incidence; pneumonia reductions in rural Gambia (23.2 per 1000 person-years) were more than ten times greater than in the USA (1.6 per 1000 person-years). For four studies that measured the less specific outcome of clinical pneumonia,^{14,15,19,22,25} the VE ranged from 6% to 7% across three studies and was less than 0% in the fourth.²⁴

Table 5.

VE and VPDI for pneumococcal conjugate vaccine in trials measuring radiological pneumonia outcomes, by trial site

	Serotypes in vaccine (n)	Vaccine type invasive pneumococcal disease		All serotype invasive pneumococcal disease		Pneumonia with lobar radiological consolidation/pleural effusion18		Clinical pneumonia
		VE (%)	VPDI*	VE (%)	VPDI*	VE (%)	VPDI*	VE (%)	VPDI*
California, USA19,22	7	94%†	2.43	89%†	2.58	28%	1.55	7%	3.30
South Africa15 (HIV negative)	9	83%†	0.75	42%	0.43	20%†	2.31	7%	3.94
Navajo/White Mountain Apache, USA23	7	83%†	1.92	52%	0.77	−11%	−3.97	NA	N/A
The Gambia12	9	71%†	3.67	45%†	3.78	35%†	23.2	6%†	15.4
Philippines24,25	11	−199%	−0.33	0.4%	0	16%	3.62	−1%	2.05

VE=Vaccine effectiveness. VPDI=vaccine-preventable disease incidence. All results given are from intention-to-treat analysis for all enrolled children. Adapted from the Cochrane review of pneumococcal conjugate vaccines.²⁴ VPDI is expressed per 1000 person-years of follow-up.

^*Denominators were number of children for California, USA; South Africa; The Gambia; and Philippines; and were person-years for Navajo/White Mountain Apache, USA.

^†VE has 95% CIs that exclude 0.

The vaccine probe approach provides a minimum estimate of aetiological fraction of pneumonia caused by the serotypes of pneumococcus included in the vaccine. In the per-protocol analysis of the trial in The Gambia, the measured VE for radiologically confirmed pneumonia was 37%, suggesting that at least a third of all pneumonia episodes are caused by the nine serotypes of pneumococci included in the vaccine.¹⁴ In participants who received a placebo, the proportion of culture-positive episodes of radiologically confirmed pneumonia that was caused by vaccine serotypes was 58% (26 of 45 cases). If this accurately shows the vaccine serotype coverage of non-bacteraemic episodes, then a similar vaccine, with 100% VE against all serotypes, would have prevented 64% (ie, 37% divided by 58%) of radiologically confirmed pneumonia cases. The actual efficacy of the vaccine against pneumococcal pneumonia is unknown, but it is unlikely to exceed the measured VE against vaccine-type invasive pneumococcal disease (77%), because mucosal disease probably needs higher antibody concentrations for protection than invasive disease.^26,27 On this basis, a vaccine would have prevented at least 83% (ie, 64% divided by 77%) of all cases of radiologically confirmed pneumonia, implying that four-fifths of all radiologically confirmed pneumonia was caused by Streptococcus pneumoniae in the specific context of The Gambia, which had high and sustained Hib vaccine coverage at the time.

One other unique aspect of this trial was its ability to probe mortality from pneumococcal disease.^14,21 The VE against all-cause mortality in the trial was 16% (95% CI 3–28%) among children aged 6 weeks to 2 years, translating into a vaccine-preventable mortality reduction of 5 per 1000 person-years.¹⁴ The extent of the potential mortality reduction in this setting was not known before this vaccine probe analysis.

Influenza

In a non-randomised probe study in pilgrims attending the Hajj in Mecca,⁹ participants chose whether or not to receive the influenza vaccine, and the study documented the presence of influenza-like illness, other respiratory symptoms, and drug use during the Hajj. The unadjusted attack rates (the cumulative incidence of infection in a group of people during a period of time), for influenza-like illness among unvaccinated and vaccinated pilgrims were 62% and 36%, respectively, during the Hajj. The adjusted VE was 38%, suggesting that at least 38% of influenza-like illness episodes were caused by influenza virus. Overall, the influenza vaccine prevented 220 cases of influenza-like illness, 170 courses of antibiotics, and 230 courses of cold drug use per 1000 pilgrims.

A review of split-virion influenza vaccine studies estimated VPDIs by multiplying VE by the incidence or attack rate among the unvaccinated population.²⁸ The review showed that four studies of working age adults, which used relatively similar case definitions, reported VEs that varied widely from 25% to 80%. At the same time, the range for VPDI was narrow at 209–350 per 1000 people per year or per influenza season. Similarly, three studies found VEs of 36–84% against workplace absenteeism, but found similar VPDIs of 530–740 days per 1000 persons per year or per influenza season. One potential explanation for these results is that although influenza incidence was similar across study sites (leading to similar VPDIs), the incidence of non-influenza causes of the same clinical syndrome, such as respiratory syncytial virus, parainfluenza virus and rhinovirus, led to varying VEs. In another study²⁹ that tested whether influenza vaccine could reduce major cardiac events in people aged 50 years or older with previous acute coronary syndrome, researchers recorded a VPDI of 97 per 1000 people per year (value calculated subsequently, because investigators did not directly report this result).

In a study³⁰ comparing influenza versus 23-valent pneumococcal polysaccharide vaccination given to pregnant women in Bangladesh, receipt of influenza vaccine prevented 29% more respiratory illnesses with fever episodes and 42% more clinic visits through age 6 months in children born to vaccinated women.

Rotavirus

Findings of a probe analysis¹³ using a cohort design of the Rotashield rotavirus vaccine in the USA showed that up to 85% of gastroenteritis hospital admissions in young children were due to rotavirus, compared with estimates of 48–53% from surveillance. The first year of national rotavirus vaccine use in Finland showed that the VPDI for all-cause acute gastroenteritis (10.7 per 1000 person-years) was 2.7 times greater than the VPDI for laboratory-confirmed rotavirus acute gastroenteritis (3.9 per 1000 person-years) in children aged younger than 1 year.³¹

In resource-poor settings where access to health care is restricted, and also where most gastroenteritis deaths happen, it is crucial to define disease burden accurately to prioritise scarce resources.³² In a trial of pentavalent rota virus vaccine in rural Kenya, the VE against severe rotavirus gastro enteritis presenting at a health facility in the first year of life was 84%, and 33 cases were prevented per 1000 person-years.^33,34 By contrast, through active monthly follow-up of the same study children at home, the VE against all-cause gastroenteritis with severe dehydration was just 34%, consistent with another study in South Africa.^34,35 The aetiological fraction for rotavirus in community based cases of severe diarrhoea in infancy can be estimated as VE against all-cause gastro enteritis with severe dehydration (34%) divided by VE against rotavirus-confirmed severe gastroenteritis (84%), or 40%. The study also estimated VPDI among children in the community as 190 per 1000 person-years in the first year of life, six times higher than in children presenting to health-care facilities. This finding shows that VPDI for outcomes dependent on access to care can vary across settings because the use of health-care facilities varies.

Vaccine probes to define interactions in disease causality

Vaccine probe studies can be used to test causality in disease. For example, malaria has been postulated to induce haem oxygenase, which limits the production of bactericidal reactive oxygen species, increasing susceptibility to intracellular bacterial diseases such as non-typhoid Salmonella.³⁶ A vaccine that effectively reduces infection with Plasmodium falciparum could be used to test this hypothesis directly in a trial in which the intervention was malaria vaccine and the outcome was invasive salmonella disease.

Epidemiological and laboratory evidence suggests that infection with influenza virus is followed, after a short interval, by an increased susceptibility to bacterial pneumonia, especially that due to pneumococcus or Staphylococcus aureus.^37–40 The causal nature of this association, and the magnitude of the effect, could be confirmed and measured in a vaccine probe study of an effective influenza vaccine in which the outcome is culture-confirmed bacterial pneumonia. The pathogenesis of both diseases shares common characteristics and so they might be acting sim ultaneously to cause one disease episode.⁴¹ In this hypothesis, the efficacy of a vaccine against pneumococcal pneumonia would differ between influenza-exposed and influenza-unexposed pop ulations. The statistical test of the hypothesis, in the analysis of pneumococcal disease, would be the interaction between influenza infection and vaccination status. Researchers exploring these mechanisms have not usually tested such interaction, probably because most primary vaccine studies are only powered to examine main effects.^38,42,43 When designing vaccine probe studies that are intended to test the disease consequences of pathogen infection, the key point is to specify the causal model under investigation at the outset, and select the appropriate vaccine intervention and analysis to match that hypothesis.

Challenges in the design and interpretation of vaccine probe studies

The major assumption of vaccine probe studies in estimating the aetiological fraction of a pathogen is that a reasonable estimate of true VE exists. This value can be obtained from a previous study or from a nested study of VE against microbiologically confirmed outcomes during the course of the probe study of a broader outcome. Nested studies have the advantage that they control for variability in vaccination conditions, such as cold chain integrity, and population characteristics (eg, HIV prevalence). In vaccines in which VE varies each year depending on the antigenic match, such as for influenza, a nested study is needed. The measured VE for a microbiologically confirmed endpoint might not represent the true VE for a broader endpoint. Unlike VPDI, measuring aetiological fraction requires an estimate of VE against aetiologically confirmed disease. It also assumes that this measured VE is equivalent to the VE against non-aetiologically confirmed disease. If the VE against aetiologically confirmed disease is greater than that against disease truly due to a pathogen, but that remains aetiologically non-confirmed (such as non-bacteraemic Hib or pneumococcal pneumonia), the aetiological fraction will be underestimated, and vice versa.

By contrast, VE is not needed to estimate VPDI—the amount of disease prevented by the vaccine is a simple subtraction of rates. The true incidence of the target disease is given as the VPDI divided by VE, but for the purposes of public health policy, the VPDI alone is most important because it shows the amount of disease preventable by vaccination. Hence, even a vaccine with a low VE can be of public health interest if the disease it targets has a high background incidence, such as happened during assessment of the RTS,S malaria vaccine.^44,45 Similarly, because VE might be related to the force of infection, the VE could be lower in a high incidence than in a low incidence setting, whereas the VPDI would be greater in the high incidence setting, as was shown with rotavirus vaccines.^34,35

In some vaccine trials, the first event of a disease is specified as the primary outcome measure;^14,15,33,35 this is useful for measuring vaccine performance for regulatory purposes because people who have recurrent events can have substantially different vaccine responses. However, first-event analyses, by definition, can only estimate the VPDI or aetiological fraction of first disease episodes and will underestimate the disease burden if the pathogen causes recurrent vaccine-preventable episodes, even if the VE against recurrent episodes is lower. Additionally, first and subsequent disease episodes incur similar public health costs, and so it would be relevant to include recurrent episodes when investigating the full public health effect of a vaccine.⁴⁶ Similarly, vaccine probe studies should use an intention-to-treat rather than per-protocol analysis because intention-to-treat provides a realistic estimate of VPDI that accounts for imperfect vaccine delivery.

Vaccines can also have a substantial positive effect on the unvaccinated population by reducing pathogenic transmission. For some vaccines, such as pneumococcal conjugate vaccine, this indirect or so-called herd protection can prevent more cases in the population than the direct protective effect.⁴⁷ For a vaccine with important indirect effects, the estimate of VPDI should incorporate both direct and indirect protection by using a cluster-randomised trial design. This was the approach taken by the Indonesian Hib vaccine study.⁶ An individually randomised design would have compared the combination of direct and indirect effects with indirect effects alone; if indirect effects were high, such a study might find no additional benefit to the vaccinated group.

The interpretation of probe studies relies on the assumption that the vaccine does not prevent disease other than that caused by the target organism. However, rotavirus vaccine might prevent other viral gastroenteritis causes, such as adenovirus and sappovirus.¹³ Moreoever, some researchers have postulated that some vaccines have non-specific immune-modulating effects.⁴⁸ These non-specific vaccine effects would not alter estimates of the VPDI for a clinical syndrome, but would lead to erroneous estimates of the pathogen-specific aetiological fraction.

Probe studies are subject to the same design issues as other vaccine studies, including selection bias and confounding. As in studies to establish VE, study limitations tend to be greater with non-randomised designs. In investigations before and after vaccine introduction, factors other than vaccine introduction, such as secular trends in rates of hospital admission or mortality, can affect the estimate of disease burden, and thus inaccurately estimate VPDI. This bias can be measured by studying a comparison disease, not affected by vaccine, but with similar epidemiology and disease presentation (eg, following pneumococcal meningitis in a Hib vaccine study).

VE against aetiologically confirmed disease is normally similar across different populations and time periods, although rotavirus vaccine is an exception with lower VE in low-income populations.^34,35 By contrast, VE against non-aetiologically confirmed disease syndromes and aetiological fraction both vary with fluctuations in the epidemiology of other aetiologies causing the syndrome. However VPDI varies with aetiology-specific disease incidence, and so might be a more appropriate measure to compare vaccine effect across study sites than aetiological fraction or VE for non-aetiologically confirmed disease outcomes. Both VPDI and aetiological fraction can vary by patient age as immune response and disease epidemiology change, as well with vaccine formulation. For example, 13-valent pneumococcal conjugate vaccine shows higher VPDI and aetiological fraction than 7-valent vaccine, in settings in which the additional serotypes cause disease.

Ethical considerations

Incorporating an assessment of VPDI or aetiological fraction into a study that is prospectively designed to measure vaccine efficacy does not generally present ethical problems. However, if a study is designed from inception as a randomised controlled trial to estimate VPDI, then the ethical justification needs careful consideration because some participants would be randomly assigned not to receive a vaccine, even though it is known to be effective. Thus, from the point of view of the vaccinated individual, vaccine probe studies with a randomised design might only be regarded as ethically justified if insufficient evidence exists of the disease burden to predict whether the vaccine will provide a sufficient marginal benefit to the trial participant.

However, the level of individual marginal benefit sufficient to preclude a randomised controlled trial is difficult to define. For example, even if existing disease burden data suggest a small individual benefit from vaccination, there might be insufficient data to justify the introduction of the vaccine at population level. This argument has allowed some ethical committees to justify randomised trials to document VPDI in a particular setting considering vaccine use.⁶ However, randomised trials to define VPDI should not delay vaccine introduction when the disease risk in both the individual and population are known, and recommendations for universal vaccine implementation exist.⁴⁹

Use of the VPDI to develop public health messages presents some challenges. Parents might care most about whether a particular vaccine will protect their own child, as shown by the VE (against either aetiologically confirmed or non-aetiologically confirmed disease), rather than total preventable burden. Consequently, the communication of the results of vaccine probe studies, including their role in developing models for burden of disease and health economics, could be most effective when aimed at public health decision makers and funders.

Future vaccine probe studies

There are several areas where vaccine probe studies could be useful in the future. Influenza is a difficult infection to study with conventional methods. Influenza can cause asymptomatic disease, interact with other causes of respiratory disease, be undetectable by the time a person presents at a health-care facility, even if the virus was in the causal chain of infection, and might not be detectable through diagnostic tests. Every year, a vaccine is formulated to target circulating strains, and these vaccines could be incorporated into probe studies to overcome these challenges in understanding influenza burden, and help estimate the public health benefit of immunisation. Influenza vaccine probe studies have unique issues in that they need to take into account seasonal and yearly fluctuations in overall and vaccine type disease, and the contribution of influenza in causing pneumonia, which can vary in tropical versus subtropical areas.^50,51

The most important clinical outcome of rubella infection is congenital rubella syndrome. Although typically characterised by cataracts, congenital heart disease, and hearing impairment, the potential consequences of intrauterine rubella infection are much broader. A well-designed maternal immunisation trial could define the VPDI of rubella-associated stillbirth and congenital anomalies.

In studies of the RTS,S malaria vaccine, with patients followed-up for 12 months and receiving three vaccine doses at monthly intervals, VE against severe malaria was 37% for infants enrolled at 6 to 12 weeks of age and 47% for toddlers enrolled at 5 to 17 months of age.^44,45 However, the VPDI for severe malaria could be estimated from the results as 9 per 1000 person-years for infants and 23 per 1000 person-years for toddlers. Although the vaccine efficacy might seem low, assuming that it is sustained over time, the VPDI means that the public health value of this vaccine’s direct effects could be judged to be as high as that of other new childhood vaccines, such as Hib and pneumococcal vaccines. A vaccine probe approach for a malaria vaccine also could assess the contribution of malaria to non-specific child-health outcomes such as anaemia, growth, neurological development, and invasive bacterial infections, including salmonellosis.⁵²

Although probe studies in low-income countries have focused on the preventable burden of severe disease and death, more probe studies could be undertaken in high-income countries focusing on vaccine preventable health system and economic outcomes, such as drug use and hospital utilisation.⁸

Conclusions

Vaccine probe studies have been used successfully to provide disease burden data to support public health decision making for Hib, pneumococcal, influenza, and rotavirus vaccines. The tremendous investment in vaccines by the GAVI Alliance, extending coverage of existing vaccines and introducing a broad array of new vaccines, has focused public health attention on assessing the value of vaccines. This approach goes beyond the demonstration of VE to the measurable effect of vaccines in the population. Vaccine probe studies directly support this approach, and could be used more widely. First, the public health usefulness of VE trial results could be greatly enhanced by adding the outcomes of VPDI and aetiological fraction to existing analysis plans. Moreover, VE studies of new vaccines should be designed from inception to capture common clinical and non-specific public health outcomes, even if the VE is expected to be moderate. Second, vaccine probe studies could be designed prospectively to estimate the disease burden of common vaccine-preventable diseases. Third, the vaccine probe approach could be used more widely to define the role of pathogens in causing disease. Vaccine probe studies are unique methods to reveal the biological role of a pathogen and the public health importance of a vaccine.

Search strategy and selection criteria.

Studies for inclusion were identified through PubMed searches for articles published between Jan 1, 1990, and Dec 31, 2013, in any language using the following terms: (“vaccine probe” or “probe study”) and (“Haemophilus influenzae type b” or “Hib” or “Streptococcus pneumoniae” or “pneumococcus” or “rotavirus” or “influenza”). This search yielded only a few references presumably because so few studies have been prospectively designed as probe studies. We augmented the search using references of identified articles and other studies known to the investigators. We included articles that reported or allowed calculation of vaccine-preventable disease incidence or aetiological fraction identified by vaccination.