Skip to main content
Clinical and Vaccine Immunology : CVI logoLink to Clinical and Vaccine Immunology : CVI
. 2016 Mar 7;23(3):228–235. doi: 10.1128/CVI.00604-15

Correlates of Protection against Influenza in the Elderly: Results from an Influenza Vaccine Efficacy Trial

Andrew J Dunning a, Carlos A DiazGranados a,, Timothy Voloshen a, Branda Hu a, Victoria A Landolfi a, H Keipp Talbot b
Editor: H F Staatsc
PMCID: PMC4783426  PMID: 26762363

Abstract

Although a number of studies have investigated and quantified immune correlates of protection against influenza in adults and children, data on immune protection in the elderly are sparse. A recent vaccine efficacy trial comparing standard-dose with high-dose inactivated influenza vaccine in persons 65 years of age and older provided the opportunity to examine the relationship between values of three immunologic assays and protection against community-acquired A/H3N2 influenza illness. The high-dose vaccine induced significantly higher antibody titers than the standard-dose vaccine for all assays. For the hemagglutination inhibition assay, a titer of 40 was found to correspond with 50% protection when the assay virus was antigenically well matched to the circulating virus—the same titer as is generally recognized for 50% protection in younger adults. A dramatically higher titer was required for 50% protection when the assay virus was a poor match to the circulating virus. With the well-matched virus, some protection was seen at the lowest titers; with the poorly matched virus, high levels of protection were not achieved even at the highest titers. Strong associations were also seen between virus neutralization test titers and protection, but reliable estimates for 50% protection were not obtained. An association was seen between titers of an enzyme-linked lectin assay for antineuraminidase N2 antibodies and protection; in particular, the proportion of treatment effect explained by assay titer in models that included both this assay and one of the other assays was consistently higher than in models that included either assay alone. (This study has been registered at ClinicalTrials.gov under registration no. NCT01427309.)

INTRODUCTION

The burden of influenza-associated morbidity and hospitalization falls disproportionately on the elderly. Among U.S. persons ≥65 years of age, 4 to 5 times greater rates of influenza-associated hospitalization have been reported than in the general population (1). Influenza-associated mortality among U.S. persons ≥75 years of age in the 10 influenza seasons from 1997–1998 to 2006–2007 averaged 141.15 per 100,000, compared to 11.78 per 100,000 for all age groups (2). Vaccination is considered the most effective strategy for the prevention of influenza, and vaccines targeted specifically for this population have been developed (3). However, low rates of detectable influenza in the elderly and ethical requirements for an active comparator have meant that randomized trials to assess the clinical efficacy of these vaccines have been very large—44,000 and 32,000 subjects (4, 5). Improved understanding of the relationship between the immune response to vaccination as measured by immunological assays and subsequent occurrence of influenza illness in the elderly would further facilitate efforts to prevent influenza in this population through vaccination.

Here are reported the immunogenicity and correlates of protection results from one such efficacy trial, which compared a high-dose influenza vaccine with a standard-dose vaccine in 32,000 subjects aged ≥65 years.

Hemagglutination inhibition (HAI) antibody titers are believed to provide the best measure of protection against influenza (6), and elderly persons have been shown to have reduced immune response to vaccination as measured by HAI fold rise and to achieve, on average, lower levels of antibody as measured by geometric mean titers (GMTs) (7, 8). Candidate novel vaccines for the elderly have elicited higher HAI titers (4, 9, 10), in some cases approaching levels comparable to those seen in younger adults (11), and improvements in efficacy and effectiveness relative to standard inactivated trivalent vaccine have been seen (4, 5, 12).

A number of studies have quantified HAI protective titers in adults and healthy children. In an analysis of 12 such studies, de Jong et al. estimated titers for 50% protection to range from 15 to 65, despite differences in study design, immunological background, titration techniques, virus (sub)types, source of antibody (natural infection, vaccination), source of challenge (natural infection, experimental), and case definition; the median 50% protective titer was 28 (13). In a meta-analysis of 15 studies including 5,899 adult subjects and 1,304 influenza cases, Coudeville et al. estimated the overall 50% protective titer to be 17 (95% confidence interval [CI], 10, 29) (14) and, in a reanalysis of data from the classic Hobson et al. study (15), found such titer to be 29 (95% CI, 5, 195). From the same data, de Jong et al. calculated a titer for 90% protection of 192, and Coudeville's analysis indicated additional benefit becoming marginal above a titer of 150.

It has more recently been suggested that a protective HAI titer of 110 corresponds to 50% protection in children (16). However, information on protective HAI titers in the elderly is sparse, and it is not known if the generally accepted protective titer of 40 for younger adults is equally applicable in this older population.

Virus neutralization is a highly sensitive and specific method for detecting antibodies that inhibit virus entry or release and has the advantage of directly measuring functional virus neutralization; typically a microneutralization assay is used, and it has been considered superior to HAI for the detection of antibody response to some pandemic vaccines (17). A comparison of HAI and microneutralization responses to seasonal vaccine found a strong correlation between the assays (Pearson r, 0.75 to 0.87); however, microneutralization demonstrated greater fold-rise and seroconversion rates than HAI, suggesting that it may better measure vaccine response and so correspond more closely with vaccine efficacy (18). No standard microneutralization titer for 50% protection has been recognized.

Whereas hemagglutinin (HA) antibody is thought to inhibit infection by binding to the virus, neuraminidase (NA) antibody may act by impeding the release of virus from infected cells, reducing the severity of illness in some cases to asymptomatic levels (17). In a detailed study of protection against pandemic H1N1 in adults 18 to 49 years old, levels of serum and nasal antibody to hemagglutinin and neuraminidase were shown to be highly statistically significant in predicting both infection and infection with illness in univariate analyses. In multivariate analyses, serum antihemagglutinin and antineuraminidase were shown to be independently significantly predictive; further, among those infected, serum antineuraminidase was shown to be significantly predictive of illness, whereas serum antihemagglutinin only approached significance (19).

There is a need both to better understand HAI correlates of protection in the elderly and to explore assays reflecting different aspects of immune protection that may be relevant to the development of new and improved influenza vaccines; a recent influenza vaccine efficacy trial in the elderly provided the opportunity to investigate these topics.

MATERIALS AND METHODS

The efficacy trial has been described elsewhere (5). Briefly, it was a phase IIIb/IV, multicenter, randomized, double-blind, active-control trial conducted in adults aged 65 years and older over two influenza seasons. In year 1, 2011–2012, 14,500 eligible subjects were enrolled, and in year 2, 2012–2013, 17,489 eligible subjects were enrolled; subjects who had enrolled in year 1 were allowed to reenroll in year 2. Each year, subjects were randomized 1:1 to two treatment groups, receiving either high-dose inactivated influenza vaccine (IIV-HD) or standard-dose inactivated influenza vaccine (IIV-SD) before the start of the influenza season. The trial found that IIV-HD reduced the rate of influenza illness by 24.2% relative to IIV-SD for the primary endpoint case definition and by 18.3% to 51.1% for other case definitions.

The trial was approved by three institutional review boards (Quorum Review IRB, Western Institutional Review Board, and Vanderbilt University Institutional Review Board), and all subjects gave written informed consent. IIV-HD was Fluzone High-Dose (Sanofi Pasteur), and IIV-SD was Fluzone (Sanofi Pasteur).

IIV-HD contained 60 μg and IIV-SD contained 15 μg of hemagglutinin of each of the three strains of influenza virus specified by the U.S. Food and Drug Administration for inclusion in the year's vaccine. For year 1, the strains were A/California/7/2009 (H1N1), A/Victoria/210/2009 (H3N2), and B/Brisbane/60/2008, and for year 2 A/California/7/2009 (H1N1), A/Victoria/361/2011 (H3N2), and B/Texas/6/2011 (a B/Wisconsin/1/2010-like virus).

Each year, beginning 14 days after vaccination, active and passive surveillance for respiratory illness was conducted until the end of the influenza season. Subjects exhibiting any of seven respiratory symptoms (experiencing a respiratory illness [RI]) provided nasopharyngeal swabs within 5 days of symptom onset for laboratory detection of influenza, and they were monitored for 30 days to record other symptoms of influenza-like illness.

Laboratory-confirmed influenza was defined by a swab positive for influenza by culture or PCR. Culture-positive swabs were tested by hemagglutination inhibition with ferret antisera to determine antigenic similarity to a vaccine strain. A protocol-defined influenza-like illness (PD-ILI) was defined as one or more of the symptoms sore throat, cough, sputum production, wheezing, or difficulty breathing concurrent with one or more of the following symptoms: temperature of ≥37.2°C, chills, tiredness, headaches, or myalgia. A modified Centers for Disease Control and Prevention-defined influenza-like illness (CDC-ILI) was defined as sore throat or cough concurrent with temperature of ≥37.2°C.

Twenty-eight to 35 days after vaccination, a random subset of one-third of subjects provided blood samples for immunogenicity and correlates of protection assessment. All samples were tested by HAI for antibodies to the strains of virus in the vaccines. Two hundred fifty subjects who had provided blood samples developed laboratory-confirmed influenza, and of these, 152 developed A/H3N2 influenza in year 2. Therefore, the investigation of the correlates of protection examined the relationship between these cases and HAI, neutralization, and neuraminidase assays of year 2 samples, as there were insufficient cases of other types/subtypes and year to yield reliable results.

A/H3N2 was the predominant circulating subtype in year 2, but the circulating virus was a poor antigenic match to the vaccine virus (20). Although antigenic mismatch between vaccine and circulating viruses is generally the consequence of virus antigenic drift, for this particular influenza season the antigenic mismatch was related to mutations in the egg-propagated A/Victoria/361/2011 reference virus used as the vaccine seed. In contrast, the majority of A/H3N2 circulating viruses tested by public health authorities for this season remained antigenically similar to the cell-propagated reference A/Victoria/361/2011 virus. Egg-propagated reference viruses differed from cell-propagated reference viruses by the following amino acid substitutions: H156Q, G186V, and S219Y (21).

Samples from the 152 subjects who were year 2 A/H3N2 cases and who had consented to further testing of their samples (123 subjects), together with 552 randomly selected consenting noncases (selected from the 5,599 A/H3N2 noncases of year 2; a case-cohort design [22]) were assayed by HAI for antibody to the circulating A/H3N2 virus, by neutralization test (NT) against both the vaccine and circulating viruses, and by an enzyme-linked lectin assay (ELLA) for antineuraminidase N2 antibodies. A flowchart of the subjects enrolled and the assays conducted is shown in Fig. S1 in the supplemental material.

Sera to be tested in the influenza virus HAI assay were tested in 2 independent runs. Samples were first heat inactivated and pretreated with neuraminidase to eliminate the nonspecific inhibitors and the anti-turkey red blood cell (anti-TRBC) hemagglutinins, which may interfere with the test results. The treated serum samples were then serially titrated, starting at a 1/10 dilution, and incubated with 4 hemagglutinating units/25 μl of either vaccine virus (egg-grown PR8 reassortant of each vaccine strain) or the year 2 circulating A/H3N2 virus (Madin-Darby canine kidney [MDCK] cell-grown A/Victoria/361/2011). After incubation at 37°C for 1 h, a 0.5% TRBC suspension was added to the plates and was incubated at ambient temperature for an additional hour. Plates were then read using the tilt method, and the HAI titer was assigned as the reciprocal of the highest serum dilution that exhibited complete inhibition of hemagglutination. The geometric mean titer of both independent runs was used to determine the final titer.

The influenza virus neutralization test (NT) is an in vitro functional assay that measures the level of influenza virus neutralizing antibodies in human sera. This NT is based on the ability of neutralizing antibodies against influenza virus to inhibit the infection of MDCK cells with influenza virus. Twofold serially diluted, heat-inactivated sera were preincubated with 100 50% tissue culture infectious doses/50 μl of either the year 2 A/H3N2 vaccine virus (egg-grown PR8 reassortant of A/Victoria/361/2011) or the year 2 circulating A/H3N2 virus (MDCK cell-grown A/Victoria/361/2011) prior to addition of MDCK cells. Following overnight incubation, the cells were fixed and cell infection was determined by measuring the presence of influenza A virus nucleoprotein by enzyme-linked immunosorbent assay (ELISA). The absence of infectivity constitutes a positive neutralization reaction and indicated the presence of influenza virus-specific neutralizing antibodies in human sera. The endpoint of the assay was expressed as the titer (1/dilution) and was calculated by the intersection of the neutralization test sample optical density (OD) curve with the line representing the 50% neutralization point of the virus control ODs.

The N2 ELLA was based on the method described by Lambre et al. and Couzens et al. (23, 24). The ELLA measures neuraminidase (NA) inhibiting antibody by quantifying enzyme activity using peanut agglutinin (PNA) to bind to terminal galactose moieties that are exposed after NA-mediated enzymatic cleavage of fetuin. Briefly, serial dilutions of sera and a standard amount of H6N2 virus reassortant (N2 from A/Victoria/361/2011) were added to duplicate wells of a fetuin-coated 96-well plate. This mixture was incubated overnight, and the next day peroxidase-conjugated PNA was added to the washed plate followed by steps in which color is developed. The absence of color indicates inhibition of NA activity due to the presence of NA-specific inhibiting antibodies. The titer was assigned as the reciprocal of the last dilution with an OD equal to or less than the midpoint between the mean OD of the virus-only control wells and the mean OD of the background wells on each plate.

Statistical methods.

The immunogenicity of the vaccines was assessed by calculating geometric mean titers (GMTs) for each assay and treatment group, together with GMT ratios and 95% two-sided confidence intervals based on the t distribution.

The correlates of protection analysis assessed the five assays: HAI assay for antibody to the A/Victoria/361/2011 (H3N2) vaccine virus, HAI assay for antibody to the A/Victoria/361/2011 (H3N2) circulating virus, NT against the A/Victoria/361/2011 (H3N2) vaccine virus, NT against the A/Victoria/361/2011 (H3N2) circulating virus, and ELLA for antibody to neuraminidase N2. The analysis investigated protection against six case definitions: laboratory-confirmed influenza associated with an RI, laboratory-confirmed influenza associated with a PD-ILI, laboratory-confirmed influenza associated with a CDC-ILI, antigenically similar influenza associated with an RI, antigenically similar influenza associated with a PD-ILI, and antigenically similar influenza associated with a CDC-ILI. The correlates of protection statistical analysis broadly followed the framework of Qin and Gilbert (25, 26).

Logistic regression was first used to confirm that rates of influenza illness decreased with increasing titer, i.e., that each assay could be considered a correlate of risk as defined in the framework for each case definition. To compare the strength of the associations between assays and case definitions, log-transformed titers were centered and standardized by subtracting the mean and dividing by the standard deviation before being entered into the logistic regression.

The surrogacy value of each assay—the degree to which it might substitute for the observation of clinical illness—was evaluated by two similar methods. The Prentice criterion for a surrogate endpoint, which posits that the probability of disease at a given titer should be independent of treatment group (27), was tested by multiple logistic regression. The method for determining the proportion of treatment effect explained (28) was used to quantify the extent to which assay titer explained the observed difference in rates of influenza illness between the two treatment groups, IIV-HD and IIV-SD.

Protection curves estimating the level of protection at each assay titer were calculated using the scaled logit model and extensions, which incorporate both protection related to assay titer and exposure (29, 30). Goodness-of-fit of models were assessed by the method of Hosmer and Lemeshow (31); models with goodness of fit less than 0.5 were considered unreliable and results were not reported. From the protection curve for each assay and case definition, titers for 50% and 80% protection were estimated, and 95% confidence intervals (CIs) were calculated by bootstrapping. The proportion of subjects with titers falling within the 95% CI of the titer for 50% protection was calculated, which provides a scale-free measure of the precision of the estimate and may be regarded as a measure of the utility of the estimate. Relative vaccine efficacy predicted by the models was compared to the efficacy observed.

All statistical methods took into account the case-cohort design for the relevant assays; results are reported for the full (intent-to-treat) analysis set.

RESULTS

Geometric mean titers are shown in Table 1. All titers were consistently higher for IIV-HD than for IIV-SD, with GMT ratios ranging between 1.42 and 1.96. HAI titers against the year 2 A/H3N2 vaccine virus were higher than against the A/H3N2 circulating virus, though for the latter the titer induced by IIV-HD remained significantly higher than that induced by IIV-SD. Unexpectedly, the assayed NT titer against the vaccine virus was lower than against the circulating virus.

TABLE 1.

Postvaccination geometric mean titers

Yr, assay, strain, virus type Value(s) for each treatment group
GMT ratio (95% CI)
IIV-HD
IIV-SD
N GMT (95% CI) N GMT (95% CI)
HAI assays
    2011–2012, HAI, A/California/7/2009 (H1N1), vaccine 2,375 481.75 (457.69, 507.07) 2,382 271.84 (257.40, 287.10) 1.77 (1.64, 1.91)
    2011–2012, HAI, A/Victoria/210/2009 (H3N2), vaccine 2,375 685.53 (651.43, 721.42) 2,382 349.85 (332.09, 368.57) 1.96 (1.82, 2.11)
    2011–2012, HAI, B/Brisbane/60/2008, vaccine 2,375 138.07 (132.16, 144.23) 2,382 97.55 (93.26, 102.03) 1.42 (1.33, 1.51)
    2012–2013, HAI, A/California/7/2009 (H1N1), vaccine 2,879 406.95 (390.17, 424.45) 2,872 227.40 (216.85, 238.46) 1.79 (1.68, 1.91)
    2012–2013, HAI, A/Victoria/361/2011 (H3N2), vaccine 2,879 459.97 (440.80, 479.96) 2,872 252.78 (241.64, 264.43) 1.82 (1.71, 1.94)
    2012–2013, HAI, B/Texas/6/2011, vaccine 2,879 98.20 (94.52, 102.04) 2,872 61.79 (59.44, 64.22) 1.59 (1.51, 1.68)
    2012–2013, HAI, A/Victoria/361/2011 (H3N2), circulating 317 49.60 (44.35, 55.47) 356 33.59 (30.04, 37.56) 1.48 (1.26, 1.73)
NT assays
    2012–2013, NT, A/Victoria/361/2011 (H3N2), vaccine 318 218.3 (193.6; 246.0) 357 142.3 (126.1; 160.5) 1.53 (1.29; 1.82)
    2012–2013, NT, A/Victoria/361/2011 (H3N2), circulating 318 506.7 (441.2; 581.9) 357 288.9 (249.3; 334.9) 1.75 (1.43; 2.15)
ELLA
    2012–2013, N2 ELLA, N2 from A/Victoria/361/2011 318 54.55 (49.11; 60.59) 357 38.34 (34.49; 42.62) 1.42 (1.23; 1.65)

For all five assays included in the analysis of correlates of protection, increasing titer was strongly associated with reduction in occurrence of A/H3N2 influenza illness for all six case definitions, as shown in Table S1 in the supplemental material. The HAI assay using the circulating virus was consistently more strongly associated with influenza illness than the assay using the vaccine virus. Similarly for the neutralization assays, the assay using the circulating virus was consistently more strongly associated with influenza illness than the assay using the vaccine virus. The neuraminidase assay was strongly associated with the three antigenically similar case definitions.

The assessment of surrogacy value of assays by the Prentice criterion found that for none of the 30 combinations of assay and case definition did the P value for the effect of treatment on clinical illness reach statistical significance when assay titer was also included in the model (two-sided P value range, 0.12 to 0.98), demonstrating for each assay that titer was an acceptable surrogate for treatment for the purpose of predicting influenza illness and justifying pooling across treatment groups in the assessment of correlates of protection.

The surrogacy value of each assay was also assessed by the method for determining the proportion of treatment effect explained; the results are shown in Table 2. (Note that the proportion of treatment effect on clinical illness explained by assay titer can sometimes exceed 1, because titer is a continuous variable with more ability to explain outcome than treatment group, a dichotomous variable.) As might be expected, the HAI and NT assays using the circulating virus better explained the effect of treatment on influenza illness than the corresponding assays using the vaccine virus. The neuraminidase assay was somewhat less explanatory of treatment effect than the HAI and NT assays.

TABLE 2.

Proportion of treatment effect on A/H3N2 influenza illness occurrence explained by assays

Case definitiona (n) Proportion of treatment effect explained by assay
HAI, vaccine virus (n = 5,752) HAI, circulating virus (n = 673) NT, vaccine virus (n = 675) NT, circulating virus (n = 675) N2 ELLA (n = 675)
Laboratory-confirmed influenza associated with RI (n = 152/123) 0.82 0.97 0.85 1.32 0.58
Laboratory-confirmed influenza associated with PD-ILI (n = 121/98) 0.95 1.13 1.12 1.81 0.75
Laboratory-confirmed influenza associated with CDC-ILI (n = 52/48) 0.86 0.65 0.75 0.99 0.51
Antigenically similar influenza associated with RI (n = 37/34) 0.48 0.47 0.32 0.49 0.53
Antigenically similar influenza associated with PD-ILI (n = 30/28) 0.50 0.65 0.41 0.75 0.65
Antigenically similar influenza associated with CDC-ILI (n = 18/17) 0.27 0.27 0.17 0.32 0.28
a

n indicates the number of cases. The first number is for the HAI vaccine virus assay, the second is for the other assays.

The extent to which pairs of assays taken together better explained the effect of treatment on influenza illness than each assay taken separately was assessed. The proportion of treatment effect explained by pairs of assays compared to the greater of the proportions explained by each singly, as well as the increase, is shown in Table S2 in the supplemental material. For the pair of HAI (circulating virus) and NT (circulating virus), the proportion explained by the pair together was slightly less than the proportion explained by either assay individually. For the pairs which included the N2 ELLA, however, the proportion explained was in every case greater than the proportion explained by either individually for all case definitions. The increase in proportion explained was greatest for the pairs of HAI (vaccine virus) and N2 ELLA and of NT (vaccine virus) and N2 ELLA but was also substantial for the pairs of HAI (circulating virus) and N2 ELLA and of NT (circulating virus) and N2 ELLA.

The estimated protection curves for the HAI assay using the circulating virus against the laboratory-confirmed influenza illness case definitions are shown in Fig. 1. Estimates of the titer at which protection was 50% for all assays and case definitions are shown in Table 3.

FIG 1.

FIG 1

Protection curves for the A/Victoria/361/2011 HAI assay using the circulating virus against A/H3N2 illness by three laboratory-confirmed influenza (LCI) case definitions (defn.), showing titers for 50% and 80% protection, with 95% CIs.

TABLE 3.

Estimated titers for 50% protection against A/H3N2 influenza illness

Case definitiona (n) Titer for 50% protectionb (95% CI)
HAI, vaccine virus (n = 5,752) HAI, circulating virus (n = 673) NT, vaccine virus (n = 675) NT, circulating virus (n = 675) N2 ELLA (n = 675)
Laboratory-confirmed influenza associated with RI (n = 152/123) 359 (95.8, 574) 44.7 (17.4, 62.1) 180 (17.2, 368) 93.0 (17.1, 184)
Laboratory-confirmed influenza associated with PD-ILI (n = 121/98) 285 (69.6, 500) 40.6 (10.5, 58.7) 141 (68.9, 238) 16.8 (11.3, 115)
Laboratory-confirmed influenza associated with CDC-ILI (n = 52/48) 265 (131, 408) 23.7 (7.4, 52.7) 219 (82.9, 365) 16.8 (9.0, 103)
Antigenically similar influenza associated with RI (n = 37/34) 437 (99.7, 1347) 36.2 (12.3, 93.0) 47.7 (25.8, 1163) 34.8 (18.7, 38.8)
Antigenically similar influenza associated with PD-ILI (n = 30/28) 399 (83.2, 1273) 22.4 (9.0, 44.8) 47.3 (21.7, 696) 33.5 (14.3, 49.4)
Antigenically similar influenza associated with CDC-ILI (n = 18/17) 203 (97.8, 302) 23.7 (12.0, 52.7) 196 (60.2, 257) 17.6 (10.3, 27.3)
a

n indicates the number of cases. The first number is for the HAI vaccine virus assay, the second is for the other assays.

b

—, goodness of fit of <0.5.

For the HAI assay using the circulating virus, titers for 50% protection ranged from 22.4 to 44.7 for the different case definitions. For the HAI assay using the vaccine virus, titers for 50% protection ranged from 203 to 437.

Models with adequate goodness of fit were found for only 6 of the 12 combinations of neutralization test and case definition, and there was variability in the titers for 50% protection. There was less variability in the point estimates for 50% protection for the neuraminidase assay, though one case definition—laboratory-confirmed influenza associated with an RI—was an outlier.

Estimates of titers for 80% protection are shown in Table S3 in the supplemental material. The results broadly parallel those for 50% protection. For the HAI assay using the circulating virus, titers for 80% protection ranged from 27.5 to 83.3; for the HAI assay using the vaccine virus, the lowest titer for 80% protection was 271, and for two case definitions 80% protection was not achieved even at the highest titers. Similarly for the NT assay using the vaccine virus, for the two models with acceptable fit, 80% protection was not achieved; for the NT assay using the circulating virus and with acceptable fit, titers for 80% protection ranged from 151 to 221. For the N2 ELLA, titers for 80% protection were only slightly greater than those for 50% protection for three case definitions, suggesting rapidly increasing protection at these values.

The percentage of subjects with titers within the 95% CI for 50% protection is shown in Table S4 in the supplemental material, with a smaller percentage indicating better precision and greater utility of the estimate. In general, the proportion of subjects with titers falling within the 95% CI was large; for more than half the 24 combinations of assay and case definition with acceptable goodness of fit, the percentage of subjects falling within the 95% CI exceeded 50%. For two combinations, however—N2 ELLA and antigenically similar influenza associated with RI, as well as N2 ELLA and antigenically similar influenza associated with CDC-ILI—the percentage was less than 20%.

The vaccine efficacy of IIV-HD relative to that of IIV-SD, predicted by the estimated protection curves and compared with the efficacy observed among the subjects on whose data the estimates were based, is shown in Table S5 in the supplemental material. For 11 out of the 24 comparisons, predicted efficacy was within ±5 percentage points of observed efficacy; however, for three comparisons the difference exceeded 20 percentage points. In 16 instances the protection curve underestimated observed efficacy, and in 8 instances it overestimated observed efficacy.

DISCUSSION

The improved immune response to IIV-HD relative to IIV-SD observed was consistent with previous findings both for HAI (9, 11, 32) and for neuraminidase (33); higher titers were also seen in this study for NT, with GMT ratios comparable to those seen for HAI.

Overall, all assays were strongly associated with reduction in influenza illness. In the aggregate, the strength of association for the NT assays closely approached the values seen for the HAI assays, and the ELLA was comparable with the HAI and NT vaccine strain assays in this respect.

All assays also had good surrogacy value. In terms of proportion of treatment effect explained, the NT assay could be said to have performed at least as well as the HAI assay in the results seen here. Prediction of observed efficacy was no worse for the ELLA than for the HAI assay. These results have particular relevance for the NT assay and ELLA, which are not referenced nearly as frequently as the HAI assay, and suggest they could potentially have utility similar to that of HAI for influenza vaccine correlates of protection, particularly with further refinement and additional, consistent quantification of protective titers.

An important result is that the estimated HAI titers for 50% protection in the elderly are closely similar to those that have been found for younger adults when the assay virus matches the circulating virus. The range of 22.4 to 44.7 found here may be compared with the range of 15 to 65 found by de Jong et al. (13), with the estimates of 17 and 29 in Coudeville et al. (14), and with the values of 15, 30, 32, 40, and 64 from various studies referenced by Plotkin (34). The results suggest that the generally accepted protective titer of 40 may reasonably be applied to the elderly as it is to younger adults.

Dramatically higher HAI titers were required to achieve the same level of protection when the assay virus was poorly matched to the circulating virus, and this was seen across the whole range of titers and levels of protection. At high titers, close to 100% protection was readily achieved against the matched virus, whereas 80% protection was sometimes not achieved even at the highest titers (up to 10,240) against the mismatched vaccine virus. Titers between 203 and 437 were required for 50% protection against mismatched virus. At low titers some measure of protection was seen against matched virus (similar findings have been reported elsewhere [14, 35, 36]) but none against mismatched virus (data not shown).

It might intuitively have been expected that markedly higher antibody titers would be required to protect against a virus mismatched to these antibodies, as was seen in these results. This could be the case, for instance, because higher titers may offset the lesser avidity of mismatched antibodies for the virus and thus may achieve the moderate levels of protection engendered by lower titers of matched antibody. However, somewhat surprisingly, the same pattern was observed for case definitions restricted to illnesses classified as antigenically similar to the A/H3N2 vaccine component, for which similar protective thresholds were expected for assays using the vaccine virus and the circulating virus. This may have resulted from limitations in the assay used for classifying the isolated viruses as antigenically similar, in the assay used for measuring the HAI responses, or both.

Overall, the results presented here suggest that HAI titers against circulating influenza viruses are useful predictors of influenza risk/protection in the elderly at thresholds similar to those previously described for younger populations. However, when the antigens used in HAI assays do not match the antigens of circulating influenza viruses, the HAI titer has limited value as a predictor of influenza risk/protection. This may be the case because the threshold titer associated with a level of protection is likely to vary with the antigenic distance between the virus/antigen used in the assay and the antigen contained in the circulating virus. As this antigenic distance increases, it would be expected that the thresholds for protection will also increase (with increasingly higher titers required for any given level of protection). Multiple studies would be required to establish the yet unknown correlation of the antigenic distance between assay and circulating viruses and the protective thresholds. Until then, the protective thresholds of titers obtained using assays not matching the circulating virus will be of limited value and will have questionable external validity.

The unexpectedly lower NT titers seen against the vaccine virus may have been due to a greater quantity of noninfectious particles in the vaccine virus preparation used in the assay than in the circulating virus preparation used, with the former having been sourced from the vaccine production process while the latter was laboratory prepared. Alternatively or additionally, the circulating virus, having been propagated in MDCK cells, may have been better adapted to infect the MDCK cells utilized in the assay.

The complementary action of neuraminidase antibody with the other assays was seen in the proportion of treatment explained analyses. Whereas the HAI and NT circulating virus assays taken together did not add to the proportion of treatment effect explained, all four combinations of the N2 ELLA with one of the HAI or NT vaccine or circulating virus assays improved the proportion explained above the value for each assay separately. This indicates that the ELLA is providing additional information about the vaccine effect beyond that provided by the HAI and NT assays and vice versa. This may be because the N2 ELLA is—at least partially—measuring antibodies with a different mechanism of action than that of the antibodies measured by the HAI and NT assays. Anti-NA antibodies are thought to prevent the release of newly formed viruses from an infected host cell, whereas anti-HA antibodies are thought to prevent viral attachment to the host cell (37). In addition, this finding suggests that immunity against NA may have added value for protection against influenza to immunity against HA. The role of anti-NA antibodies in younger populations had been described previously by other investigators (19, 36, 38, 39) and was seen here in older adults as well. This may have implications for the design of influenza vaccines, as exclusion of the NA antigen would preclude the induction of anti-NA immune responses that may complement the anti-HA responses in protecting individuals against influenza illness or modifying its severity (40, 41).

An individual assay readout may not only reflect the specific type of antibodies implied in the assay-related physiology and corresponding antibody mechanism of action but may also reflect other parallel mechanisms of action. As such, the surrogacy found for the N2 ELLA, for instance, is expected to be reflecting the anti-NA antibody effect as well as the anti-HA antibody to some extent, as individuals with good anti-NA responses may also have good anti-HA responses. The immune system operates through multiple mechanisms of action, and the extent to which a single assay captures only a single, unique immunological property or alternatively reflects multiple aspects of immunological protection, and the extent to which these lie in the causal pathway mediating protection or alternatively reflect differing degrees of robustness of the immune system in different individuals, is uncertain.

Despite the role that HAI, NT, and ELLA titers may have in the evaluation of influenza vaccines or influenza risk, the proportions of subjects falling within the 95% confidence intervals for 50% protection show that in general, the estimates did not reach a level of precision which would justify their use as a reliable surrogate for observation of clinical illness: for the majority of combinations of case definitions and assays evaluated, less than 50% of subjects could be reliably classified as protected or susceptible based on their individual postvaccination titers. This underlines a more general limitation of data from vaccine efficacy trials to investigate correlates of protection, namely, insufficient power. Even when samples for assaying are taken from all subjects, the sample size required for a reasonably precise estimate of a single variable, vaccine efficacy, provides insufficient information for an adequately precise estimate of the relationship between two variables, titer and illness. In other unpublished research, two of the authors of this paper have estimated that 1,000 to 2,000 cases might be required for a reasonably precise estimate of an influenza correlate of protection, compared to the 17 to 152 cases in these analyses (as shown in Table 2). In addition, inadequate power can result in random variability in the data, yielding spurious results. The small proportion of treatment effect explained for the antigenically similar influenza associated with CDC-ILI case definition would appear to be due to the small number of cases, 17, seen for this case definition and the possibly spuriously high treatment effect observed for this case definition among assayed subjects, equivalent to relative efficacy of 71.5% (see Table S5 in the supplemental material). Inadequate power means that results of statistical analysis can only be interpreted as exploratory rather than confirmatory.

Natural variability between individuals means that at any given titer some will be protected and some not, and if protection does in fact increase with titer, the proportion protected will increase in a smooth, continuous manner. Thus, protection curves are more likely to represent the underlying relationship between assay and protection than a single-valued protective titer.

A further limitation of the methods is illustrated by the imperfect prediction of observed vaccine efficacy. Although no assays were demonstrated to have failed to meet the Prentice criterion—statistically significant differences between treatment groups were not seen when assay was included in the model—the pooling across groups masks small differences between them. Models with separate protection curves for each group would precisely predict vaccine efficacy but would require an additional parameter, further reducing precision. On the other hand, assays using mismatched viruses were not dramatically worse at predicting efficacy than other assays, suggesting that the ability to predict vaccine efficacy may be principally a characteristic of the statistical models used rather than of the assays modeled.

An additional limitation is the generalizability of the results to other settings. The results are specific to the influenza subtype (A/H3N2), the particular strain (A/Victoria/361/2011), and the assay protocols. Without confirmation, the results may not be generalizable to other subtypes, strains, and protocols. On the other hand, the study subjects could likely be considered generally representative of the population of U.S. persons ≥65 years of age with respect to their demographic characteristics, health status, immunological history, and source of infection.

In conclusion, the study suggests the utility of HAI and NT assays and ELLA as potential correlates of influenza vaccine protection in older adults, with likely added value obtained when two or more assays targeting distinct physiologic mechanisms are combined. Protective thresholds for the HAI assay in the elderly appear consistent with those previously described for younger adults, provided the assay virus matches the circulating virus. The thresholds estimated for protection are of suboptimal precision, limiting their utility for confidently classifying specific individuals as at risk for or protected against influenza. As the study was particular to a single influenza season, a single influenza subtype, and a single influenza strain, it is unknown whether the results can be generalized. Further studies would allow the confirmation of the role of these assays when used singly or in combination as correlates of protection against influenza in the elderly population.

Supplementary Material

Supplemental material

ACKNOWLEDGMENTS

We thank the study participants, the investigators from the 126 participating research sites, and the sponsor's study team for their contributions to the original study. We are also grateful to three anonymous reviewers for their comments on the draft manuscript, which significantly improved the work.

C.A.D., T.V., B.H., and V.A.L. currently are, and A.J.D. was at the time of this research, employees of Sanofi Pasteur. H.K.T. has received research funding from MedImmune, Gilead, and Sanofi Pasteur and has served as an advisor for VaxInnate and Novartis.

Funding Statement

This work was supported by Sanofi Pasteur.

Footnotes

Supplemental material for this article may be found at http://dx.doi.org/10.1128/CVI.00604-15.

REFERENCES

  • 1.Zhou H, Thompson WW, Viboud CG, Ringholz CM, Cheng PY, Steiner C, Abedi GR, Anderson LJ, Brammer L, Shay DK. 2012. Hospitalizations associated with influenza and respiratory syncytial virus in the United States, 1993-2008. Clin Infect Dis 54:1427–1436. doi: 10.1093/cid/cis211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Quandelacy TM, Viboud C, Charu V, Lipsitch M, Goldstein E. 2014. Age- and sex-related risk factors for influenza-associated mortality in the United States between 1997-2007. Am J Epidemiol 179:156–167. doi: 10.1093/aje/kwt235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Haq K, McElhaney JE. 2014. Immunosenescence: influenza vaccination and the elderly. Curr Opin Immunol 29:38–42. doi: 10.1016/j.coi.2014.03.008. [DOI] [PubMed] [Google Scholar]
  • 4.McElhaney JE, Beran J, Devaster JM, Esen M, Launay O, Leroux-Roels G, Ruiz-Palacios GM, van Essen GA, Caplanusi A, Claeys C, Durand C, Duval X, El Idrissi M, Falsey AR, Feldman G, Frey SE, Galtier F, Hwang SJ, Innis BL, Kovac M, Kremsner P, McNeil S, Nowakowski A, Richardus JH, Trofa A, Oostvogels L; for the Influence65 Study Group . 2013. AS03-adjuvanted versus non-adjuvanted inactivated trivalent influenza vaccine against seasonal influenza in elderly people: a phase 3 randomised trial. Lancet Infect Dis 13:485–496. doi: 10.1016/S1473-3099(13)70046-X. [DOI] [PubMed] [Google Scholar]
  • 5.DiazGranados CA, Dunning AJ, Kimmel M, Kirby D, Treanor J, Collins A, Pollak R, Christoff J, Earl J, Landolfi V, Martin E, Gurunathan S, Nathan R, Greenberg DP, Tornieporth NG, Decker MD, Talbot HK. 2014. Relative efficacy of high-dose trivalent influenza vaccine compared to standard-dose vaccine in adults 65 years of age and older. N Engl J Med 371:635–645. doi: 10.1056/NEJMoa1315727. [DOI] [PubMed] [Google Scholar]
  • 6.Plotkin SA, Orenstein WA, Offit PA (ed). 2013. Vaccines, 6th ed W.B. Saunders, London, United Kingdom. [Google Scholar]
  • 7.Sasaki S, Sullivan M, Narvaez CF, Holmes TH, Furman D, Zheng NY, Nishtala M, Wrammert J, Smith K, James JA, Dekker CL, Davis MM, Wilson PC, Greenberg HB, He XS. 2011. Limited efficacy of inactivated influenza vaccine in elderly individuals is associated with decreased production of vaccine-specific antibodies. J Clin Investig 121:3109–3119. doi: 10.1172/JCI57834. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Goodwin K, Viboud C, Simonsen L. 2006. Antibody response to influenza vaccination in the elderly: a quantitative review. Vaccine 24:1159–1169. doi: 10.1016/j.vaccine.2005.08.105. [DOI] [PubMed] [Google Scholar]
  • 9.Falsey AR, Treanor JJ, Tornieporth N, Capellan J, Gorse GJ. 2009. Randomized, double-blind controlled phase 3 trial comparing the immunogenicity of high-dose and standard-dose influenza vaccine in adults 65 years of age and older. J Infect Dis 200:172–180. doi: 10.1086/599790. [DOI] [PubMed] [Google Scholar]
  • 10.Frey SE, Reyes MR, Reynales H, Bermal NN, Nicolay U, Narasimhan V, Forleo-Neto E, Arora AK. 2014. Comparison of the safety and immunogenicity of an MF59-adjuvanted with a non-adjuvanted seasonal influenza vaccine in elderly subjects. Vaccine 32:5027–5034. doi: 10.1016/j.vaccine.2014.07.013. [DOI] [PubMed] [Google Scholar]
  • 11.Tsang P, Gorse GJ, Strout CB, Sperling M, Greenberg DP, Ozol-Godfrey A, DiazGranados C, Landolfi V. 2014. Immunogenicity and safety of Fluzone intradermal and high-dose influenza vaccines in older adults ≥65 years of age: a randomized, controlled, phase II trial. Vaccine 32:2507–2517. doi: 10.1016/j.vaccine.2013.09.074. [DOI] [PubMed] [Google Scholar]
  • 12.Mannino S, Villa M, Apolone G, Weiss NS, Groth N, Aquino I, Boldori L, Caramaschi F, Gattinoni A, Malchiodi G, Rothman KJ. 2012. Effectiveness of adjuvanted influenza vaccination in elderly subjects in northern Italy. Am J Epidemiol 176:527–533. doi: 10.1093/aje/kws313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.de Jong JC, Palache AM, Beyer WE, Rimmelzwaan GF, Boon AC, Osterhaus AD. 2003. Haemagglutination-inhibiting antibody to influenza virus. Dev Biol (Basel) 115:63–73. [PubMed] [Google Scholar]
  • 14.Coudeville L, Bailleux F, Riche B, Megas F, André P, Ecochard R. 2010. Relationship between haemagglutination-inhibiting antibody titres and clinical protection against influenza: development and application of a Bayesian random-effects model. BMC Med Res Methodol 10:18. doi: 10.1186/1471-2288-10-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Hobson D, Curry RL, Beare AS, Ward-Gardner A. 1972. The role of serum haemagglutination-inhibiting antibody in protection against challenge infection with influenza A2 and B viruses. J Hyg (Lond) 70:767–777. doi: 10.1017/S0022172400022610. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Black S, Nicolay U, Vesikari T, Knuf M, Del Giudice G, Della Cioppa G, Tsai T, Clemens R, Rappuoli R. 2011. Hemagglutination inhibition antibody titers as a correlate of protection for inactivated influenza vaccines in children. Pediatr Infect Dis J 30:1081–1085. doi: 10.1097/INF.0b013e3182367662. [DOI] [PubMed] [Google Scholar]
  • 17.Reber A, Katz J. 2013. Immunological assessment of influenza vaccines and immune correlates of protection. Expert Rev Vaccines 12:519–536. doi: 10.1586/erv.13.35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Cheng LW, Huang SW, Huang LM, Chang LY, Shao PL, Kiang D, Wang JR. 2012. Comparison of neutralizing and hemagglutination-inhibiting antibody responses for evaluating the seasonal influenza vaccine. J Virol Methods 182:43–49. doi: 10.1016/j.jviromet.2012.03.004. [DOI] [PubMed] [Google Scholar]
  • 19.Couch RB, Atmar RL, Franco LM, Quarles JM, Wells J, Arden N, Niño D, Belmont JW. 2013. Antibody correlates and predictors of immunity to naturally occurring influenza in humans and the importance of antibody to the neuraminidase. J Infect Dis 207:974–981. doi: 10.1093/infdis/jis935. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Centers for Disease Control and Prevention. 2013. Influenza activity–United States, 2012-13 season and composition of the 2013-14 influenza vaccine. MMWR Morb Mortal Wkly Rep 62:473–479. [PMC free article] [PubMed] [Google Scholar]
  • 21.World Health Organization. 2013. Recommended composition of influenza virus vaccines for use in the 2013-2014 northern hemisphere influenza season. Wkly Epidemiol Rec 88:101–114. [PubMed] [Google Scholar]
  • 22.Prentice RL. 1986. A case-cohort design for epidemiologic cohort studies and disease prevention trials. Biometrika 73:1–11. doi: 10.1093/biomet/73.1.1. [DOI] [Google Scholar]
  • 23.Lambre CR, Terzidis H, Greffard A, Webster RG. 1990. Measurement of anti-influenza neuraminidase antibody using a peroxidase-linked lectin and microtiter plates coated with natural substrates. J Immunol Methods 135:49–57. doi: 10.1016/0022-1759(90)90255-T. [DOI] [PubMed] [Google Scholar]
  • 24.Couzens L, Gao J, Westgeest K, Sandbulte M, Lugovtsev V, Fouchier R, Eichelberger M. 2014. An optimized enzyme-linked lectin assay to measure influenza A virus neuraminidase inhibition antibody titers in human sera. J Virol Methods 210C:7–14. [DOI] [PubMed] [Google Scholar]
  • 25.Qin L, Gilbert PB, Corey L, McElrath MJ, Self SG. 2007. A framework for assessing immunological correlates of protection in vaccine trials. J Infect Dis 196:1304–1312. doi: 10.1086/522428. [DOI] [PubMed] [Google Scholar]
  • 26.Gilbert PB, Qin L, Self SG. 2008. Evaluating a surrogate endpoint at three levels, with application to vaccine development. Stat Med 27:4758–4778. doi: 10.1002/sim.3122. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Prentice RL. 1989. Surrogate endpoints in clinical trials: definition and operational criteria. Stat Med 8:431–440. doi: 10.1002/sim.4780080407. [DOI] [PubMed] [Google Scholar]
  • 28.Lin DY, Fleming TR, De Gruttola V. 1997. Estimating the proportion of treatment effect explained by a surrogate marker. Stat Med 16:1515–1527. [DOI] [PubMed] [Google Scholar]
  • 29.Dunning AJ. 2006. A model for immunological correlates of protection. Stat Med 25:1485–1497. doi: 10.1002/sim.2282. [DOI] [PubMed] [Google Scholar]
  • 30.Dunning AJ, Kensler J, Coudeville L, Bailleux F. 2015. Some extensions in continuous models for immunological correlates of protection. BMC Med Res Methodol 15:107. doi: 10.1186/s12874-015-0096-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Hosmer DW, Lemeshow S. 2000. Applied logistic regression. Wiley Interscience, Hoboken, NJ. [Google Scholar]
  • 32.DiazGranados CA, Dunning AJ, Jordanov E, Landolfi V, Denis M, Talbot HK. 2013. High-dose trivalent influenza vaccine compared to standard dose vaccine in elderly adults: safety, immunogenicity and relative efficacy during the 2009-2010 season. Vaccine 31:861–866. doi: 10.1016/j.vaccine.2012.12.013. [DOI] [PubMed] [Google Scholar]
  • 33.Cate TR, Rayford Y, Niño D, Winokur P, Brady R, Belshe R, Chen W, Atmar RL, Couch RB. 2010. A high dosage influenza vaccine induced significantly more neuraminidase antibody than standard vaccine among elderly subjects. Vaccine 28:2076–2079. doi: 10.1016/j.vaccine.2009.12.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Plotkin SA. 2013. Complex correlates of protection after vaccination. Clin Infect Dis 56:1458–1465. doi: 10.1093/cid/cit048. [DOI] [PubMed] [Google Scholar]
  • 35.Gentile D, Doyle W, Whiteside T, Fireman P, Hayden FG, Skoner D. 1998. Increased interleukin-6 levels in nasal lavage samples following experimental influenza A virus infection. Clin Diagn Lab Immunol 5:604–608. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Memoli MJ, Czajkowski L, Reed S, Athota R, Bristol T, Proudfoot K, Fargis S, Stein M, Dunfee RL, Shaw PA, Davey RT, Taubenberger JK. 2015. Validation of the wild-type influenza A human challenge model H1N1pdMIST: an A(H1N1)pdm09 dose-finding investigational new drug study. Clin Infect Dis 60:693–702. doi: 10.1093/cid/ciu924. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Lambert LC, Fauci AS. 2010. Influenza vaccines for the future. N Engl J Med 363:2036–2044. doi: 10.1056/NEJMra1002842. [DOI] [PubMed] [Google Scholar]
  • 38.Monto AS, Petrie JG, Cross RT, Johnson E, Liu M, Zhong W, Levine M, Katz JM, Ohmit SE. 2015. Antibody to influenza virus neuraminidase: an independent correlate of protection. J Infect Dis 212:1191–1199. doi: 10.1093/infdis/jiv195. [DOI] [PubMed] [Google Scholar]
  • 39.Murphy BR, Kasel JA, Chanock RM. 1972. Association of serum anti-neuraminidase antibody with resistance to influenza in man. N Engl J Med 286:1329–1332. doi: 10.1056/NEJM197206222862502. [DOI] [PubMed] [Google Scholar]
  • 40.Johansson BE, Pokorny BA, Tiso VA. 2002. Supplementation of conventional trivalent influenza vaccine with purified viral N1 and N2 neuraminidases induces a balanced immune response without antigenic competition. Vaccine 20:1670–1674. doi: 10.1016/S0264-410X(01)00490-X. [DOI] [PubMed] [Google Scholar]
  • 41.Bosch BJ, Bodewes R, de Vries RP, Kreijtz JH, Bartelink W, van Amerongen G, Rimmelzwaan GF, de Haan CA, Osterhaus AD, Rottier PJ. 2010. Recombinant soluble, multimeric HA and NA exhibit distinctive types of protection against pandemic swine-origin 2009 A(H1N1) influenza virus infection in ferrets. J Virol 84:10366–10374. doi: 10.1128/JVI.01035-10. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental material

Articles from Clinical and Vaccine Immunology : CVI are provided here courtesy of American Society for Microbiology (ASM)

RESOURCES