Evaluating Hemagglutination Inhibition Antibody Titers as a Correlate of Protection for Influenza: A Sensitivity Analysis Based on Information Theory and Causal Inference

Abstract

Introduction:

Identifying hemagglutination inhibition (HI) antibody titers as a key immune correlate of protection (CoP) is crucial for developing, licensing, and monitoring the ongoing effectiveness of new influenza vaccines. Using a new statistical methodology, we explored the link between an inactivated quadrivalent influenza vaccine’s impact on HI antibody titers and its effectiveness against A/H1N1-associated influenza illness.

Methods:

We utilized data from a phase 3, observer-blind, randomized, controlled trial in children aged 6–35 months to assess HI antibody titers as an immune CoP. The assessment used a statistical method developed within a causal inference framework and a new information-theoretic metric of surrogacy, the so-called individual causal association (ICA).

Results:

The 75% and 85% uncertainty intervals of the ICA are 0.5511–0.8282 and 0.3632–0.8684, respectively, indicating a substantial reduction in the uncertainty about the vaccine’s effect on the absence of infection when its impact on the HI antibody titers is known.

Conclusions:

The evaluation yielded evidence supporting the validity of HI antibody titers as a CoP for influenza infection.

INTRODUCTION

Assessing a vaccine’s impact on a clinically relevant outcome, the so-called true endpoint (e.g., the rate of infection, hospitalization, intensive care admission, mortality, etc.), is crucial. However, current vaccine studies prioritize identifying immunological markers that can predict the vaccine’s effect on the true endpoint. These markers have the potential to serve as substitute endpoints, reducing study duration and costs and enhancing overall evaluation efficiency.[1,2,3]

Qin et al. introduced a framework to define immune correlates: (1) a correlate of risk is an immunological measurement associated with the clinical endpoint of protection; (2) a Level 1 surrogate of protection (SoP) predicts vaccine efficacy (VE) in similar future trials, further subdivided into statistical and principal SoP; and (3) a Level 2 SoP predicts VE across diverse trials and populations.[1]

Plotkin defined a surrogate as an immunological measurement that is indirectly protective, substituting the true correlate when it is unknown or challenging to measure.[4,5,6] He used the term “correlate” for an immune response closely related to protection, either mechanistically or not. Plotkin’s definition aligns with Qin et al.’s interpretation of a surrogate.[1,4,5,6] To streamline terminology, this work uses both “surrogate” and “correlate of protection (CoP)” interchangeably.

Several statistical methodologies for evaluating surrogate endpoints have been extensively discussed in the clinical trial and epidemiological literature, especially in the context of vaccine trials. In 1989, Prentice introduced a definition with operational criteria for identifying surrogate endpoints through hypothesis testing.[7] The validity of a statistical SoP defined by Qin et al. is demonstrated based on Prentice’s criteria.[1,7] Within vaccine studies, one of Prentice’s criteria specifies that immune responses must effectively capture any association between vaccination and protection, presenting a somewhat stringent condition. Proving this is challenging, leading to the proposal of the vaccine’s protective effect explained by immune markers or, generally, the proportion of treatment explained (PTE) by the surrogate.[8] Despite its intuitive appeal, the PTE faces fundamental criticisms alongside the original definition and criteria.[9,10]

An alternative approach to validate a Level 1 SoP is to demonstrate that it qualifies as a principal surrogate endpoint.[11,12] In this framework, the causal effect of vaccination on the absence of infection should only exist when there is a causal effect of vaccination on the immunological marker. Ultimately, proving that a correlate is a second-level SoP involves evaluation in a multiple-trial setting, enabling the prediction of VE across diverse populations and trials.[13] While this is the ultimate goal in immune correlate evaluation, its feasibility may be hindered by the need for a large volume of data.

A novel surrogacy definition and quantification, rooted in information theory and causal inference, have been proposed for the single-trial setting, where both surrogate and true endpoint information are available from a single trial.[14] The introduced metric of surrogacy in this framework, the individual causal association (ICA), quantifies the association between individual causal treatment effects (ICTEs) on the surrogate and the true endpoint. This methodology serves as an additional approach to assess the Level 1 validity of a SoP and has been developed for various settings.[15,16,17] The scenario where the clinical endpoint is binary and the surrogate endpoint is continuous is particularly relevant in vaccine trials. This study applies the new method to evaluate whether hemagglutination inhibition (HI) antibody titers can serve as CoP for influenza infection.

METHODS

Study design

The data originate from a phase 3, observer-blind, randomized, controlled, multinational trial spanning five influenza seasons and involving children aged 6–35 months across 13 countries in Europe, Central America, and Asia between October 1, 2011, and December 31, 2014.[18] Children were randomly assigned (1:1) to receive either inactivated quadrivalent influenza vaccine (IIV4) or a control vaccine (pneumococcal conjugate vaccine, hepatitis A vaccine, or varicella vaccine). Vaccine-primed children received a single dose (0.5 mL on day 0), while vaccine-unprimed children received two doses (0.5 mL each on days 0 and 28). Vaccine-primed children had previously received at least two doses of seasonal influenza vaccine, with a minimum separation of 28 days.

The study lasted 6–8 months for each participant, including vaccination, influenza surveillance, and safety monitoring. Surveillance for influenza-like episodes started 14 days after each child’s last vaccination until the influenza season’s end. Influenza-like episodes encompassed influenza-like illness, doctor-diagnosed acute otitis media, and lower respiratory infections. After reporting influenza-like episodes within 7 days of onset, a nasal swab confirmed the influenza virus by reverse transcription-polymerase chain reaction (RT-PCR). The use of RT-PCR for the detection of influenza in this study is a qualitative assay, the result of which is either positive or negative. The true endpoint was the initial occurrence of RT-PCR-confirmed influenza A/H1N1, categorized as moderate-to-severe influenza or all-severity influenza. Moderate-to-severe influenza included RT-PCR-confirmed influenza with high fever, doctor-diagnosed acute otitis media, lower respiratory infection, or serious extrapulmonary complications, leading to intensive care unit admission or supplemental oxygen use for over 8 h. Alongside the true endpoint, a subcohort of children had their HI antibody titers against influenza A/H1N1 vaccine strain measured before and 28 days after the last vaccination.

The study obtained approval from independent ethics committees or institutional review boards in each participating center and adhered to the Declaration of Helsinki, International Conference on Harmonization Good Clinical Practice guidelines. Written informed consent was secured from parents or legally acceptable representatives. All necessary details about the study design and conduct can also be found at this link: https://www.clinicaltrials.gov/study/NCT01439360?id=115345andrank=1.

Statistical analysis

The methodology is based on the ICTEs, i.e., the causal effect of a given vaccine on an outcome of interest for a given individual. In the Neyman-Rubin “Potential Outcomes Framework” of causality, each individual has one outcome that would manifest if she/he were exposed to the vaccine (z = 1) and another outcome if she/he were exposed to the placebo/control (z = 0). To formalize this idea, we consider the vector of potential outcomes Y = (T₀,T₁,S₀,S₁)^Twhere T_z and S_z denote the potential outcomes for the true and surrogate endpoint when the individual receives either the control (z = 0) or the vaccine (z = 1). The ICTEs on the true and surrogate endpoints are defined as the difference between the corresponding potential outcomes, denoted by ΔT = T₁ − T₀ and ΔS = S₁ − S₀ , respectively. The vector Δ=(ΔT ,ΔS)^T is the fundamental element to assess surrogacy. However, these individual-level causal effects are nonobservable in clinical trials because individuals can only receive either the vaccine or the control. This problem has been called the fundamental problem of causal inference.[19]

We shall say that S is a good surrogate for T if the vaccine effect on S ( ΔS ) provides substantial information about its effect on the true endpoint T ( ΔT ). The concept of mutual information, rooted in information theory, measures the amount of “shared” information between the ICTEs. The ICA metric, which quantifies the association between ΔT and ΔS , is a transformation of the mutual information. This transformation ensures that the ICA always falls within the (0, 1) interval, with zero indicating no association and one indicating a perfect association between the vaccine effect on the surrogate and the true endpoint, i.e., knowing what the treatment does to the surrogate brings full information about what it will do to the true endpoint.[20]

To estimate the ICA, a model describing the distribution f (Y ) of the potential outcomes vector Y is needed. However, the fundamental problem of causal inference implies that the parameters characterizing the distribution of Y are not fully estimable from the data. Consequently, direct estimation of the ICA from observable data is not feasible. To tackle this challenge, we employ a sensitivity analysis, sampling a substantial number of nonestimable parameters from their set of possible values compatible with the data. For each sampled parameter, we calculate the corresponding ICA, producing a set of ICA values consistent with the data. The frequency distribution of these values characterizes the behavior of the ICA across a spectrum of plausible values for the nonestimable parameters, and the spread shown in the frequency distribution indicates the uncertainty about those parameters that cannot be estimated from the data.

Alternatively, one can make plausible assumptions about the nonestimable parameters, considering specific values based on clinical insights. For instance, in placebo-controlled vaccine trials, a common assumption is monotonicity, stating that the new vaccine will not perform worse than the placebo. Reasonable assumptions can also address conditional correlations between surrogate potential outcomes; for example, they may be constrained to positive values or all be set to zero. The latter condition is termed the conditional independence assumption, presuming that the outcome for the surrogate in one group (vaccine or placebo) provides no information about the outcome in the other group given Y_T=(T₀,T₁)^T. When coupled with monotonicity, this assumption ensures that all parameters are estimable, which signifies that the ICA can also be estimated from the available data. Nevertheless, it is important to emphasize that no matter how reasonable these assumptions are, they cannot be verified based on the observed data.

When estimating the ICA, it is crucial to distinguish between two fundamental sources of uncertainty: ignorance and imprecision.[21,22] We managed the ignorance arising from the nonestimable parameters through the proposed sensitivity analysis. Simultaneously, we addressed imprecision due to the limited sample size using a bootstrap approach. This involved generating multiple bootstrap samples from the original dataset and applying the sensitivity analysis to each of them.[23]

For a comprehensive description of the methodology and the implementation of the sensitivity analysis, readers are directed to Alonso Abad et al.[17] In practice, the Surrogate package’s R function ICA.BinCont.BS can be used to conduct the analysis. This package is freely available at http://cran.r-project.org/web/packages/Surrogate/.

Analysis sets

The implementation of our method requires a dataset that should contain information on the true endpoint, the surrogate endpoint, and a treatment indicator. A case–control subsampling approach was adopted for our analysis, i.e., from a total of 11,047 subjects in the per-protocol cohort without cases before postvaccination blood sampling, only 1379 individuals were part of the immune subcohort. This subgroup comprised subjects from whom prevaccination and postvaccination blood samples were collected to assess immunogenicity.[18] In addition to the immune subcohort, the evaluation of immunogenicity also included children from the nonimmune subcohort, revealing positive results for RT-PCR-confirmed influenza. In total, 1425 subjects were included in the analysis for the current study. There were 11 primed and 629 unprimed children in the control group; meanwhile, there were 10 primed and 775 unprimed children in the IIV4 group.

RESULTS

Out of 1425 subjects, 640 (44.91%) received the noninfluenza control vaccine, and 785 (55.09%) received the experimental IIV4. Among the 640 children in the control group, 68 had RT-PCR-confirmed A/H1N1 influenza, while in the IIV4 group, of 785 children, 28 experienced the illness. Restricting the clinical endpoint to moderate-to-severe influenza, 29 and 12 children in the control and experimental groups, respectively, had the illness, with the rest considered free from the infection. In both scenarios, the estimated VE of IIV4 is approximately 66%. For more detailed information regarding the characteristics of the subjects in the study, interested readers are referred to the earlier related publication.[18]

Figure 1 illustrates that the log HI antibody titers in the control group after vaccination tend to be lower than those in the IIV4 group. Specifically, they ranged from 0.699 to 3.107 with a mean of 1.070 in the control group, while the range was 0.699–3.559 with a mean of 2.219 in the IIV4 group. The analysis using the Wilcoxon rank-sum test showed that there is a significant difference in the log HI antibody titers between the two groups (P < 0.0001). In addition, Table 1 suggests that HI antibody titers were higher in children who were not infected, though the difference is only significant in the IIV4 group when the true endpoint is all-severity influenza (P = 0.0404). Despite the seemingly positive correlation between the noninfected rate and the distribution of HI antibody titers in both groups, it is of interest to quantify the extent of the association between the ICTEs on HI antibody titers and the absence of infection.

Figure 1.

The distribution of hemagglutination inhibition antibody titer against influenza A/H1N1 vaccine strain (on the logarithm scale) in the control group (left column) and in the inactivated quadrivalent influenza vaccine (IIV4) group (right column) before (top row) and after (bottom row) vaccination. HI: Hemagglutination inhibition

Table 1.

The hemagglutination inhibition antibody titers in the control (S₀) and experimental vaccine (S₁) groups conditional on the presence (T₀ or T₁=0) or absence (T₀ or T₁=1) of influenza in the corresponding groups

	S0|T0=0	S0|T0=1	S1|T1=0	S1|T1=1
All-severity
Range	0.699–1.301	0.699–3.107	0.699–3.258	0.699–3.559
Mean	0.799	1.102	1.984	2.228
Median	0.699	0.699	2.054	2.204
Moderate- to-severe
Range	0.699–1.146	0.699–3.107	1.146–2.806	0.699–3.559
Mean	0.797	1.083	2.028	2.222
Median	0.699	0.699	2.054	2.204

Figure 2 (left) displays the frequency distribution of the ICA when the true endpoint is all-severity RT-PCR-confirmed influenza. The frequency distribution was obtained after applying the sensitivity analysis to 300 bootstrap samples of the original data set. The range interval, which represents the smallest and largest ICA that aligns with the available data after considering the sampling variability, spans from 0.0139 to 0.9999. The somewhat inconclusive findings obtained from the observed range are not completely unexpected. However, the more significant observation is that despite occasional small ICA values, 80% of the values surpass 0.6150.

Figure 2.

Frequency distribution of the individual causal association with and without accounting for the sampling variability. The true endpoint was the first occurrence of all severity (left) and moderate-to-severe (right) reverse transcription-polymerase chain reaction-confirmed influenza. ICA: Individual causal association

To enhance the precision of the sensitivity analysis, one can leave out the most extreme values obtained for the ICA, e.g., one can construct the (1−α) symmetric density interval (SDI), calculated based on the corresponding α/2 and (1−α/2) quantiles. For instance, when α =0.25, the resulting interval contains 75% of all ICA values that are consistent with the data. The 0.75 and 0.85 SDIs of the ICA are 0.5511–0.8282 and 0.3632–0.8684, respectively, offering a quantitative basis for supporting the use of HI antibody titers as a CoP.

When the sensitivity analysis does not consider sampling variability, the resulting range interval is 0.2572–0.9744, predictably narrower than the previous result. In addition, the 0.75 SDI (0.6478–0.8409) and 0.85 SDI (0.6107–0.8745) now offer enhanced precision and greater confidence in employing HI antibody titers as a substitute for the absence of infection. As evident in Figure 2, the inclusion of sampling variability produces a more frequent occurrence of lower values for the ICA. Nonetheless, we generally advise always to account for sampling variability when assessing a CoP.

Table 2 compiles the results of the sensitivity analysis incorporating additional assumptions. As anticipated, the ranges of the ICA are somewhat inconclusive. However, the SDIs offer a more precise conclusion, particularly when additional assumptions are considered. For instance, under monotonicity, the 0.75 SDI (0.5769–0.7829) and 0.85 SDI (0.5508–0.8161) show some support for using the CoP. While unverifiable from the data, support for assuming monotonicity can be drawn from information such as the proportion of subjects without infection in the control group or the proportion of infected subjects in the experimental group. When the minimum value among both proportions is close to zero, assuming monotonicity seems plausible for the given data.

Table 2.

Uncertainty intervals (range, 0.75 symmetric density interval, and 0.85 symmetric density interval) for the individual causal association obtained from the sensitivity analysis applied to the 300 bootstrap samples under different identifying assumptions

Assumption	ICA
	Range	0.75 SDI	0.85 SDI
No assumption	0.0139–0.9999	0.5511–0.8282	0.3632–0.8684
Monotonicity	0.0729–0.9976	0.5769–0.7829	0.5508–0.8161
Positive conditional correlation	0.0162–1.0000	0.5992–0.8691	0.4364–0.9168
Monotonicity and conditional independence	0.1315–0.9407	0.5894–0.7770	0.5672–0.8152

The true endpoint was the first occurrence of all severity RT-PCR-confirmed influenza. ICA: Individual causal association, SDI: Symmetric density interval, RT-PCR: Reverse transcription-polymerase chain reaction

Another scenario to consider is when the ICA becomes fully estimable. This occurs when both monotonicity and conditional independence are simultaneously assumed. In such cases, the (1−α) SDI can be interpreted as the (1−α) ×100% bootstrap confidence interval for the ICA. For instance, when the 0.85 SDI is calculated, the ICA falls between 0.57 and 0.82. This means that by jointly accepting the validity of these assumptions, one can assert, with 85% confidence, that knowing the vaccine’s effect on the CoP will reduce the uncertainty about the vaccine’s effect on the absence of infection between 57% and 82%.

An intriguing finding emerged when the true endpoint was defined as the first occurrence of moderate-to-severe RT-PCR-confirmed influenza. In the sensitivity analysis without any assumptions, the outcome remains inconclusive, even when SDI intervals were used to summarize the results. The frequency distribution of the ICA in Figure 2 (right) illustrates a dispersion across nearly the entire unit interval, with two distinct prominent areas containing the majority of the values. This large degree of uncertainty may be explained by the small number of cases in both groups. However, when the monotonicity assumption was introduced, the frequency distribution [Figure 3] and the SDI [Table 3] moderately support the use of HI antibody titers as a CoP for moderate-to-severe influenza. The fact that the proportion of subjects in the IIV4 group who tested positive for influenza was only 0.0153 may support the use of the monotonicity assumption in this analysis.

Figure 3.

Frequency distribution of the individual causal association from the sensitivity analysis of 300 bootstrap samples of the dataset assuming monotonicity. The true endpoint was the first occurrence of moderate-to-severe reverse transcription-polymerase chain reaction-confirmed influenza. ICA: Individual causal association

Table 3.

Uncertainty intervals (range, 0.75 symmetric density interval, and 0.85 symmetric density interval) for the individual causal association obtained from the sensitivity analysis applied to the 300 bootstrap samples under different identifying assumptions

Assumption	ICA
	Range	0.75 SDI	0.85 SDI
No assumption	0.0038–1.0000	0.1391–0.8078	0.1184–0.8479
Monotonicity	0.0438–0.9971	0.5591–0.8004	0.5221–0.8263
Positive conditional correlation	0.0047–1.0000	0.2372–0.8551	0.2077–0.9010
Monotonicity and conditional independence	0.0837–0.9330	0.5661–0.7984	0.5299–0.8293

The true endpoint was the first occurrence of moderate-to-severe RT-PCR-confirmed influenza. ICA: Individual causal association, SDI: Symmetric density interval, RT-PCR: Reverse transcription-polymerase chain reaction

DISCUSSION

Influenza is a virus that undergoes continual evolution, with numerous strains circulating around the world. Protection acquired from either infection or vaccination against a specific strain does not guarantee immunity to newly emerging strains. Consequently, scientists are permanently engaged in developing new vaccine technologies to stay ahead of the evolving virus.[24] Aligned with this effort, researchers and clinicians underscore the significance of identifying and validating the appropriate CoP from influenza infection to expedite clinical trials for the new vaccines.

The serum HI antibody titer is widely recognized as the gold standard correlate of anti-influenza immunity and is extensively used for vaccine licensure.[25] The current trial was marked as the first study of an influenza vaccine against four types of influenza virus in children and the statistical evaluation of the HI antibody titer as CoP has been studied following the Prentice criteria.[26] In response to certain inherent issues associated with this approach, we conducted an assessment using a newly developed sensitivity analysis based on causal inference and information theory.[17] This approach offers a more comprehensive framework for evaluating surrogate markers and provides an intuitive causal interpretation by quantifying the association between the ICTE on the surrogate and the true endpoint. Another notable feature is that clinicians can easily incorporate plausible assumptions into the analysis, allowing biological or clinical knowledge to be considered as part of a sensitivity analysis. Finally, the availability of an R package to conduct the analysis facilitates practical implementation of the method.

Aligned with the previous study, the result indicated that HI antibody titers could be moderately useful as a CoP in assessing the efficacy of IIV4 in children. When the true endpoint is all-severity influenza, 80% of the ICA values surpass 0.6150, and 0.75 SDI of the ICA is 0.5511–0.8282, showing that there is a moderate association between the treatment effect on the HI antibody titers and the absence of infection. The use of HI antibody titers as a CoP may be more justified when this result is combined with substantive knowledge about the vaccine’s characteristics and the role of HI antibody titers in developing protection.

There is ongoing discussion about the usefulness of thresholds set for a CoP. Some scientific studies suggest that an HI titer of indicates a protective response against influenza infection.[25,26,27,28] Our proposed method provides a different perspective by quantifying the ICA between the treatment effect on the surrogate and the true endpoint. A strong association indicates that the treatment effect on the surrogate can reliably reflect the effect on the true endpoint. As demonstrated by our findings, the vaccine’s effect on the HI antibody titers as a continuous measure can help predict its effect on the absence of influenza infection.

Recognizing the limitations presented by the measurement of HI antibody titers, continuing efforts are directed toward identifying alternative immune correlates of protection to enhance the capacity to predict vaccine performance.[28] These correlates should also be associated with outcomes specific to particular risk groups. In addition, it is crucial to acknowledge that multiple immune responses, including cell-mediated immunity, may interact to confer protection against infection. With this consideration, there is potential interest in developing a new metric that quantifies the association between the individual treatment effect on multiple immune markers and protection.

CONCLUSION

Our study provides support that HI antibody titers could be used as a CoP for influenza infection.

Research quality and ethics statement

This study was approved by independent ethics committees or institutional review boards in each participating center and adhered to the Declaration of Helsinki, International Conference on Harmonization Good Clinical Practice (ICH-GCP) guidelines. All necessary details about the study, including study contacts and locations, can be found at this link: https://www.clinicaltrials.gov/study/NCT01439360?id=115345andrank=1. The authors followed applicable EQUATOR Network guidelines during the conduct of this research project.

Conflicts of interest

AC is an employee of and holds shares in the GSK group of companies.

Funding Statement

This work was supported by the Special Research Fund (BOF) of Hasselt University (BOF-number: BOF2OCPO3) and GlaxoSmithKline Biologicals to FO, Baekeland Mandaat (HBC.2022.0145) and Janssen Pharmaceuticals to FS.