Comparability of neuraminidase inhibition antibody titers measured by enzyme-linked lectin assay (ELLA) for the analysis of influenza vaccine immunogenicity

Abstract

Neuraminidase-inhibition (NI) antibody titers can be used to evaluate the immunogenicity of inactivated influenza vaccines and have provided evidence of serologic cross-reactivity between seasonal and pandemic H1N1 viruses. The traditional thiobarbituric acid assay is impractical for large serologic analyses and therefore many laboratories use an enzyme-linked lectin assay (ELLA) to determine serum NI antibody titers. The comparability of ELLA NI antibody titers when measured in different laboratories was unknown. Here we report a study conducted through the Consortium for the Standardisation of Influenza SeroEpidemiology (CONSISE) to evaluate the variability of the ELLA. NI antibody titers of a set of 12 samples were measured against both N1 and N2 neuraminidase antigens in 3 independent assays by each of 23 laboratories. For a sample repeated in the same assay, ≥ 96% of N1 and N2 assays had less than a 4-fold difference in titer. Comparison of the titers measured in assays conducted on 3 different days in the same laboratory showed that a 4-fold difference in titer was uncommon. Titers of the same sera measured in different laboratories spanned 3 to 6 two-fold dilutions (i.e. 8 to 64 fold difference in titer), with an average percent geometric coefficient of variation (%GCV) of 112 and 82% against N1 and N2 antigens, respectively. The difference in titer as indicated by fold range and %GCV was improved by normalizing the NI titers to a standard that was included in each assay. This study identified background signal and the amount of antigen in the assay as critical factors that influence titer, providing important information toward development of a consensus ELLA protocol.

1. Introduction

An enzyme-linked lectin assay (ELLA) that was initially described by Lambre, et al. (1) has been optimized to evaluate serologic responses to NA (2) and evaluate NA antigenic drift (3). This assay is more practical than the traditional thiobarbituric acid assay (4) which quantifies sialic acid released after cleavage by neuraminidase (5), while the ELLA measures the amount of the penultimate sugar, galactose that binds to peanut agglutinin (PNA) when it becomes exposed following release of sialic acid (6). The ELLA is conducted in a 96-well format using PNA conjugated to an enzyme and a colorimetric substrate. ELLA has recently been used to measure serum NA-inhibition (NI) antibody titers in a number of clinical studies (7–9), including studies demonstrating that both inactivated and live attenuated vaccines have the capacity to induce NA-specific antibodies (10, 11). While NI titers correlate with resistance against disease (12), the absolute NI titer that offers protection against influenza has not been established.

The CONSISE organization includes a network of laboratories that have assessed the comparability of influenza hemagglutination inhibition (HI) and microneutralization assays (13). While validation studies of the ELLA have been conducted in individual laboratories, the comparability of NI antibody titers measured in different laboratories has not been established. This is an important factor when comparing data between assays conducted at different sites. CONSISE laboratory members from 18 different countries, representing academic institutions, commercial organizations, regulatory laboratories, WHO Collaborating Centers (CCs), and National Influenza Centers (NICs), participated in a study of the inter-laboratory variability of NI antibody titers measured by the ELLA. The participating laboratories are listed in supplementary Table S1. Laboratories were provided with an ELLA protocol (2), NA antigens of N1 and N2 subtypes and a set of 12 samples that had been prescreened for a range of titers against each antigen. The samples included 9 different human sera, plasma from a cow transgenic for human Ig that had been vaccinated with a seasonal inactivated influenza vaccine (kindly provided by Sanford Applied Biosciences, Sioux Falls, South Dakota, USA) and purified human intravenous immunoglobulin (IVIG, kindly provided by Baxter Biomedical Research, Austria); the IVIG sample was duplicated in the panel in order to examine assay repeatability. The participating laboratories measured NI antibody titers in all the samples in 3 replicate assays against both N1 and N2 antigens.

Statistical analysis was performed to evaluate the variability of titers measured in each assay (assay repeatability), the variability of titers from three assays performed in the same laboratory on different days (intra-laboratory variability), and the variability of titers from three assays performed in all of the participating laboratories (inter-laboratory variability). Analysis of the data was performed in order to identify assay parameters that reduced variability or improved performance.

2. Methods

2.1 ELLA protocol

A published protocol (2) was distributed to all participating laboratories. Briefly, all samples were heat-treated at 56 °C for 45 min prior to serial two-fold dilutions in PBS. Equal volumes of sample and a predetermined amount of antigen were mixed and then transferred to 96-well plates coated with fetuin (Sigma-Aldrich, St. Louis, MO, USA). The plates were covered and incubated at 37 °C overnight (18–20 hours). After washing the plates, peanut agglutinin conjugated to peroxidase (Sigma-Aldrich, St. Louis, MO, USA) was added. Plates were incubated at room temperature for 2 h in the dark and washed before the addition of o-phenylenediamine dihydrochloride (OPD) substrate. Some participants used 3,3′,5,5′-tetramethylbenzidine (TMB) as a substrate. The reaction was stopped by adding 1 N H₂SO₄ and the absorbance was read at 490 nm (OPD) or 450 nm (TMB). The NA inhibition (NI) titer was expressed as the reciprocal of the highest dilution that exhibited ≥ 50% inhibition of NA activity (50% end-point titer) or the 50% inhibition concentration (IC50) that was determined by four-parameter non-linear regression analysis.

2.2 Study design

Each laboratory ran a set of 12 samples against N1 and N2 antigens in 3 independent assays. In instances when the distributed protocol could not be followed completely, the laboratory reported the modification together with the results. Some laboratories provided 2 sets of data: one following the distributed protocol and the other following an in-house ELLA protocol.

A study worksheet reporting the NI antibody titers measured in each assay as well as the assay conditions used by participating laboratories, was submitted to the lead laboratory and statistician. Results from laboratories that did not perform 3 replicate assays (data set M1) or did not use the distributed antigen (datasets N2, O1 and O2) were excluded from the analysis. Even though the reported results indicated that some assays did not meet all qualification criteria, all data from assays using the distributed sera and antigens were included in the analysis of variability. Each laboratory was assigned a letter code, with each dataset assigned a number. For example, laboratory H provided 2 datasets, one set of results was generated following the designated study protocol, and the second set of results was generated following an in-house ELLA procedure.

The lead laboratory measured NI antibody titers in a panel of de-identified sera that had previously been used for HI assays (14). These sera were collected from volunteers who had been vaccinated with an inactivated split 2009/10 seasonal influenza vaccine. Eight samples were selected for use in the ELLA study: these ranged in NI titers against N1 and N2 antigens of seasonal H1N1 influenza virus A/Brisbane/59/2007 (BR/07) and H3N2 virus A/Uruguay/716/2007 (UR/07), the antigens contained in the 2009/10 trivalent influenza vaccine. Other samples included in the study panel were a commercial human serum pool (Lonza, Basel, Switzerland) with a low NI antibody titer against each antigen, a preparation of purified human immunoglobulin (kindly provided by Baxter Biomedical Research, Austria) with high NI antibody titer against each antigen, and cow plasma obtained from an animal transgenic for human immunoglobulin genes that had been vaccinated with the 2012/2013 trivalent influenza vaccine (kindly provided by Sanford Applied Biosciences, USA). Within the set, one sample was repeated, resulting in a set of 12 samples tested in each study assay. The samples were labeled as S001 – S012 and shipped to participating laboratories on dry ice. Sample identities were maintained blind and their identities were revealed, as shown in Supplementary Table S2, only after the data had been submitted for statistical analysis.

2.4 Antigens

The antigens used in ELLA were in the form of inactivated reassortant H6N1 and H6N2 whole viruses. These reassortant viruses were generated by reverse genetics as previously described, with each reassortant expressing the targeted NA antigen, the HA of an avian H6 virus, and all other proteins from A/PR/8/34 (15). The viruses were cultured in 9–12 day old specific-pathogen-free embryonated chicken eggs (Sunrise Farms, NY, USA) and inactivated by incubation with beta-propiolactone (BPL) for 4 hours at 4 °C prior to inactivating residual BPL at 37 °C for 30 min. Inactivation of the virus was confirmed by serial blind-passage in eggs. Virus aliquots were stored at −80 °C. Prior to performing the assay, each laboratory titrated the antigen in order to determine the appropriate dilution to be used.

2.5 Statistical analysis

Within-assay repeatability was assessed by comparing the titers of samples 1 and 10 reported in each assay, since these were identical. The variability of titers generated within each laboratory (intra-laboratory variability) was evaluated by comparing the titers measured in three independent assays by the same operator. To measure the variability of titers reported by different laboratories for each of the 12 samples (inter-laboratory variability), the geometric mean titers (GMTs), the geometric standard deviation (GSD) and percent geometric coefficient of variation (%GCV) across laboratories were calculated; GSD = exp(SD(log titer)) and %GCV =100*(GSD-1). When there is no variation, GSD=1 and %GCV=0.

3. Results and Discussion

3.1 Assay repeatability

Assay repeatability was assessed by determining the ratio of the 50% end-point titers for the same sample that had been duplicated in the study set (i.e. S001 and S010). The ratios were determined for the 3 assays conducted in each laboratory. In 69 of 71 N1 assays (97%) and 68 of 71 N2 assays (96%), the difference in titer between S001 and S010 was 2-fold or lower (Supplementary Table S3).

3.2 Intra-laboratory assay variability

Intra-laboratory assay variability was assessed in each laboratory by evaluating the fold differences in 50% end-point NI titers measured for all 12 samples across three independent assays. Differences in IC50 titers reported in assays conducted in the same laboratory on different days, was also considered (Table 1). The majority of laboratories reported results in which the difference across independent assays in NI antibody end-point titers for all samples was < 4-fold (15 and 16 of the 23 datasets for the N1 and N2 antigens, respectively). When differences in titers reported as IC50 were compared, the variability was even lower, with 18 of the 21 datasets having titers against the N1 antigen that were < 4-fold different in repeat assays for all 12 samples and 17 of 21 IC50 datasets had titers against N2 that were < 4-fold different across independent assays for all 12 samples.

Table 1.

Intra-laboratory assay variability: Number of samples that had ≥4-fold difference in 50% end-point or IC50 titer when run in three independent replicate assays (n=12)

Number of sera with titers that were ≥4-fold different against
	H6N1		H6N2
Lab	50% end-point	IC50	50% end-point	IC50
A	0	0	0	0
B	0	NAa	0	NA
C	0	0	1	0
D	1	0	1	0
E	0	0	0	0
F	1	0	1	0
G	0	0	0	0
H1	0	0	0	0
H2	2	0	0	0
I	3	0	2	2
J	0	1	0	1
K	0	0	0	0
L	1	0	4	0
M2	0	0	0	0
N1	0	1	0	1
P	2	0	0	1
Q	0	0	0	0
R	0	0	0	0
S	0	0	0	0
T	0	0	0	0
U	0	0	0	0
V	1	1	1	0
W	1	NA	2	NA

^aNA, not applicable because the IC50 values were not provided for this dataset.

To examine the benefit of calculating the IC50, the difference in variability in all results using 50% end point analysis and IC50 was assessed for all datasets and all samples. For H6N1, the difference in assay variability determined using the 50% end-point and IC50 titers was statistically significant (p=0.021), while the corresponding difference in assay variability for H6N2 was not significant (p=0.854).

3.3 Inter-laboratory assay variability

3.3.1 Overall inter-laboratory assay variability

Variability in NI titers measured in different laboratories was assessed by examining the differences in GMTs in datasets submitted by each laboratory for each of the 12 samples regardless of whether the assay had met all acceptance criteria specified in the protocol. The 50% end-point GMT of each sample in all datasets is shown for the N1 antigen in supplementary Table S4 and the N2 antigen in supplementary Table S5. The variability in titer across laboratories (%GCV) is shown in Table 2. Dataset M1 was excluded because assays were not repeated 3 times; datasets N2, O1 and O2 were excluded from the analysis because the assays were significantly different from the distributed protocol (used different antigens and/or a significantly different method). When taking all remaining datasets into account, the mean %GCV across 12 samples for 50% end point titers against the N1 antigen was 112% and against the N2 antigen was 82%. The variability in titers across laboratories was greater for assays using the N1 antigen than for assays using the N2 antigen (p= 0.00094, 2 sample t-test).

Table 2.

Inter-laboratory variability assessed as %GCV and fold range of absolute (Abs.) and normalized geometric mean NA inhibition titers against N1 and N2 antigens.

Sample	%GCV				Fold range (log 2)
	Abs. N1	Normalized N1	Abs. N2	Normalized N2	Abs. N1	Normalized N1	Abs. N2	Normalized N2
1	95	69	60	43	3.3	3.0	3.0	2.0
2	168	0	96	60	5.3	0	3.0	2.7
3	111	49	84	57	4.5	2.2	3.7	2.5
4	98	65	98	106	3.6	2.4	4.3	5.0
5	123	50	75	47	3.9	2.5	3.0	2.1
6	87	69	64	52	2.8	2.5	2.4	2.0
7	90	65	74	53	3.3	2.7	3.3	2.7
8	96	55	74	0	3.2	2.7	3.3	0
9	114	54	114	73	3.8	2.2	4.5	3.2
10	116	56	84	54	3.9	2.0	3.3	2.7
11	119	53	80	43	4.2	2.3	3.2	2.1
12	130	61	80	58	4.5	2.1	3.3	2.4
Mean	112	59	82	59	3.9	2.4	3.4	2.7

To assess whether assay variability was related to the NI antibody titer, the correlation between NI antibody titer (log2) and %GCV was evaluated (Supplementary Figure S1). The correlation coefficients between GMT and %GCV suggested that there was a positive relationship between the titer and variability for the N1 antigen, but the relationship was not statistically significant (R²=0.161). The relationship was even poorer for the N2 antigen (R²=0.036), showing that there is little association between variability and the magnitude of the titer.

Table 2 also shows the range (in log2 scale) in titers for each sample. The relative frequency of end-point NI antibody titers for each serum sample is shown in Figure 1 for the N1 antigen and in Figure 2 for the N2 antigen. The titers of the same sera measured in different laboratories spanned 3 to 6 two-fold dilutions, i.e., 8 to 64 fold difference in titer. In most instances, the distribution of titers measured followed a bell-shaped curve, with the greatest number of laboratories measuring the median titer.

Figure 1.

Relative frequency of end-point NA inhibition antibody titers against the N1 antigen for each sample (n = 24): the y axis shows the percent of laboratories that attained a particular titer; the x axis shows the lower bound of the GMT of the NI antibody titer (log 2) reported for the 3 assays conducted in each laboratory, i.e., a value on x-axis of 7.3 includes GMTs of 2^7.3 to <2^8.3 (160 to <320).

Figure 2.

Relative frequency of end-point NA inhibition antibody titers against N2 antigen for each sample (n = 24): the y axis shows the percent of laboratories that attained a particular titer; the x axis shows the lower bound of the GMT of the NI antibody titer (log 2) reported for the 3 assays conducted in each laboratory, i.e., a value on x-axis of 7.3 includes GMTs of 2^7.3 to <2^8.3 (160 to <320).

3.3.2 Normalizing the data reduces assay variability

Samples that had moderate NI antibody titers against N1 and N2 antigens were selected to normalize the data; sample S002 was elected to normalize titers against the N1 antigen, and S008 was elected to normalize the titers against N2. The normalized titers are shown in supplementary Tables S6 and S7. The variability in end-point titer was reduced when the GMTs were normalized, resulting in an average reduction in %GCV from 112 to 59 and 82 to 59 for N1 and N2 ELLAs, respectively (Table 2). This improved consistency in normalized titers (50% reduction in average %GCV), is similar to the reduced variability observed in inter-laboratory studies of hemagglutinin inhibition and microneutralization assays (16, 17). Except for the NI antibody titer for S004 against the N2 antigen, all of the ranges in titers measured in different laboratories against both the N1and N2 antigens were reduced when the data were normalized; the average range (log 2) of N1 titers was reduced from 3.9 to 2.4 and the range of N2 titers was reduced from 3.4 to 2.7 (Table 2). These data suggest that serum standards should be developed and made available for use in serology studies as a means to compare results generated in different laboratories.

3.3.3 Assay parameters that impact variability of titers measured in different laboratories

A review of the assay conditions used by each participating laboratory identified some parameters that may have contributed to differences in titers reported and hence inter-laboratory assay variability. The assays differed in the amount of antigen used, the maximum signal in each assay (i.e., the signal of antigen (inactivated virus) alone), the signal to noise ratio and the substrate used for colorimetric read-out. To examine whether these differences impacted the titers and assay variability across different laboratories, the mean GMTs and %GCV between subgroups of participating laboratories were compared (Table 3). The overall GMT (geometric mean of titers from 12 samples tested) and %GCV for assays that met the acceptance criteria (acceptable signal strength and background <10% of the positive signal) and followed the given protocol with the exception of substrate, were compared; datasets A, E, H1, L, N1 used OPD as substrate (shown as Group 1 in Table 3) while datasets B, H2, I, K, M2 used TMB as the substrate (shown as Group 2 in Table 3). The GMTs as well as %GCVs for results reported using OPD and TMB for both H6N1 and H6N2 antigens were similar (p ≥0.06). The reproducibility in results for assays using different substrates is exemplified by datasets provided by laboratory H that used OPD and TMB in parallel assays (datasets H1 and H2 respectively, supplementary Tables S4 and S5).

Table 3.

Potential factors affecting inter-laboratory variability of results: Comparison of geometric mean NI antibody titers (GMT) and assay variability (%GCV) for assays performed in multiple laboratories using different acceptance criteria, substrates or antigen amounts

Factor	End-point measured
	GMT: mean ± SDa		%GCV: mean ± SDb
	Group 1	Group2	Group 1	Group 2
Assays using the H6N1 antigen
Substratec	81±4.9	90±5.6	80 ±20	137±39
Background signald	93±5.2	129±5.2	105±18	233±103
Positive signal strengthe	99±5.3	104±5.1	131±31	128±36
Antigen amountf	106±5.3	84±4.9	117±27	137±22
Assays using the H6N2 antigen
Substratec	94±6.1	114±6.3	77±23	77±34
Background signald	115±5.8	129±5.7	84±20	151±37
Positive signal strengthe	112±6.2	137±5.1	89±18	115±37

^aThe geometric mean titers and standard deviations (SD) were calculated for Groups 1 and 2 using the mean GMT of all samples in each dataset included in the group.

^bThe % geometric coefficient of variation and SD was calculated for Groups 1 and 2 as the arithmetic mean of the %GCV reported for datasets within each group.

^cGroup 1: datasets A, E, H1, L, N1 used OPD; Group 2: datasets B, H2, I, K, M2 used TMB.

^dGroup 1: A, B, E, G, H1, H2, I, J, K, L, M1, M2, N1 had background values ≤10% of maximum signal; Group 2: datasets C, D, V had background values >10% of maximum signal.

^eGroup 1: A, B, D, E, H1, H2, I, K, L, M2, N1 (maximum signal ≥1.7); Group 2: C, G, J, V (maximum signal <1.7).

^fGroup 1: datasets A, B, D, H1, H2, I, K, L, M1, M2, N1 that used ≥1:60 dilutions of the H6N1 antigen; Group 2: datasets C, E, G, J, V that used greater amounts of antigen i.e. less dilute (<1:60) antigen stock. Both Group 1 and Group 2 datasets used similar dilutions of H6N2 antigen and therefore comparison of data from different H6N2 antigen amounts was not possible.

To determine whether the titers and variability were impacted by background values, the GMT and %GCV were calculated for results reported from assays that met the background signal acceptance criterion (≤10% of positive signal strength) or had >10% background values, shown as Group 1 and Group 2 in Table 3 respectively; Group 1 included datasets A, B, E, G, H1, H2, I, J, K, L, M1, and M2 and Group 2 included datasets C, D, and V. The GMT of 50% end-point titers against N1 and the %GCV in both N1 and N2 assays were statistically greater when the background signal was higher than recommended (p≤0.02). This suggests that laboratories should adhere to this parameter as an assay acceptance criterion, particularly if data are to be compared between different laboratories.

Some laboratories reported data from assays in which maximum absorbance was <1.7, while others had maximum absorbance levels of approximately 4.0. To determine whether titers obtained in assays with low maximum signal strength were significantly different to assays in which signal strength was ≥1.7, GMTs and %GCVs were compared between A, B, D, E, H1, H2, I, K, L, M2, and N1 (Group 1, maximum signal ≥1.7) vs. C, G, J, and V (Group 2, maximum signal <1.7). Signal strength did not have a significant impact on either GMT (p-value=0.75 and 0.07 for N1 and N2 antigens respectively) or %GCV (p=0.39 and 0.57 for N1 and N2 antigens, respectively).

Titration of antigen before conducting the ELLA is critical as this allows the user to identify the dilution of antigen to use in the assay. The antigen titration should yield absorbance values that result in an S-shaped curve, with the signals at low virus dilutions forming a plateau that is followed by a linear relationship between virus dilution and signal. Published studies show that the amount of antigen used in the ELLA is critical for assay robustness, with consistent titers measured when the amount of antigen used in the assay is within the linear range of the virus (antigen) titration curve (2, 3). The assay protocol that was distributed to study participants gave instructions to use a dilution of virus (antigen) that gives 90–95% of maximum activity, however the dilution of H6N1 antigen that was selected differed between laboratories because the antigen titrations and analytical tools were not the same. To determine whether using different amounts of H6N1 antigen impacted assay variability, GMTs and %GCVs of datasets A, B, D, H1, H2, I, K, L, M1, M2, and N1 (Group 1) from assays that used ≥1:60 dilution of the H6N1 antigen and datasets C, E, G, J, and V (Group 2) from assays using a greater amount of H6N1 antigen (i.e., less than a 1:60 dilution), were compared. The average GMTs from assays with more antigen were significantly lower than GMTs reported in datasets using less antigen (p=0.05, ANOVA taking into consideration sample variability). The variability in results was statistically similar for Groups 1 and 2. While the data from this study are insufficient to draw a conclusion regarding the sensitivity of ELLA under different conditions, others have pointed out that use of excessive amounts of antigen in the ELLA is likely to reduce the sensitivity of the assay (2, 3). All laboratories used a similar amount of N2 antigen (the stock was diluted 1:20 or 1:40) and therefore an analysis to evaluate the impact of antigen dilution on NI antibody titers against N2 could not be tested. These results confirm the importance of using an amount of antigen that is within the linear range of the titration curve. The protocol was consequently revised to indicate that a dilution of virus that gives 90% of maximum signal should be used, with a recommendation to use 4-parameter logistics to determine antigen dilution to be used in assays.

This international study provided an opportunity for laboratories that had not previously conducted the ELLA, to become proficient in measuring NI antibody titers. Discussion among the participants also identified improvements that can be implemented in future assays. For example, a buffer that has a pH at which NA enzyme activity is optimal allows the assay to be performed in a shorter time period (9), and recombinant NA (18) or VLPs (8, 9) can be used as a source of antigen, thereby bypassing the need to generate H6 reassortants. Future studies will be needed to evaluate whether assays performed with improved conditions or with different types of antigens, yield results that are comparable with the ELLA protocol used in this study.

4. Conclusions

Assay repeatability as well as intra- and inter-laboratory variability was assessed in an international CONSISE study of the ELLA. The NI titers of samples repeated within the same assay differed by no more than 2-fold. Assays repeated within the same laboratory gave consistent results, with most datasets having <4-fold differences in titer. As expected, there was greater variability in titers across different laboratories than within each laboratory. Although up to a 64-fold difference in NI antibody titer was observed, this is lower than differences in hemagglutination inhibition and microneutralization titers measured in different laboratories (the range of titers was 80-fold for HI titers and 109-fold for microneutralization assays) (17). The variability of NI antibody titers measured in different laboratories was also reasonable, with %GCV that was reduced further by normalizing the NI antibody titers to the titer of a sample elected as a standard that was run in every assay. Comparison of the results reported by different laboratories confirmed that the amount of antigen used in the assay is a critical parameter and therefore should be carefully established by repeat titrations prior to performing ELLA. The analysis also suggests that data should only be accepted from assays that meet the background acceptance criterion (background should be ≤10% maximum positive signal). Although the data suggest the current assay has low variability and is robust, further assay improvements, such as the use of recombinant antigens or establishment of conditions that allow shorter incubation times, are anticipated. Such improvements will further enable the measurement of NI antibody titers and assessment of these antibodies as a correlate of immunity against influenza.

Abbreviations

CONSISE

Consortium for the Standardisation of Influenza SeroEpidemiology

ELLA

Enzyme-linked lectin assay

NI

Neuraminidase inhibition

GMT

Geometric mean titer

%GCV

Percent geometric coefficient of variation