A New Electrochemiluminescence-Based Multiplex Assay for the Assessment of Human Antibody Responses to Bordetella pertussis Vaccines

Abstract

Introduction

Commercially available enzyme-linked immunosorbent assay (ELISA) kits designed for pertussis diagnostic purposes are frequently used to assess antibody responses to pertussis vaccines in clinical trials, but have limited accuracy and are not calibrated against international standards. We developed a new electrochemiluminescence (ECL)-based multiplexed assay and compared its performance to two commercial Bordetella pertussis ELISA kits and to historical in-house ELISAs.

Methods

The ECL assay quantifies serum concentrations of antibodies against four B. pertussis antigens: pertussis toxin (PT), filamentous hemagglutinin (FHA), pertactin (PRN), and fimbrial agglutinogen (FIM). The assay was validated for precision, accuracy, dilutability, lower limit of quantification, and specificity. Sera from a clinical trial (CTRI/2016/11/007434) were used to compare the ECL assay to two commercial ELISA kits available from GenWay BioTech and Demeditec Diagnostics for accuracy, linearity, specificity, and concordance to both internal (WWO-2-043) and international (NIBSC 06/140) references. Sera from four clinical trials (NCT02587520, NCT00255047, NCT00347958, NCT01346293) were used to compare the concordance to clinical ELISAs. Informed consent was ensured prior to using any sera.

Results

Precision, accuracy, dilutability, lower limit of quantification, and specificity were demonstrated for the ECL assay. Concordance between the ECL assay and established clinical ELISAs was met for antibody responses to PT, FIM, and PRN, but not for FHA. The ECL assay demonstrated higher accuracy and linearity than the ELISA kits. While concordance between the ECL and commercial kits was low, the ECL assay better distinguished between pre- and post-vaccination clinical samples.

Conclusion

The new ECL assay was validated for the quantitative evaluation of anti-PT, anti-FHA, anti-FIM, and anti-PRN IgG antibodies in samples from clinical trials, and demonstrated equivalent or better performance than two commercially available ELISA kits.

Key Summary Points

Why carry out this study?

Commercially available enzyme immunoassay diagnostic kits are not validated for the evaluation of antibody responses to whole-cell pertussis vaccines in clinical trials.

We developed and validated a new multiplexed electrochemiluminescence-based assay for the detection of human IgG antibodies against four specific Bordetella pertussis antigens.

We used clinical trial sera and international standards to compare the performance of the new electrochemiluminescence assay to two commercially available Bordetella pertussis enzyme immunoassay kits and to historical in-house single-antigen enzyme-linked immunosorbent assays.

What was learned from this study?

The new electrochemiluminescence assay was validated in terms of precision, accuracy, dilutability, lower limit of quantification, and specificity.

The accuracy and linearity of the observed versus the expected response were better for the new electrochemiluminescence assay than the comparator kits.

The new electrochemiluminescence assay allows more precise, accurate, and specific analysis of antibodies to Bordetella pertussis antigens in clinical trials.

Introduction

Pertussis is still endemic in many regions of the world, due not only to a combination of sub-optimal vaccination coverage and vaccination recommendations [1] but also to some intrinsic limitations of currently available pertussis vaccines [2]. As there is no established immunological correlate of protection for pertussis, the only way to assess the potential efficacy of pertussis vaccines in clinical trials is to rely on their immunogenicity, since clinical endpoint trials cannot be performed [3]. The evaluation of serological responses to whole-cell pertussis (wP) and subunit vaccines has been a subject of debate for many decades [4]. Traditionally, the evaluation of immune responses in clinical trials involving wP vaccines has used a functional assay based on the measurement of the capacity of sera from vaccine recipients to agglutinate in vitro a specific strain of Bordetella pertussis [5]. This agglutination assay has been the only one used to correlate immunogenicity data with clinical efficacy data obtained in historical clinical endpoint efficacy trials performed with wP vaccines [6, 7]. In these historical trials, a level of agglutinin (as measured by this functional assay) from 1/80 to 1/320 (i.e., the highest dilution of the sera still presenting agglutination activities) was shown to correlate relatively well with protection against pertussis disease [8]. Furthermore, good correlation between agglutinin antibody concentrations and anti-fimbriae (anti-FIM) antibodies was shown with agglutinin antibody concentrations above the 1/320 concentration threshold [9, 10]. Anti-pertussis toxin (anti-PT) antibodies, and, to a lesser extent, anti-FIM and anti-pertactin (anti-PRN) antibodies seem to be the most important to B. pertussis antibodies responsible for this agglutination activity [11, 12]. Due to the difficulties inherent in the implementation of the B. pertussis agglutination assay (e.g., high variability, executional difficulties, and subjective read-out), recent clinical trials of wP vaccines have used either in-house enzyme-linked immunosorbent assays (ELISAs) or commercial ELISAs developed to measure antibodies directed against the entire B. pertussis organism or to specific antigens. The key drawback of these assays is that they have been developed primarily for clinical diagnostic purposes, and their non-standardized use both in terms of coating antigens (their nature and quality) and/or the presentation of results can lead to major difficulties in their interpretation [13]. To assess the immunogenic potential of any vaccine, and to be able to correlate laboratory results with clinical events, reliable laboratory methodologies that provide valid assessments of antibody responses are crucial in both nonclinical and clinical studies. For this, robust method development and validation are important, and are necessary bioanalytical components of any vaccine marketing authorization application. Existing regulatory guidance documents on the validation of methods address immunoassays in only a limited manner [14], and so assay standardization and robustness is central in the search for reliable ways to compare results across vaccines and clinical development programs [15]. At Sanofi Pasteur, we recently developed and validated a new electrochemiluminescence (ECL)-based multiplexed assay [using technology from Meso Scale Diagnostics (MSD), Rockville, MA, USA] [16] to assess antibody responses to a range of vaccine antigens, including the well-characterized B. pertussis antigens, pertussis toxin (PT), filamentous hemagglutinin (FHA), FIM, and PRN in human serum samples. The aims of this study were (1) to report the development and validation of the ECL assay, and (2) to compare its performance to two commercial B. pertussis ELISAs, that have been used in wP vaccine clinical development programs, and to historical in-house ELISAs, that have been used in acellular pertussis (aP) vaccine clinical development programs.

Methods

Human Serum Samples

Serum samples (pre- and post-vaccination) used for comparison of the ECL method and the commercial kits were collected in a Phase I/II randomized, active-controlled, open-label clinical study conducted in toddlers and infants at 4 sites in India between November 2016 and October 2017 (Clinical Trials Registry India Number CTRI/2016/11/007434) [17]. Serum samples used for ECL method validation and concordance to in-house single antigen ELISAs were from studies conducted in the US with the following ClinicalTrials.gov identifiers: NCT02587520 (a Phase I/II dose and formulation ranging study to assess the safety and immunogenicity of an aP vaccine candidate in healthy adolescents, adults, and older adults in the US), NCT00255047 (a Phase III study in healthy full term infants to corroborate that pentavalent and quadrivalent combination diphtheria–tetanus-aP [DTaP] vaccine formulations from the same manufacturer were sufficiently interchangeable to tailor use to local preference or availability) [18], NCT00347958 (a Phase IV study in 15–69 year olds following revaccination with ADACEL^® vaccine 4–5 years after first vaccination), NCT01346293 (a Phase III study to compare the safety and immunogenicity of DTaP-IPV with DAPTACEL^® + inactivated poliovirus vaccine as the 5th dose booster in children ≥ 4 to < 7 years of age) [19]. The samples covered a wide assay range (negative, low, medium, and high concentration samples), as well as a wide age range to be representative of the general population, and, for the comparison to the in-house ELISAs, samples were inclusive of the age groups represented in previous clinical trials. The use of all serum samples was approved by the institutional ethics committee, and the conduct of the clinical study in which the samples were collected was consistent with the Declaration of Helsinki and compliant with the International Council for Harmonization guidelines for Good Clinical Practice as well as with all local and national regulations. An informed consent form previously signed by each participant’s parents or legally acceptable representatives authorized the use of all serological samples for research purposes. All samples used in the assessment were blinded and re-randomized by a Sanofi Pasteur biostatistician, and aliquoted by an independent sample management team.

Assays Used

ECL-Based Multiplexed Assay Methodology

The ECL-based multiplexed assay was used to quantify serum IgG concentrations to four pertussis antigens: anti-PT, anti-FHA, anti-PRN, and anti-FIM (including types 2 and 3). Additionally, the ECL assay included diphtheria and tetanus antibody analysis, but these data are out of scope of this publication and not presented.

Sector plates (6 spots/well in a 96-well plate format) were coated with tetanus toxoid, diphtheria toxoid, PT, FHA, PRN, and FIM 2&3 antigens in concentrations ranging from 10 to 20 ng/spot by MSD. Antigens used to coat the microtiter plates were purified preparations manufactured by Sanofi Pasteur and supplied to MSD to produce the multi-spot-coated plates. The pre-coated plates were stored in a refrigerated (2–8 °C) unit upon receipt. Assay control sera included four samples (Internal Quality Controls: IQC1, IQC2, IQC3, and IQC4) which are sera samples with antibody concentrations representative of, respectively, high, mid, low, and negative [or near the assay lower limit of quantification (LLOQ)] for the six analytes. The conjugate used for this method was goat anti-human IgG, conjugated with a Sulfo-Tag label by MSD.

Multi-spot-coated test plates were blocked with blocking buffer then covered and incubated in an ambient temperature incubator at ~ 700 rpm for 45 ± 15 min. Following incubation, the plates were washed using 0.01 M PBS-Tween (0.01 M phosphate-buffered saline + 0.05% Tween 20) prior to the addition of standards, controls, and samples, pre-diluted in MSD Diluent 100. Reference standard WWO-2-043 (an in-house reference standard of pooled human sera prepared by combining sera from participants vaccinated with a DT5aP-containing vaccine and calibrated against NIBSC or US reference standards) was pre-diluted 1/100 in the first well of a deep well plate followed by a series of 11 four-fold dilutions. The final well was used as a blank. The 12 wells containing the serially diluted reference standard and the blank were then transferred to the test plates in duplicate. All positive controls and the negative control were pre-diluted to an initial dilution appropriate for the linear range of the assay, and then diluted through a series of four four-fold dilutions and transferred to each test plate in singleton. The test samples were tested at an initial 1/100 starting dilution followed by a series of four four-fold dilutions and transferred to the test plates in singleton.

The test plates were sealed and incubated at 21 ± 3 °C for 120 ± 15 min on an orbital plate shaker set to ~ 700 rpm. Following incubation, the plates were washed using 0.01 M PBS-Tween, and the MSD-conjugated goat anti-human IgG antibody was diluted to the optimal working dilution and 25 µL was added to all wells. The plates were again sealed and incubated at 21 ± 3 °C for 60 ± 5 min on an orbital plate shaker set to ~ 700 rpm. Following incubation, the plates were again washed using 0.01 M PBS-Tween, and 150 µL of 1× MSD Read Buffer (a chemiluminescent substrate) was added to all wells. The plates were then read on an MSD Sector S 600 reader within 40 min. Six different datasets, one for each analyte, were generated from each microtiter plate. The signal generated from the individual antigen spots was directly proportional to the amount of antigen-specific IgG present in the sample. The concentration of antigen-specific IgG was calculated using the qualified assay reference evaluated with a 4-parameter logistic curve and SoftMax^® Pro data analysis software.

Commercial ELISA Methodology

Two commercially available assay kits (NovaLisa™ B. pertussis IgG ELISA; GenWay Biotech, San Diego, CA, USA [20], and B. pertussis IgG ELISA; Demeditec Diagnostics, Kiel, Germany [21]) were used to evaluate antibodies to B. pertussis antigens, but did not differentiate between the four antigens described for the ECL assay.

For both the GenWay and Demeditec anti-wP assays, microtiter strip wells were pre-coated with a heterogenous mixture of B. pertussis antigens to bind corresponding antibodies in the sample. Diluted samples were pipetted into the wells of the microtiter plate to initiate binding between the serum IgG antibodies and the B. pertussis antigens. After incubation and washing the wells to remove all unbound sample material, horseradish peroxidase-labeled anti-human IgG conjugate was added to bind captured B. pertussis antigen-specific antibodies, followed by further incubation and washing. The immune complexes formed were visualized by adding a tetramethylbenzidine substrate, the intensity of color development being proportional to the amount of B. pertussis antigen-specific IgG antibodies in the sample. Sulfuric acid was added to stop the reaction and the absorbance was read at 450 nm with a reference wavelength of 620 nm using an ELISA microwell plate reader. Full details are provided in the product inserts [20, 21].

Historical In-House ELISA Methodology

The surface of microtiter plates was coated with an optimized concentration of purified PT, FHA, FIM (2&3), or PRN for 18 h ± 2 h at ambient temperature in a humidified incubator in carbonate (pH 9.6) coating buffer. Plates were then washed with PBS containing 0.05% Tween 20 and stored at 2 to 8 °C in a sealed plastic bag for a maximum of 14 days.

Prior to use, plates were blocked by the addition of blocking buffer (PBS/Tween 20 + 1% normal goat serum) to all wells and incubated at 18 °C to 24 °C for 45 min ± 5 min. During the incubation, the assay reference, controls, and samples were prepared in PBS/Tween 20 assay buffer. The assay reference standard curve is comprised of eight two-fold serial dilutions of an in-house reference human antiserum calibrated against US reference pertussis antiserum (human) Lot 3 and Lot 4 with a defined concentration of anti-PT, anti-FHA, anti-FIM, and anti-PRN IgG in EU/mL. Each plate included a positive and negative control (qualified human sera with assigned ranges) that was tested at seven and four serial two-fold dilutions, respectively. Test samples were prepared at an initial dilution of 1/100, followed by seven serial two-fold dilutions. Following incubation, plates were washed (with PBS/Tween 20) and prepared reference, controls, and samples were added to the plates and incubated at 37 °C ± 2 °C in a humidified incubator for 60 min ± 5 min. Following the incubation, the plates were washed with PBS/Tween 20 wash buffer and then incubated with peroxidase conjugated goat anti-human IgG conjugate at 37 °C ± 2 °C in a humidified incubator for 60 min ± 5 min. The plates were then washed with PBS/Tween 20 buffer and developed following the addition of TMB Microwell Peroxidase Substrate (KPL) for 45 min ± 5 min at 18 °C to 24 °C. The development was stopped following the addition of 2 N sulfuric acid, and the optical density of the wells was measured spectrophotometrically at 450 nm (with 540 nm as background) on a Spectramax microplate reader (Molecular Devices). The concentration of samples and controls were extrapolated from the plate reference curve with a parallel line calculation method using Softmax Pro software.

ECL-Based Multiplexed Assay Validation Methodology

The ECL assay was validated for precision, accuracy, dilutability, LLOQ, and specificity using samples from the US studies. The validation methodology and acceptance criteria are summarized in Table 1.

Table 1.

Summary of method validation

Validation parameter	Testing method	Acceptance criteria	Results
Specificity: competition	Positive samples with competing antigens (homologous and heterologous) at multiple concentrations ≤ 20 µg/mL	Heterologous competitors: ≤ 25% competition Homologous competitors ≥ 75% competition	All antigens and all samples ≤ 25% (heterologous competitors) and ≥ 75% (homologous competitors) at concentrations ≤ 20.0 µg/mL
Specificity: matrix effects	Spike recovery of assay reference standard	Recovery in the range of 100% ± 20% for ≥ 80% of samples for each matrix	100% of samples for each matrix had recovery 83–119%
Accuracy	Spike recovery using a sample of known concentration, spiked into negative/low sera	Recovery of 100% ± 20% (samples > 3× LLOQ) and 100% ± 25% (samples ≤ 3× LLOQ)	80–114% (samples > 3× LLOQ) and 84–124% (samples ≤ 3 × LLOQ)
Precision	Multiple determinations (≥ 5) of a sample panel (≥ 58 samples) for each antigen using at least two instruments for this experiment	Upper 95% CI of overall %GCV ≤ 25% and ≥ 80% samples with %GCV ≤ 20%	Upper 95% CI %GCV of ≤ 13.1% (repeatability) and ≤ 17.0% (intermediate precision) and ≥ 93% (repeatability) and > 88% (intermediate precision) samples had %GCV ≤ 20%
LLOQ verification	Clinical study samples (0.5× to 4× LLOQ) and assay reference standard	Overall %GCV ≤ 25% Spiked samples ≤ 3× LLOQ within ± 25% %RE	Overall %GCV ≤ 13.1% Spiked samples ≤ 3× LLOQ –16% to 24% %RE
Linearity/dilutability	Multiple determinations at 3–6 dilutions	Coefficient of determination > 0.95 and regression line slope 0.80–1.25	Coefficient of determination ≥ 0.9743 and regression line slopes 0.8010–1.1258

GCV geometric coefficient of variation, LLOQ lower limit of quantification, %RE percent relative error

Precision

A minimum of 58 positive serum samples for each analyte was assessed for precision. At least 2 analysts tested the panel of samples to generate 5–15 determinations for each sample over at least 3 runs executed over a period of at least 3 days. Data were generated from at least 2 MSD Sector S readers.

Accuracy

Accuracy was verified through a spike recovery study using US reference pertussis antiserum (human) Lot 3 for FIM, PT, and FHA and Lot 4 for PRN. These were spiked into a negative/low serum sample at 7 concentrations inclusive of high, mid, low, and within 3× LLOQ of the assay with a minimum of 5 determinations per concentration. The mean of all determinations for each sample was compared to the expected result for the sample.

Dilutability

A minimum of 10 serum samples was assessed for dilutability. Each sample was prepared at 3–6 dilutions to cover a wide range of the assay including a concentration approximately 3 times the estimated LLOQ. A minimum of 3 independent determinations were generated for each of the samples at each dilution.

Lower Limit of Quantification

A minimum of 16 serum samples that consisted of non-clinical, commercially obtained samples (Keystone Biologics) and incurred clinical study samples ranging from 0.5 to 4× LLOQ and a sample created using assay reference standard [US reference pertussis antiserum (human) Lot 3 and Lot 4] targeting the LLOQ of the assay were tested by a minimum of 2 analysts for a minimum of 10 independent determinations.

Specificity (Competition and Matrix Effects)

Competition studies were performed using 10 positive samples with 6 competing antigens (1 homologous and 5 heterologous) at multiple concentrations ≤ 20 µg/mL. When available, the concentration of homologous competitor demonstrating a minimum of 75% competition was selected to evaluate heterologous competitors. If the percent competition with homologous antigen was < 75% at all of the concentrations evaluated, or with a heterologous antigen was > 25% at the lowest concentration at which homologous competition was ≥ 75%, the effective dose 50 (ED₅₀) value of the heterologous antigen was compared to that of the homologous antigen.

Matrix effects were assessed by spike recovery of assay reference standard in the following matrices: hemolytic, lipidic, icteric, and a specimen known to contain unrelated antibodies. The assay reference standard spiked into each matrix at 6 concentrations over a wide range of the assay and the matrix without reference spike were tested at least 5 times at each spike concentration.

Comparison of the ECL-Based Multiplexed Assay to Historical In-House Single Antigen ELISAs

The results generated by the validated ECL-based multiplexed assay were compared to the results for the same samples generated by the historical in-house ELISAs used by Global Clinical Immunology at Sanofi Pasteur for assessing the antibody responses to each of the single antigens. Pre- and post-vaccination concentrations were analyzed for the same samples using each assay (PT, FHA, FIM, and PRN for the ECL assay and the historical in-house ELISAs). Subsequently, a concordance analysis was conducted to compare the results using each of the single antigen ELISAs and the ECL assay.

Comparison of the ECL-Based Multiplexed Assay to Two Commercial B. pertussis ELISAs

The results generated by the validated ECL-based multiplexed assay were compared to the results for the same samples generated by the two commercially available assays.

Pre- and post-vaccination concentrations were analyzed for the same samples using each assay (PT, FHA, FIM, and PRN for the ECL assay and B. pertussis antibodies for the GenWay and Demeditec assays).

Subsequently, a concordance analysis was conducted to compare the results using each of the two commercial kits and the ECL assay.

Statistical Analyses

The intra-assay precision (repeatability) and intermediate precision Geometric Coefficient of Variation (GCV) values (%) were evaluated using a mixed model and variance component analysis shown as follows:

$$y_{\mathit{ijk}}=\mu +{\alpha }_I+ru{n}_{j(I)}+re{p}_{k(j(I))}+{\epsilon }_{\mathit{ijk}}$$

where y_ijk is the observed result in log-scale, α_I $\left({\sum }_i{\alpha }_I=0\right)$ is the constant difference between the mean of Ith sample and mean of the panel (µ), while run and rep represent the independent runs of the ECL assay and replicates within each run, respectively. Additionally, α_I is a fixed effect in the mixed model and run and rep are both random effects following normal distributions.

$ru{n}_{j(I)}\sim N\left(0,{\sigma }_{\mathit{run}}^2\right)$, $re{p}_{k(j(I))}\sim N\left(0,{\sigma }_{\mathit{rep}}^2\right)$, ${\epsilon }_{\mathit{ijk}}\sim N\left(0,{\sigma }_{}^2\right)$.

The intra-assay and intermediate variance component can be estimated as ${\widehat{\sigma }}_{\mathit{rep}}^2+{\widehat{\sigma }}_{}^2$ and ${\widehat{\sigma }}_{\mathit{rep}}^2+{\widehat{\sigma }}_{\mathit{run}}^2+{\widehat{\sigma }}_{}^2$ where ${\widehat{\sigma }}_{\mathit{rep}}^2$,${\widehat{\sigma }}_{\mathit{run}}^2$ and ${\widehat{\sigma }}_{}^2$ are variance component estimators of ${\sigma }_{\mathit{rep}}^2$,${\sigma }_{\mathit{run}}^2$ and ${\sigma }^2$. Therefore, the intra-assay precision and intermediate precision %GCV can be calculated as:

$100\%\times \left({10}^{\sqrt{{\sigma }_{\mathit{rep}}^2+{\widehat{\sigma }}_{}^2}}-1\right)$ and $100\%\times \left({10}^{\sqrt{{\sigma }_{\mathit{rep}}^2+{\sigma }_{\mathit{run}}^2+{\widehat{\sigma }}_{}^2}}-1\right)$ , respectively.

To evaluate comparability (or concordance) of the assay, the orthogonal regression was used following errors-in-variables model [22]. The limit of agreement was estimated using Altman–Bland analysis [23–25].

Results

ECL-Based Multiplexed Assay Development and Validation

Table 2 provides a summary of the assay validation for each antigen. Individual parameter results are summarized below.

Table 2.

Validation summary results for PT, FHA, FIM, and PRN

Parameter	Antigen
	PT	FHA	FIM	PRN
Specificity: competition	Homologous competition > 75% for 100% of samples at competitor concentrations ≤ 5.0 µg/mL	Homologous competition > 75% for 100% of samples at competitor concentrations ≤ 1.25 µg/mL	Homologous competition > 75% for 100% of samples at competitor concentrations ≤ 5.0 µg/mL	Homologous competition > 75% for 100% of samples at concentrations ≤ 0.313 µg/mL
	Heterologous competition < 25% for 100% of samples at concentration of homologous competitor that achieved  ≥ 75% competition
Specificity: matrix effects	% recovery of the assay reference standard within 100% ± 20% at all levels and for all matrices evaluated
Accuracy	84% to 97%	85% to 111%	90% to 117%	90% to 100%
Precision	Repeatability: 12.4% (95% CI 11.8%, 13.1%) Intermediate precision: 13.9% (95% CI 10.1%, 17.0%)	Repeatability: 9.0% (95% CI 8.6%, 9.5%) Intermediate precision: 9.9% (95% CI 7.7%, 11.8%)	Repeatability: 12.1% (95% CI 11.6%, 12.9%): Intermediate precision: 11.8% (95% CI 10.6%, 13.1%)	Repeatability: 9.8% (95% CI 9.3%, 10.3%) Intermediate precision: 10.2% (95% CI 8.8%, 11.5%)
LLOQ verification	LLOQ 2.00 EU/mL with intermediate precision near LLOQ of 14.5% and relative error -16%	LLOQ 2.00 EU/mL with intermediate precision near LLOQ of 9.0% and relative error -12%	LLOQ 2.00 EU/mL with intermediate precision near LLOQ of 13.5% and relative error -4%	LLOQ 2.00 EU/mL with intermediate precision near LLOQ of 10.8%, and relative error -10
Linearity/dilutability	Coefficient of determination ≥ 0.9956 and regression line slope 0.9431–1.1258	Coefficient of determination ≥ 0.9926 and regression line slope 0.8714 to 1.0824	Coefficient of determination ≥ 0.9918 and regression line slope 0.9092 to 1.1057	Coefficient of determination ≥ 0.9940 and regression line slope 0.9467 to 1.0411

LLOQ lower limit of quantification

Precision

Repeatability: across the four pertussis antigens, the upper limits of the 95% confidence interval (CI) of the overall GCV were ≤ 13% and ≥ 93% of the samples had GCV ≤ 20%.

Intermediate precision: across the four pertussis antigens, the upper limits of the 95% CI of the overall GCV were ≤ 17% and ≥ 88% of the samples had GCV ≤ 20%.

Accuracy

All samples across the four pertussis antigens tested with concentrations > 3× LLOQ had percent recovery ranging from 80 to 114%. All samples tested that targeted concentrations near the LLOQ (2.00 EU/mL) had percent recovery ranging from 84 to 124%.

Dilutability

For each sample across the four pertussis antigens, R² was ≥ 0.9743 and the regression line slopes were in the range of 0.8010–1.1258.

Lower Limit of Quantification

Across the four pertussis antigens, the overall GCV of the samples evaluated for LLOQ verification was ≤ 13.1%. The relative error of spiked samples at or within 3xLLOQ was –16 to 24%.

Specificity (Competition and Matrix Effects)

For competition effects, percent competitions for heterologous competitors were ≤ 25% and homologous competitors were ≥ 75% at concentrations ≤ 20.0 µg/mL for all antigens and all samples.

For matrix effects, across the four pertussis antigens overall, 100% of samples for each matrix had percent recovery 83–119%.

Comparison of the ECL-Based Multiplexed Assay to Historical In-House Single Antigen ELISAs

As shown in Table 3, concordance analysis showed that the DTP-ECL and historical PT ELISA results had a concordance slope (90% CI) of 0.996 (0.963, 1.030), which was included in the interval 0.80–1.25, demonstrating no proportional bias. The estimated percent difference (90% CI) is − 14.1% (− 17.2, − 10.9) which is included in the interval (− 25%, 25%), and the linear relationship is shown graphically in Fig. 1. Similar results were obtained for concordance between the DTP-ECL and PRN and FIM ELISAs (Table 3; Fig. 1). Although the acceptance criteria for slope and percent difference for PT, FIM, and PRN were met, the acceptance criterion for percent difference (90% CI, − 25% to 25%) was not met for FHA, as the 90% CI of the percent difference was slightly out of range at 23.5% (19.1, 28.0) which was outside the acceptance limits of (− 25%, 25%). The acceptance criterion for slope for FHA was met as the concordance slope was 0.925 (90% CI, 0.900, 0.950), which was within the interval 0.80–1.25, demonstrating no proportional bias.

Table 3.

Summary of concordance between DTP-ECL and historical single antigen ELISAs

Antigen	na	Concordance slope (90% CI)	Percent difference (90% CI)
PT	311	0.996 (0.963, 1.030)	− 14.1% (− 17.2, − 10.9)
FHA	627	0.925 (0.900, 0.950)	23.5% (19.1, 28.0)
FIM	580	1.002 (0.984, 1.021)	19.0% (15.1, 23.0)
PRN	541	1.013 (0.995, 1.032)	9.6% (6.4, 12.8)

^aResults below lower limit of quantification were not included in the analyses CI confidence interval, FHA, filamentous hemagglutinin, FIM fimbrial agglutinogen, n number of subjects, PRN pertactin, PT pertussis toxin

Fig. 1.

Statistical linear relationship between the results of historical in-house single-antigen ELISAs vs. DTP-ECL. (A) Anti-PT IgG (EU/mL, (B) anti-FHA IgG (EU/mL), (C) anti-FIM IgG (EU/mL), (D) anti-PRN IgG (EU/mL). The Y-axes show the log₁₀-transformed results from the indicated in-house single-antigen IgG ELISA in EU/mL, whereas the X-axes show the log₁₀-transformed results for the indicated DTP-ECL antigen IgG in EU/mL

Comparison of the ECL-Based Multiplexed Assay to Two Commercial B. pertussis ELISAs

The precision of two commercial B. pertussis ELISAs, NovaLisa™ B. pertussis IgG ELISA from GenWay Biotech and B. pertussis IgG ELISA from Demeditec Diagnostics, was evaluated using the 10 individual concentrations of two reference sera (WWO-2-043 and NIBSC 06/140). Intra-assay precision [coefficient of variation (CV)] (Repeatability) for the GenWay and Demeditec assays was 7.3% (6.0, 9.3) and 32.8% (26.3, 43.5), respectively, and inter-assay (intermediate) precision was 13.3% (6.9, 93.1%) and 42.0% (26.0, 105.1).

In absolute percent difference against the assigned anti-PT and anti-FHA concentration, the ECL assay overall accuracy (% difference) against assigned ELISA concentrations for the reference sera [0% (PT) and 1% (FHA)] was more accurate than the GenWay [− 80% (PT) and -64% (FHA)] and Demeditec [37% (PT) and − 15% (FHA)] assays (Table 4). Similarly, the ECL assay was accurate when assessed using relative assignment from the same assay [% recovery 96–112% (PT) and 100–117% (FHA) for WWO-2-043; 82–93% (PT) and 88–108% (FHA) for NIBSC 06/140] (Table 5).

Table 4.

Accuracy assessment against assigned ELISA concentration for GenWay, Demeditec, and ECL assays

Internal sera sample panel for wP testing		ELISA reference assignment (EU/mL)		GenWay			Demeditec			ECL
				GMC (NTU)	% difference vs.		GMC (U/mL)	% difference vs.		GMC (EU/mL)		% difference vs.
Sample	Original Sample	PT	FHA		PT	FHA		PT	FHA	PT	FHA	PT	FHA
1	WWO-2–043 (100%)	151.92	112.26	24.5	− 84%	− 78%	139.3	− 8%	24%	130.8	93.8	− 14%	− 16%
2	WWO-2–043 (75%)	113.94	84.2	21.4	− 81%	− 75%	85.8	− 25%	2%	98.1	70.3	− 14%	− 17%
3	WWO-2–043 (50%)	75.96	56.13	16.9	− 78%	− 70%	57.4	− 24%	2%	73.5	51.1	− 3%	− 9%
4	WWO-2–043 (20%)	30.38	22.45	9.0a	− 70%	− 60%	19.6a	− 35%	− 13%	25.9	19.2	− 15%	− 14%
5	WWO-2–043 (1%)	1.52	1.12	0.6a	− 61%	− 47%	1.8a	19%	62%	1.3	1.1	− 17%	− 2%
6	NIBSC 06/140 (100%)	335	130	31.4	− 91%	− 76%	148.5	− 56%	14%	426.0	155.7	27%	20%
7	NIBSC 06/140 (75%)	251.25 ara>	97.5	27.6	− 89%	− 72%	113.5	− 55%	16%	298.7	110.9	19%	14%
8	NIBSC 06/140 (50%)	167.5	65	23.2	− 86%	− 64%	80.9	− 52%	24%	180.9	71.2	8%	10%
9	NIBSC 06/140 (20%)	67	26	14.2	− 79%	− 45%	25.2	− 62%	− 3%	69.6	27.5	4%	6%
10	NIBSC 06/140 (1%)	3.35	1.3	1.1	− 66%	− 13%	2.3	− 32%	75%	3.7	1.7	11%	29%
	Overall	–	–	–	− 80%	− 64%	–	− 37%	− 15%	–	–	0%	1%

ECL electrochemoluminescence, ELISA enzyme-linked immunosorbent assay, FHA filamentous hemagglutiin, GMC geometric mean concentration, PT pertussis toxin, wP whole-cell pertussis

^aBelow the cutoff value for the respective kits (< 11 NTU for GenWay and < 20 U/mL for Demeditec)

Table 5.

Accuracy assessment using relative assignment from the same assay for GenWay, Demeditec, and ECL assays

Internal sera sample panel for wP testing		Genway			Demeditec			ECL (PT)			ECL (FHA)
Sample	Original sample	Observed	Expected	Percent Recovery	Observed	Expected	Percent Recovery	Observed	Expected	Percent Recovery	Observed	Expected	Percent Recovery
1	WWO-2-043 (100%)	24.5	N/A	N/A	139.3	N/A	N/A	130.8	N/A	N/A	93.8	N/A	N/A
2	WWO-2-043 (75%)	21.4	18.4	116%	85.8	104.5	82%	98.1	98.1	100%	70.3	70.4	100%
3	WWO-2-043 (50%)	16.9	12.2	138%	57.4	69.6	82%	73.5	65.4	112%	51.1	46.9	109%
4	WWO-2-043 (20%)	9.0	4.9	185%	19.6	27.9	70%	25.9	26.2	99%	19.2	18.8	103%
5	WWO-2-043 (1%)	0.6	0.2	243%	1.8	1.4	130%	1.3	1.3	96%	1.1	0.9	117%
6	NIBSC 06/140 (100%)	31.4	N/A	N/A	148.5	N/A	N/A	426.0	N/A	N/A	155.7	N/A	N/A
7	NIBSC 06/140 (75%)	27.6	23.6	117%	113.5	111.4	102%	298.7	319.5	93%	110.9	116.7	95%
8	NIBSC 06/140 (50%)	23.2	15.7	148%	80.9	74.2	109%	180.9	213.0	85%	71.2	77.8	92%
9	NIBSC 06/140 (20%)	14.2	6.3	226%	25.2	29.7	85%	69.6	85.2	82%	27.5	31.1	88%
10	NIBSC 06/140 (1%)	1.1	0.3	361%	2.3	1.5	153%	3.7	4.3	88%	1.7	1.6	108%

ECL electrochemoluminescence, FHA filamentous hemagglutiin, N/A not applicable, PT pertussis toxin, wP whole-cell pertussis

For the assessment of observed versus expected concentration linearity, the slope (R²) for the ECL assay was 1.0278 (0.9920) (PT), 1.0132 (0.9959) (FHA), 1.0428 (0.9973) (PRN), and 1.0064 (0.9975) (FIM) for WWO-2-043 and 0.9296 (0.9955) (PT), 0.9481 (0.9991) (FHA), 0.9794 (0.9999) (PRN), and 0.9587 (0.9985) (FIM) for NIBSC 06/140 (Fig. 2). For the GenWay and Demeditec assays (wP), the values were 1.1245 (0.9680) and 0.8287 (0.9978) for WWO-2-043, and 1.0953 (0.9263) and 1.0464 (0.9938) for NIBSC 06/140 (Fig. 3). The R² values were therefore closer to 1 for the ECL assay than for the GenWay or Demeditec assays, although R² was greater than 0.95 in all cases except for the GenWay assay for NIBSC 06/140 (R² = 0.9263).

Fig. 2.

Assessment of linearity of ECL assay (anti-PT, anti-FHA, anti-PRN, and anti-FIM IgG EU/mL). (A) anti-PT ECL using WWO-2-043; (B) anti-PT ECL using NIBSC 06/140; (C) anti-FHA ECL using WWO-2-043; (D) anti-FHA ECL using NIBSC 06/140; (E) anti-PRN ECL using WWO-2-043; (F) anti-PRN ECL using NIBSC 06/140; (G) anti-FIM ECL using WWO-2-043; (H) anti-FIM ECL using NIBSC 06/140. The dashed red line shows the line of perfect linear agreement. The blue or green circles and the solid blue or green line show the actual data points and the associated trendline for observed versus expected results for the indicated DTP-ECL antigen using the indicated reference standard as the source of the samples

Fig. 3.

Assessment of linearity of GenWay and Demeditec assays. (A) GenWay ELISA using WWO-2-043; (B) GenWay ELISA using NIBSC 06/140; (C) Demeditec ELISA using WWO-2–043; (D) Demeditec ELISA using NIBSC 06/140. Results are reported in NTU (GenWay Units) for the GenWay ELISA and in arbitrary Units/mL (U/mL) for the Demeditec ELISA. The dashed red line shows the line of perfect agreement. The dashed red line shows the line of perfect linear agreement. The blue or green circles and the solid blue or green line show the actual data points and the associated trendline for observed versus expected results for the indicated commercial ELISA using the indicated reference standard as the source of the samples

For the ECL assay post-vaccination titers increased by 33.25-fold (anti-PT), 2.69-fold (anti-FHA), 142.51-fold (anti-FIM), and 12.79-fold (anti-PRN) compared to pre-vaccination concentrations. For the GenWay and Demeditec assays, post-vaccination concentrations (wP) were 1.58-fold and 6.94-fold higher than pre-vaccination (Table 6).

Table 6.

Pre- and post-vaccination geometric mean concentration and geometric mean concentration ratio for subjects who received wP vaccine

Assay	Antigen	Geometric mean concentration (n)
		Pre-vaccination GMC	Post-vaccination GMC	GMC ratio
ECL	PT	3.9 (36)	128.7 (36)	33.25
	FHA	10.1 (33)	27.1 (31)	2.69
	FIM	8.5 (36)	1213.1 (35)	142.51
	PRN	2.1 (36)	27.5 (36)	12.79
GenWay	wP	9.9 (34)	15.7 (29)	1.58
Demeditec	wP	2.7 (35)	19.1 (36)	6.94

FHA filamentous hemagglutinin, FIM fimbrial agglutinogen, n number of subjects, PRN pertactin, PT pertussis toxin, wP whole-cell pertussis

The concordance analysis performed between the ECL assay and the GenWay and Demeditec assays is summarized in Table 7. Between the ECL and GenWay assays, the concordance slope obtained was 0.510 (anti-PT), 1.230 (anti-FHA), 0.343 (anti-FIM), and 0.672 (anti-PRN) (Fig. 4); for the ECL and Demeditec assays, the concordance slope was 0.207 (anti-PT), 0.636 (anti-FHA), 0.084 (anti-FIM), and 0.188 (anti-PRN) (Fig. 5). While the concordance of the ECL assay was slightly better with the GenWay assay than the Demeditec assay, neither commercial assay showed perfect concordance to the ECL assay. As a result of the differences in assay parameters, as well as antigens against which antibody responses were measured, it was not expected that the DTP-ECL assay would have good concordance with the commercial assays.

Table 7.

Summary of concordance between DTP-ECL and GenWay and Demeditec wP ELISAs

Antigen	Concordance with GenWay			Concordance with demeditec
	n	Concordance slope (90% CI)	Percent difference (90% CI)	n	Concordance slope (90% CI)	Percent difference (90% CI)
PT	99	0.510 (0.472, 0.551)	− 64.86% (− 72.22, − 57.08)	91	0.207 (0.157, 0.273)	− 19.46% (− 40.85, 9.67)
FHA	88	1.230 (0.978, 1.547)	− 60.84% (− 66.80, − 53.82)	81	0.636 (0.549, 0.738)	− 19.61% (− 28.21, − 9.98)
FIM	97	0.343 (0.302, 0.389)	− 92.36% (− 94.58, − 89.23)	89	0.084 (0.049, 0.145)	− 82.94% (− 89.46, − 72.37)
PRN	99	0.672 (0.566, 0.796)	− 10.27% (− 25.24, 7.69)	91	0.188 (0.109, 0.324)	77.33% (37.79, 128.22)

Fig. 4.

Concordance analysis between ECL and GenWay assays. (A) anti-PT; (B) anti-FHA; (C) anti-FIM; (D) anti-PRN. Results are log₁₀-transformed results reported in NTU (GenWay Units) for the GenWay ELISA and in EU/mL for the DTP-ECL. The red circles represent the data points for pre-vaccination sera and the blue squares represent the data points for post-vaccination sera. The dashed blue line shows the line of perfect agreement, and the solid red line shows the associated trendline for the combined data points for all samples

Fig. 5.

Concordance analysis between ECL and Demeditec assays. (A) anti-PT; (B) anti-FHA; (C) anti-FIM; (D) anti-PRN. Results are log₁₀ transformed results reported in arbitrary Units/mL (U/mL) for the Demeditec ELISA and in EU/mL for the DTP-ECL. The red circles represent the data points for pre-vaccination sera and the blue squares represent the data points for post-vaccination sera. The dashed blue line shows the line of perfect agreement, and the solid red line shows the associated trendline for the combined data points for all samples

The range of post- and pre-vaccination concentrations was wider for the ECL assay (anti-PT, anti-FHA, anti-FIM, and anti-PRN) than for the GenWay and Demeditec assays (anti-wP) (Fig. 6).

Fig. 6.

Pre- and post-vaccination antibody concentration distribution for ECL assay (anti-FHA, anti-PT, anti-FIM, and anti-PRN), GenWay (anti-wP) assay, and Demeditec (anti-wP) assay. (A) anti-PT IgG ECL (EU/mL), (B) anti-FHA IgG ECL (EU/mL), (C) anti-FIM IgG ECL (EU/mL), (D) anti-PRN IgG ECL (EU/mL), (E) anti-wP IgG GenWay ELISA (NTU) and 6F anti-wP IgG Demeditec ELISA (Units/mL). The pre-vaccination concentration distribution is shown in blue for the ECL methods or green for the anti-wP ELISAs and the post-vaccination concentration distribution is shown in red for all

Discussion

Commercially available pertussis diagnostic kits are frequently used to assess the antibody response to wP pertussis vaccines in clinical trials [26–28]. However, having been developed for diagnostic purposes, these kits are not suited for the precise, accurate, and reproducible evaluation that is required to quantify vaccine responses in clinical trials. The commercial kits, being pre-coated with an undefined mixture of pertussis antigens, do not enable the assessment of antibody responses to specific antigens with established protective importance, and are not calibrated against established international reference standards. As such, the diagnostic assays tend to be qualitative but not quantitative in nature, and so the results are not considered to be well suited for use in clinical trials. Other crucial considerations include the assay antigen target (bacterial lysate or purified or semi-purified antigens) and the lot-to-lot variability of the antigen batches used to coat the enzyme immunoassay plates distributed by each assay manufacturer. Previous reports have documented that the assay antigen target is critical to assay performance, and have indicated that the use of purified PT as the coating antigen for ELISAs resulted in superior diagnostic performance than a mixture of B. pertussis-derived antigens [29, 30]. Such elements should be taken into consideration when analyzing and interpreting serological results obtained from clinical trials using vaccines containing diphtheria, tetanus, and wP antigens and commercial assays. The improved performance of purified single-antigen ELISAs or multiplexed immunoassays using purified antigens over ELISAs with mixed pertussis-coating antigens for laboratory assessment of anti-pertussis antibodies in response to vaccination or infection has also been demonstrated by others [29, 30].

For these reasons, we recently developed and validated a new multiplexed ECL assay for the measurement of anti-PT, ant-FHA, anti-FIM, and anti-PRN IgG antibodies, in accordance with WHO guidance [31, 32], and with other international guidance documents related to analytical assay validation [33–35], for use in the clinical development of aP and wP vaccines at Sanofi Pasteur. The ECL assay was validated for precision, accuracy, dilutability, LLOQ, and specificity, and demonstrated acceptable performance for all parameters evaluated.

The new multiplexed ECL assay generated similar results compared to the currently validated in-house ELISAs. Although the acceptance criterion was not met for FHA, the variance is probably due to results in the low range of the assay. The clinical significance is therefore considered to be minimal and thus the validated ECL assay is considered suitable for its intended use to evaluate pertussis antibodies in clinical samples. The use of a multiplexed immunoassay for assessment of antibodies against B. pertussis antigens, diphtheria, and tetanus using Luminex technology has also been previously described [36, 37], and correlation with single-antigen ELISAs using purified antigens has been demonstrated [36].

In comparison to the two commercially available assay kits, the ECL assay showed greater accuracy and better linearity of the observed versus the expected responses. The accuracy of the ECL assay was maintained following sample dilution, but was less so for the Demeditec assay and still less for the GenWay assay. Additionally, the ECL assay could better distinguish between pre- and post-vaccination samples than the comparator assays, and the concordance between the ECL and each commercially available comparator assay was poor. These results further indicated the suitability of the ECL assay for the evaluation of clinical samples.

The ECL method performed comparably or better than the commercial kits and is more suitable for the evaluation of anti-pertussis antibody responses in vaccine development. It has been clinically validated for the simultaneous analysis of anti-PT, ant-FHA, anti-FIM, and anti-PRN IgG antibodies, and exhibits a broader dynamic range across all age groups tested than the commercially available semi-quantitative assay kits. This could be a necessary criterion in understanding the full range of vaccine-induced immune responses. The accuracy, sensitivity, and linearity of the assay were demonstrated using calibrated international reference serum, which allowed the quantitative evaluation of IgG concentrations. For the ECL assay, LLOQs are defined, compared to arbitrarily defined thresholds for the commercial kits, and the fully quantitative method can be used to accurately measure changes in antibody concentrations from pre- to post-vaccination.

Limitations

This manuscript describes the validation of a multiplexed immunoassay and comparisons between that multiplexed assay to single-antigen in-house ELISAs and two commercial ELISAs. The commercial ELISAs evaluated in this study used mixed B. pertussis-derived antigens as coating antigens. It is possible that comparisons to other commercial ELISAs, particularly those coated with purified single antigens, could have been different.

Conclusions

A new ECL assay was validated for the quantitative evaluation of anti-PT, ant-FHA, anti-FIM, and anti-PRN antibodies in samples from clinical trials, and performed better than commercially available assays, coated with mixed pertussis antigens, developed for pertussis diagnostic purposes. The preferential use of this assay is warranted for the determination of immune responses to pertussis containing vaccines in clinical trials.