Validation of a diphtheria toxoid multiplex bead assay for serosurveys

Abstract

We validated a multiplex bead assay for diphtheria toxoid IgG antibodies against the Vero cell toxin neutralization test using 1,300 specimens (correlation=0.88). At the ≥0.01 IU/ml cutoff for minimal seroprotection, sensitivity was 95% and specificity was 83%. Agreement for three categories (<0.01, 0.01–<0.1, ≥0.1 IU/ml) was 81% (kappa=0.71).

Diphtheria is a potentially fatal, toxin-mediated bacterial illness. Generally, greater antibody levels from vaccination and, in some cases, natural infection result in improved clinical protection and longer duration of immunity [1]. Serosurveys of IgG antibodies against diphtheria are frequently used for monitoring national immunization program performance [1].

The Vero cell toxin neutralization test (TNT), although labor-intensive and expensive, is considered the gold standard for assessing diphtheria immunity [1]. Antibody concentrations of 0.01 International Units (IU)/mL provide minimal protection against death; 0.1 IU/mL is protective against symptomatic disease; and ≥1.0 IU/mL is associated with long-term protection [1]. Commercial diphtheria ELISA options exist, but non-specific binding at low seroprotective levels requires the use of a higher (0.1 IU/ml) assay cutoff [1]. Multiplex bead assays (MBAs) allow assessment of antibody responses to multiple antigens, creating resource-savings; several diphtheria MBAs have been validated and used in national serosurveys [2–5].

A Luminex^®-MBA for various viral, bacterial, and parasitic diseases developed at the Centers for Disease Control and Prevention (CDC) in Atlanta, USA has previously been described [6, 7]. Here, we describe validation of a diphtheria toxoid component of the MBA against the Vero cell TNT for assessing diphtheria IgG antibodies in serosurveys.

Deidentified specimens from a national serosurvey of persons aged 1–24 years in Tajikistan, all with consent to long-term storage and future testing, were used for the validation [8]. This activity was determined by CDC to be non-research, not requiring further ethics review. Samples were assayed using the Vero cell TNT at Public Health England in London, UK, as previously described [8]. Results from two-fold sample dilutions were reported ranging from <0.16 to >8 IU/ml. TNT values <0.016 IU/ml were considered negative [9, 10]. From the 2,459 eligible specimens [8], 1,400 were randomly selected across four strata (n=350 for <0.016; 0.016–<0.10; 0.10–<1.0; ≥1.0 IU/ml). Of these, 100 (7%) samples had insufficient volume or issues with identification numbers.

A detailed protocol for the coupling of antigens to SeroMap beads (Luminex Corp., Austin, TX) has been described [11]. Diphtheria toxoid (60 μg) (List Biological Laboratories, Campbell, CA) and a negative control of recombinant Schistosoma mansoni glutathione-S-transferase (GST) (15 μg) [12] were each coupled to 1.25 × 10⁷ beads in a 1 ml volume of buffer containing 50 mM 2-(N-morpholino)ethanesulfonic acid (MES) with 0.85% NaCl at pH 5.0 [11–13]. Serum dilution (1:400) and assay conditions for the detection of bound IgG antibodies have also been described [11]. Samples were run in duplicate at CDC, median fluorescent intensity minus background (MFI-bg) values were determined, and positive values with coefficients of variation (CV) >15% were repeated as previously described [11].

The WHO International Standard for Diphtheria Antitoxin (10/262; 2 IU/ml; National Institute for Biological Standards and Control, Potters Bar, UK) was diluted 1:400, and 2-fold serial dilutions were made to generate a 10-point standard curve. BioPlex Manager software (version 6.2; Bio-Rad, Hercules, CA) generated the following 5-parameter logistic curve equation for the interconversion of MFI-bg and IU/ml values: MFI-bg = 51.7565 + (27448.3 – 51.7565) / [1 + (Conc / 0.18275)^−4.39825]^0.333019. BioPlex software determined the lower and upper limits of quantitation for the MBA as 0.004 and 0.226 IU/ml, respectively (Supplementary Figure 1); lower and upper limits of detection (LOD) were 0.0001 and 0.99 IU/ml from the single dilution used in the MBA. Samples with responses below or above the LOD were assigned imputed values of 0.0001 or 0.99 IU/ml, respectively.

A linear regression plot of the logarithm of MBA and TNT values (Figure 1A) was created with Prism version 8.4.3 (GraphPad Software, San Diego, CA). The slope of the line was 0.80 (95% CI: 0.77–0.84) and intercept was −0.48 (95% CI:−0.52– −0.44). The Spearman’s rank correlation coefficient (SigmaPlot 13.0, Systat Software, Inc., San Jose, CA) was 0.88, which is “high correlation” with TNT results [9, 10, 14, 15].

Figure 1. Scatterplots of the logarithm of diphtheria toxoid antibody levels (n=1,300 specimens) by multiplex bead assay (MBA) vs. a reference toxin neutralization test (TNT).

(A) Inter-assay comparison of diphtheria antitoxin levels by MBA and TNT by linear regression (solid line) using values ranging 0.016 to 1.024 IU/ml by TNT (n=789). The line of identity (dashed line) is shown for reference. Spearman’s rank correlation coefficient (R), the slope of the regression (S), and the y-intercept (D) were calculated. (B) Points are jittered to better visualize the distribution of MBA responses relative to individual titers in the TNT. Median MBA responses for each TNT antibody titer are shown in red. Cutoffs at 0.01 IU/ml (dashed) and 0.1 IU/ml (solid) are shown in blue.

Sensitivity and specificity of the MBA at the ≥0.01 IU/ml cutoff was calculated using SAS v9.4 (Cary, NC) compared with TNT using the 0.016 IU/ml cutoff. MBA sensitivity was 95% (95%CI: 94%–96%) and specificity was 83% (95%CI: 79%–87%). Receiver-operating characteristic analysis (SigmaPlot 13.0) yielded an area under the curve of 0.97 and an optimized MBA cutoff of 0.015 IU/ml (Supplementary Figure 2). Use of a ≥0.015 IU/ml cutoff resulted in an MBA sensitivity of 91% (95%CI: 89%–93%) and specificity of 92% (95%CI: 89%–95%).

A comparison of TNT and MBA results (n=1,300) is shown in Figure 1B and Table 1 [9, 10]. The overall agreement was 81% (95% CI: 79%–83%) using the 0.01 IU/ml cutoff for the MBA. The three-way kappa score calculated using GraphPad QuickCalcs (San Diego, CA) [16] was 0.71 (95% CI: 0.68–0.74), which is considered good agreement [9, 10]. Using the 0.015 IU/ml MBA cutoff increased the number of correct assignments for the TNT <0.016 IU/ml category but decreased the number of correct assignments for the 0.016–<0.10 IU/ml category (Table 1).

Table 1.

Diagnostic agreement by Cohen’s kappa score for the reference diphtheria toxin neutralization test (TNT) vs. the multiplex bead assay (MBA)

Antibody level (IU/ml)		TNT			% agreement (95% CI)	Kappa score (95% CI)
		≥0.1 (n=659)	0.016 to <0.1 (n=312)	<0.016 (n=329)
MBA	≥0.1	516	5	2	81% (79%–83%)	0.71 (0.68–0.74)
	0.010 to <0.1	139	264	54
	<0.01	4	43	273
	≥0.1	516	5	2	80% (78%–82%)	0.70 (0.66–0.73)
	0.015 to <0.1	139	224	23
	<0.015	4	83	304

When the proportion seroprotected for the serum set was calculated at the ≥0.016 IU/ml cutoff for TNT, seroprotection was 75% (95% CI: 72%–77%), compared with 75% (95% CI: 73%–78%) for the ≥0.01 IU/ml cutoff and 70% (95% CI: 67%–72%) for ≥0.015 IU/ml in the MBA. Because of the slightly improved kappa score and better correlation of seroprotection results, we have chosen to use the unadjusted 0.01 IU/ml cutoff for the MBA going forward. At the 0.1 IU/ml cutoff, seroprotection was 51% (95% CI: 48%–53%) by TNT and 40% (95% CI: 38%–43%) by MBA, demonstrating that seroprotection using the 0.1 IU/ml cutoff is likely to be underestimated with the MBA (Table 1).

Diphtheria MBAs have been shown previously to have superior performance to most ELISAs when compared with TNT [9, 10]. With all binding assays (e.g., MBA, ELISA), it is possible to detect non-neutralizing antibodies to the toxoid [1], including to vaccines containing CRM197 genetic mutants of diphtheria toxin. The diphtheria MBA described here has already been used for multi-antigen serosurveys in several developing countries [17–19] and should be broadly useful as part of integrated monitoring of public health program impact where use of the TNT would be cost prohibitive and neutralization results are not required [6].

Highlights.

• We developed and validated a multiplex bead assay (MBA) for diphtheria IgG antibodies

• MBA values correlated well (R=0.88) with the toxin neutralization test (TNT)

• MBA sensitivity was 95% and specificity was 83%, compared with TNT (≥0.01 IU/ml)

• MBAs including diphtheria facilitate integrated serosurveys and program monitoring