Performance of Commercial Enzyme-Linked Immunosorbent Assays for Detection of Antibodies to Bordetella pertussis

Abstract

Measuring antibodies to Bordetella pertussis antigens is mostly done by enzyme-linked immunosorbent assays (ELISAs). We compared the performance of ELISA kits that were commercially available in Germany. Eleven measured IgG antibodies, and nine measured IgA antibodies. An in-house ELISA with purified antigens served as a reference method. Samples included two WHO reference preparations, the former Food and Drug Administration (FDA)/Center for Biologics Evaluation and Research (CBER) reference preparations, serum samples from patients with clinically suspected pertussis, and serum samples from patients having received a combined tetanus, diphtheria, and pertussis (Tdap) vaccination. Kits using pertussis toxin (PT) as an antigen showed linearity compared to the WHO Reference preparation (r² between 0.82 and 0.99), and these kits could quantify antibodies according to the reference preparation. ELISA kits using mixed antigens showed no linear correlation to the reference preparations. Patient results were compared to results of in-house ELISAs using a dual cutoff of either ≥100 IU/ml anti-PT IgG or ≥40 IU/ml anti-PT IgG together with ≥12 IU/ml anti-PT IgA. The sensitivities of kits measuring IgG antibodies ranged between 0.84 and 1.00. The specificities of kits using PT as an antigen were between 0.81 and 0.93. The specificities of kits using mixed antigens were between 0.51 and 0.59 and were thus not acceptable. The sensitivities of kits measuring IgA antibodies ranged between 0.53 and 0.73, and the specificities were between 0.67 and 0.94, indicating that IgA antibodies may be of limited diagnostic value. Our data suggest that ELISAs should use purified PT as an antigen and be standardized to the 1st International Reference preparation.

Pertussis continues to be a frequently occurring disease irrespective of effective childhood vaccination. In vaccinating countries, such as the United States, the disease now occurs mainly in young unvaccinated infants, schoolchildren, adolescents, and adults (13). Due to atypical symptoms in older vaccinated children, adolescents, and adults, a laboratory confirmation is often required, and this can be done by detecting Bordetella DNA by PCR or Bordetella pertussis-specific antibodies by enzyme-linked immunosorbent assay (ELISA) or by multiplexed immunoassays (13, 16, 19). ELISAs for pertussis antigens have been employed in acellular vaccine trials and in seroepidemiological studies, and they are used for diagnostic purposes in various countries.

ELISAs can be done with purified or mixed antigens, of which only pertussis toxin (PT) is specific for B. pertussis (13). Studies have focused mostly on antibodies of isotype IgG, and the role of antibodies of isotype IgA especially in a vaccinated population is not totally clear (10, 13). Concentrations of antibodies to Bordetella antigens can be measured quantitatively and expressed in international units per ml (IU/ml) according to the 1st International Standard preparation (24).

The interpretation of results is based on dual-sample serology with an increase in antibody concentration or rarely also a decrease in antibody concentration (7, 13). In clinical practice, however, pertussis diagnosis is made mostly by single-sample serology using a single or a dual cutoff. Single-sample serology together with PCR was recently found to be the most sensitive method for diagnosing pertussis (1).

An urgent need for standardization of ELISAs has recently been stressed in a meeting convened by the CDC (18). Standardization has been performed for in-house ELISAs in two international collaborative studies (12, 24). The CDC has recently developed and validated a PT ELISA for use in public health laboratories (14). In the United States, Food and Drug Administration (FDA)-cleared commercial ELISA systems for pertussis serology are not available. In contrast, various commercial ELISA systems are registered in the European Union. Apart from diagnostics for some infectious diseases (i.e., HIV, hepatitis C virus), producers of diagnostic ELISA kits in the European Union have to submit documents describing the robustness, technical sensitivity, and specificity of the kits, they have to include a package insert, and they need to maintain a quality management system. The kits are then CE marked without further testing, and they can be distributed in all European Union countries (5). Apart from Germany, commercial pertussis ELISAs evaluated in this study are distributed throughout Europe and in many countries worldwide.

Ten years ago, a comparison of commercially available ELISAs in Germany showed insufficient sensitivity, specificity, and robustness (11). We chose to compare all ELISA kits that were commercially available and used in Germany in 2008 in relation to the recently available International Reference preparation (24).

MATERIALS AND METHODS

Serum samples. (i) Reference material.

Table 1 shows the declared content of the reference preparations used in this study. Since 2009, the WHO 1st International Standard Pertussis Antiserum (Human) (NIBSC code 06/140), and a working reference with an anti-PT IgG content close to the suggested cutoff [WHO 1st Reference Reagent Pertussis Antiserum (Human), NIBSC code 06/142] have been available. The measurement units, IU/ml, are the same as the former ELISA units/ml (EU/ml) that were defined for reference preparations (pooled sera) 3, 4, and 5 from the Center for Biologics Evaluation and Research (CBER)/FDA, Bethesda, MD, which were broadly used before the WHO preparations were available. Preparation 3 contained IgG and IgA antibodies to PT and filamentous hemagglutinin (FHA). Preparation 4 was meant for antibodies to pertactin (PRN), and preparation 5 contains higher levels of IgA antibodies to PT, FHA, and PRN. Both WHO reference preparations were used undiluted and in a 1:10 dilution in the study.

TABLE 1.

Antibody concentrations of the reference preparations used

Reference prepn	Concna
	Anti-PT IgG	Anti-PT IgA	Anti-FHA IgG	Anti-FHA IgA
WHO Reference: 1st International Standard	335	65	130	65
NIBSC working reference	106	18	122	86
CBER/FDA reference 3	200	15	200	100
CBER/FDA reference 5		140		280

^aIU/ml for WHO preparations; EU/ml for CBER/FDA preparations.

(ii) Patient sera.

A total of 57 patient serum samples were selected as follows: 24 samples were from patients with recent contact to bordetellae (B. pertussis PCR [IS481]-positive samples and/or samples that showed a titer increase between acute and convalescent samples). Another 24 samples were from patients with clinical symptoms (more than 2 weeks of coughing) but without laboratory evidence of recent contact with bordetellae (PCR-negative samples, with all antibodies below cutoff values). Eight serum samples were from a vaccine study of adolescents with a Tdap vaccine (10), and one sample contained very low or unmeasurable antibodies to pertussis antigens and was also used in the validation study for the WHO reference preparation (24).

ELISAs.

Kits were selected according to their availability on the market in Germany, as assessed by an external quality control scheme (INSTAND) in 2008, and purchased from the manufacturers. Table 2 displays information about the ELISA kits tested, as well as their intra-assay variation. The manufacturers were asked to give information about the antigen(s) used when such information was not specified in the package insert.

TABLE 2.

Composition of commercial ELISAs and intra-assay variation of IgG antibodies to pertussis antigens

Producer	Name	Antigen(s)a	Isotype(s)	Unit of measurement	% CVb
					High	Low
Euroimmun	Bordetella IgG/IgA	PT	IgG/IgA	U/ml	10	6
Virotech	Bordetella IgG/IgA	PT and FHA	IgG/IgA	Arbitrary VE/ml	3	2
Virotech	Pertussis Toxin IgG/IgA	PT	IgG/IgA	Arbitrary VE/ml	5	1
Dr.Merk & Kollegen	ELIMMUN IgG/IgA	PT and FHA	IgG/IgA	Arbitrary U/ml	10	2
Virion\Serion	Pertussis IgG/IgA	PT and FHA	IgG/IgA	U/ml	3	5
Virion\Serion	Pertussis Toxin IgG	PT	IgG	U/ml	3	5
Trinity Biotech	Bordetella pertussis ELISA	PT and FHA	IgG,A,M	Arbitrary U/ml	4	14
Siemens Novagnost	Novagnost Bordetella pertussis IgG/IgA	PT and FHA	IgG/IgA	Arbitrary U/ml	4	11
Novatec	NovaLisa Pertussis IgG/IgA	PT and FHA	IgG/IgA	Arbitrary U/ml	1	2
IBL	Bordetella pertussis IgG/IgA	?	IgG/IgA	Arbitrary U/ml	19	11
MP Biomedicals	Bordetella pertussis IgG/IgA	?	IgG/IgA	Arbitrary U/ml	2	1

^a?, not declared by producer.

^b% CV high, intra-assay CV of 1st International Standard; % CV low, intraassay CV of NIBSC working reference; VE, Virotech units.

Commercial ELISAs were bought from Euroimmun AG, Lübeck; IBL Gesellschaft für Immunchemie und Immunbiologie GmbH, Hamburg, Labor Dr.Merk & Kollegen GmbH, Ochsenhausen, MP Biomedicals Germany GmbH, Eschwege, Novatec Immundiagnostika GmbH, Dietzenbach, Siemens Healthcare Diagnostics Verwaltung GmbH, Eschborn, Trinity BioTech GmbH, Lemgo, Institut Virion\Serion GmbH, Würzburg, and Genzyme Virotech GmbH, Rüsselsheim, all in Germany.

The ELISAs were performed manually according to the instructions given in the package inserts. The package inserts were evaluated in respect to basic ELISA procedures, such as washing steps, incubation time and temperature, reading conditions, calculation, and interpretation of results. Washing steps for microtiter plates were done with a TECAN Columbus microplate washer according to the instructions given in the package inserts. Microtiter plates were read by an e-max reader (Molecular Devices GmbH, Ismaning, Germany) with a dual-wavelength program according to the instructions given in the package inserts. If the tests used a calibration curve, this was calculated by software (SoftMax Pro 5.0; Molecular Devices) employing a 4-parameter logistic. For ELISAs using a single-point calibration, this was done by the formula suggested in the package insert. Results were regarded as valid when all criteria set by the manufacturer were met. Quantitative results of antibody concentrations were interpreted according to the ELISA's package insert as being negative, positive, or, if this option was offered by the manufacturer, indeterminate.

Reference in-house ELISA.

In-house ELISAs with purified PT or FHA measuring isotypes IgG and IgA (22) were used as a reference system. In brief, these ELISAs used purified PT and FHA, kindly provided by GSK Biologicals SA, Rixensart, Belgium, and employed alkaline phosphatase-conjugated goat anti-human IgG and IgA antibodies (Kierkegard & Perry Laboratories, Gaithersburg, MD) and were adapted from ELISA systems standardized during the acellular vaccine trials (7). Eight dilutions for every sample were used, and the results were calculated by a four-parameter logistic. The ELISAs were standardized to the 1st WHO reference preparation for pertussis antibodies.

Definition of cutoff values.

Various cutoff values for anti-PT IgG in adolescents and adults have been proposed, and they are summarized in Table 3. In Massachusetts a high specificity was obtained using approximately ≥200 EU/ml anti-PT IgG (≥20 μg/ml) (25). In the Netherlands, a cutoff of ≥125 EU/ml anti-PT IgG with higher specificity and a cutoff of ≥62 EU/ml with slightly lower specificity have been used (4, 20) In Germany, a cutoff of approximately ≥40 EU/ml anti-PT IgG together with approximately ≥40 EU/ml anti-FHA IgA was suggested (23). A seroepidemiological survey in Europe used a cutoff of ≥125 EU/ml, and these data may be used as a population-based reference (15). A serosurveillance of U.S. sera modeled three separate populations according to their levels of anti-PT IgG: a higher cutoff was estimated at ≥94 EU/ml, and a lower cutoff was estimated at ≥49 EU/ml (2, 3). These data suggested that values between 49 EU/ml and 94 EU/ml may be regarded as indeterminate and would need further confirmation. Using a ≥50-IU/ml cutoff may be especially useful in outbreak situations, where higher sensitivity is required (9). A recent summary from Denmark, Netherlands, and the United Kingdom comparing receiver operating characteristics (ROC) curves found that values between 60 IU/ml and 75 IU/ml gave the best single cutoff estimate (6a).

TABLE 3.

Suggested cutoff values for anti-PT IgG for adolescents and adults

Location	Type of study	Cutoff (IU/ml)	Sensitivity (%)	Specificity (%)	Reference
United States (MA)	Population study	∼200	67	99.9	25
Netherlands	Population study	125	70	99	4
		62	80	95	4
Germany	Population study	40	80	95	23
European Union	Epidemiological survey	125	NAa	NA	15
United States	Epidemiological survey, model	94	NA	NA	2
		49	NA	NA	2
Australia	Clinical validation	50	NA	NA	9

^aNA, not applicable.

As a consequence, results of the antibody concentration in patient sera were interpreted according to an algorithm described in Fig. 1. As no standardized cutoffs for anti-PT IgA are available, the lower limit of quantitation (LLQ) for the anti-PT IgA ELISA was used, which is 12 IU/ml.

FIG. 1.

Diagnostic algorithm for evaluation of anti-PT IgG in patient sera.

Statistical analysis.

A linear regression of quantitative values of anti-PT IgG was constructed using SigmaPlot software (SPSS Science Software GmbH, Erkrath, Germany).

RESULTS

Handling and package inserts.

All kits were CE marked when they were bought and fulfilled the formal criteria from the European Union legislation for in vitro diagnostic kits (5). Kits of one company (MP Biomedicals) were primarily sent without package inserts. Some package inserts were not clear in their descriptions of critical steps such as the number of washings and the soak times. Some package inserts cited only references dated before 1990. Some microplates were coded only on the envelope and not on the microtiter plate itself, which may result in a mix-up of isotypes and creates results that are not machine readable.

Antigen composition and calibration.

Table 2 summarizes the composition of the tests. Many ELISAs used a nonspecified mixture of PT and FHA, and some used purified PT from different sources. Results of some kits were measured in “FDA” units/ml, meaning that the kits were meant to be calibrated to a reference preparation from the CBER/FDA and are using measurements in EU/ml. The CBER/FDA reference preparations, however, declare values only for antibodies to purified antigens and cannot be used for a mixture of antigens. Some manufacturers used arbitrary units, or a ratio between a positive control and the sample, which was then converted into arbitrary units.

Results of reference preparations.

Results of quantitative pertussis serology can be expressed in IU/ml (24), and these units correspond to those of reference preparations of CBER/FDA, which were standardized in ELISA units/ml (EU/ml). Expressing results in IU/ml requires that the kits contain purified antigens and no antigen mixture, such as PT plus FHA. Figure 2 shows the results for kits that expressed concentrations of anti-PT IgG antibodies in IU/ml or EU/ml. The values of the Virotech PT ELISA were converted into IU/ml as suggested by the kit's insert. The figure shows that results of the in-house ELISA and two commercial kits showed a linear correlation to the expected values from the reference preparations. The regression coefficients (r²) were 0.99 for the in-house ELISA (y = 0.93x + 6.3), 0.99 for the Euroimmun ELISA (y = 0.71x + 3.0), 0.98 for the Virotech PT ELISA (y = 0.92x + 16.7), and 0.82 for the Virion\Serion PT ELISA (y = 0.30x + 28.4).

FIG. 2.

Linearity of anti-PT IgG concentrations as measured by ELISA kits and compared to expected values of reference preparations.

Kits containing a mixture of antigens showed a skewed relation to the reference preparations (WHO reference, WHO reference diluted 1:10; WHO working reference, WHO working reference diluted 1:10; CBER/FDA reference 3), and no linear regression could be constructed.

Figure 3 compares the linearity of ELISAs measuring anti-PT IgA and expressing results in IU (EU)/ml. It shows that the in-house ELISA and one commercial kit showed sufficient linearity (r² = 0.99 and 0.98, respectively).

FIG. 3.

Linearity of anti-PT IgA concentrations measured by ELISA kits and compared to expected values of reference preparations.

In order to test for positioning effects on the microtiter plates, reference preparations were analyzed in duplicate in rows 2 and 12 of the microtiter plates. No relevant position effects were noted for any kits except for both Virion\Serion kits, which showed a nonsignificant tendency for lower values in row 12.

Table 2 shows that intra-assay variations of reference preparations were acceptable for all ELISAs, and they varied between 1% and 19%.

Results of patient sera.

Patient sera were evaluated according to the diagnostic algorithm described in Fig. 1. Table 4 shows the sensitivity and specificity estimates for IgG antibodies compared to those of the in-house ELISA. The sensitivity of all kits was good (84% to 100%), but the specificity estimates of kits using a mixture of PT and FHA were mostly unacceptable (51% to 81%). Some kits using PT as the antigen used 10 to 20 IU (or EU, respectively)/ml as a cutoff, whereas the cutoff used for the in-house ELISA was 40 IU/ml. The interpretation of these kits was thus also done with a 40 IU/ml cutoff, which increased the specificity estimates (Table 4). All kits detected the IgG antibodies induced by vaccination with a Tdap vaccine in adolescents.

TABLE 4.

Sensitivity and specificity of commercial IgG ELISAs compared to an anti-PT IgG in-house ELISA^a

ELISA	Sensitivity (%)	Specificity (%)
Euroimmun Pertussis IgG	91/91	90/90
Virion\Serion PT IgG	89/78	70/85
Virotech PT IgG	84/80	87/93
Dr.Merk Pertussis IgG	100	51
IBL Pertussis IgG	100	52
MP Biomedical Pertussis IgG	100	51
Novagnost Pertussis IgG	94	55
Novatec Pertussis IgG	95	54
Trinity Biotech Pertussis EIA	91	59
Virion\Serion Pertussis IgG	86	56
Virotech Pertussis IgG	87	81

^aIndeterminate results according to the kit's insert were interpreted as positive. Sensitivity and specificity of kits measuring anti-PT were also estimated when the in-house ELISA cutoff (≥40 IU/ml) was used (second number where two numbers are given).

Cutoffs for IgA antibodies to PT are less well standardized, and we estimated the sensitivity and specificity of kits measuring IgA antibodies compared to the LLQ for the in-house ELISA, which is 12 IU/ml. Table 5 shows that the sensitivity estimates for kits measuring IgA ranged between 0.53 and 0.73 and the specificity ranged between 0.67 and 0.94. Kits using a mixture of PT and FHA also detected IgA antibodies in postvaccination sera, as the vaccination induced anti-FHA IgA antibodies (10).

TABLE 5.

Sensitivity and specificity of IgA ELISAs compared to the lower limit of quantitation (LLQ) (12 IU/ml) of an anti-PT IgA in-house ELISA^a

ELISA	Sensitivity (%)	Specificity (%)
Dr.Merk & Kollegen Pertussis IgA	53	74
Virion/Serion Pertussis IgA	68	67
Virotech Pertussis IgA	69	70
Virotech PT IgA	71	84
Euroimmun Pertussis IgA	73	94
Novagnost Pertussis IgA	72	75
IBL Pertussis IgA	64	68
Novatec Pertussis IgA	68	75

^aIndeterminate results according to the kit's insert were interpreted as positive.

DISCUSSION

PCR and serology are the main diagnostic methods for pertussis, and ELISAs measuring antibodies to Bordetella antigens have been developed during the acellular vaccine trials (7). Mostly, these ELISAs use purified antigens, although the use of mixed antigens has also been described (8, 13, 18). The availability of an international reference preparation for pertussis serology has increased the comparability between ELISAs (24). This prompted us to redo a comparison between various ELISAs that were commercially available in Germany (11). The linearity of values measured by kits using purified PT as an antigen was good to acceptable, but ELISA kits using a mixture of antigens showed no linear correlation to the WHO reference preparation. One PT-containing kit (Virotech PT) was evaluated prior to being used in a European seroepidemiology study (ESEN), and these results were very comparable to our findings (17). The ESEN study also stressed the importance of antigen preparations for PT-containing ELISAs (6). In accordance with those findings, we observed that in three patient sera anti-PT IgG levels of >100 IU/ml were detected by the in-house ELISA and by one commercial ELISA but not by two other commercial ELISAs using a different source of PT.

When comparing diagnostic results of ELISAs with the in-house ELISA, we found that only kits using PT as an antigen showed overall good sensitivity and specificity. In contrast, kits using a mixture of antigens, such as PT and FHA, had an insufficient specificity for single sample serology.

Concerning the diagnostic use of IgA antibodies, conflicting results have been published (7, 13, 21). Recently, a European collaboration (6a) suggested that IgA antibodies are of marginal value for the serological diagnosis of pertussis and may be used as an additional method only to test sera with anti-PT IgG concentrations in indeterminate ranges when no second sample will be available. This recommendation is confirmed here by showing that the results of only a few commercial ELISAs measuring anti-PT IgA compared well to the LLQ of an in-house ELISA.

The need for standardization of pertussis serology was recently also addressed in a meeting convened by the CDC (18). In accordance with the findings reported here, a newly developed ELISA for use in public health laboratories in the United States contains PT as the only antigen (14).

Other points deserve to be mentioned. Some ELISAs measured the “low” WHO reference preparation (106 IU/ml) as higher than the “high” WHO reference preparation (335 IU/ml). Two assays declared the IgA content of reference preparation 5 from FDA/CBER as “borderline,” whereas the declared content is 140 IU/ml anti-PT IgA and 280 IU/ml anti-FHA IgA. One assay measured all samples except the negative-control sample as highly positive. Many assays using mixed antigens stated that results should be expressed in “FDA” units/ml, which is by definition not possible for ELISA systems with mixed antigens.

As a consequence of this study, producers of commercial ELISAs measuring antibodies to B. pertussis antigens should be urged to use purified PT as an antigen, to produce kits with a broad linear range, and to express results quantitatively as IU/ml. It also seems necessary to reevaluate the CE-marking process for this type of diagnostic device, as all kits are formally correct in declaring that they measure antibodies against B. pertussis but actually no clinical validation of the kits' diagnostic performance is required.