Technical and Diagnostic Performance of Five Commercial Anti-Diphtheria Toxoid IgG Enzyme-Linked Immunosorbent Assay Kits

Abstract

The technical and diagnostic performances of five commercially available enzyme-linked immunosorbent assays for the measurement of anti-diphtheria toxoid IgG antibodies were evaluated. There was good agreement between the relative sensitivities of the five assays, but the relative specificity of one of the assays differed from that of the other four assays. Three of the five assays possessed recoveries of the international reference material NIBSC 00/496 within the range of 90% to 110% at antibody levels >0.1 IU/ml. The data suggest that there are manufacture-dependent differences in relative sensitivity, specificity, and accuracy for the determination of anti-diphtheria toxoid IgG antibodies that could result in different diagnostic interpretations.

Diphtheria is caused by toxigenic strains of Corynebacterium diphtheriae and is almost entirely preventable through a regular immunization schedule (9). Since routine immunization against diphtheria has become standard practice in industrialized nations, clinical cases are rare; however, it is still a problem in many developing countries, such as the new nations formed from the breakup of the former Soviet Union (29), as well as in people without adequate immunity.

As well as determining the rates of immunity within broad populations and the immune status of at-risk individuals, accurate measurements of anti-diphtheria toxoid IgG levels are important in assessing the response to vaccination and the efficacy of an immunization schedule (13, 16, 31) and in evaluating individuals for potential immunodeficiency disorders (5). Vaccination response studies are part of the clinical testing recommended for the diagnosis of primary immunodeficiency (2, 5).

For determination of anti-diphtheria toxoid IgG antibodies, the in vivo neutralization test (NT) and in vitro Vero cell assay (VCA) are considered the gold-standard methods (10, 18). However, enzyme-linked immunosorbent assay (ELISA)-based methodologies offer a simpler, safer technique, as live toxin is not required and an ELISA is a more rapid and less expensive technique than a neutralization test. It is often preferred by clinical laboratories to measure anti-diphtheria toxoid IgG levels.

Standardization of anti-diphtheria toxoid IgG tests has been facilitated by the availability of reference material, allowing results to be given in international units (IU). The international standard for anti-diphtheria toxoid IgG (NIBSC 00/496) was assigned a value of 0.8 IU/ml by comparison to the British standard for equine anti-diphtheria toxin (NIBSC 66/153) in the mouse in vivo neutralization test. The World Health Organization (WHO) states that a specific IgG concentration of 0.1 IU/ml is usually considered protective by standard ELISA (27). Many studies have incorporated the WHO guidelines and interpret results <0.01 IU/ml as not protective against infection, 0.01 to 0.09 IU/ml as basic protective levels, and >0.1 IU/ml as protective levels (4, 5, 7, 8).

The titer of anti-diphtheria IgG antibodies in patients with inadequate immunity or an immunodeficiency disease can be very low, often <0.1 IU/ml. To be able to accurately and reliably measure low-level titers, ELISAs need to be extremely sensitive. This report compares the sensitivities of five commercially available anti-diphtheria toxoid IgG ELISA kits and suggests that the manufacture-dependent differences in preparation of these assays may affect the clinical interpretation of data.

MATERIALS AND METHODS

Anti-diphtheria toxoid IgG antibodies were measured according to the manufacturers' instructions using the following ELISA kits with the corresponding measuring ranges: Euroimmun, Lübeck, Germany (0.01 to 2 IU/ml); Scimedx Corp., Denville, NJ (0.1 to 5 IU/ml); Serion-Virion, Würzburg, Germany (0.05 to 2 IU/ml); Binding Site Group Limited (BS), Birmingham, United Kingdom (0.012 to 3 IU/ml); and Genzyme Virotech, Rüsselsheim, Germany (0.1 to 5 IU/ml). The lengths of time taken to run the assays were as follows: Euroimmun, 105 min; Scimedx Corp., 90 min; Serion-Virion, 120 min; BS, 90 min; and Genzyme Virotech, 90 min. Results were generated as per the manufacturers' instructions. Assays were considered valid when quality control parameters were in range, as per the manufacturers' product inserts.

Intraassay precision for all five kits was measured using three serum samples (low, medium, and high levels) and assayed in six-well repeats at the same time. For Euroimmun, Serion, and BS, a further sample was used for the measurement of precision <0.1 IU/ml. For interassay precision, the same measurements were performed over two consecutive days. The intra- and interassay precisions were assessed by calculating the coefficient of variation.

Normal human sera (unknown vaccination status) and pre- and post-diphtheria toxoid vaccination serum samples were obtained from Research Sample Bank, Inc., Pompano Beach, FL, and Golden West Biologicals Inc., Temecula, CA, and stored at −20°C prior to testing.

For the comparison between two assays, we calculated the relative sensitivities and relative specificities on the same set of samples run in both assays. The determination of whether a sample was positive or negative was relative to the cutoff value of 0.1 IU/ml in a particular assay. The values obtained served only for the comparison between the two assays, and the results were expressed as percentages. The following equations were used: for relative sensitivities, [number of samples positive in assays A and B/(number of samples positive in assays A and B + number of samples positive in assay A and negative in assay B)] × 100; for relative specificities, [number of samples negative in assays A and B/(number of samples negative in assays A and B + number of samples positive in assay A and negative in assay B)] × 100.

The reference material NIBSC 00/496 (NIBSC, Hertfordshire, United Kingdom) was used to evaluate the calibration in the ELISAs. For use, NIBSC 00/496 was reconstituted according to the manufacturer's instructions (working concentration of 1.6 IU/ml) and serially diluted with the appropriate sample diluents to a final concentration of 0.012 IU/ml. The calibration of each assay was assessed by calculating the “recovery” of NIBSC 00/496 by obtaining values (IU/ml) for serially diluted NIBSC 00/496 from the assay calibration curve and comparing them to the expected values according to the following equation: (obtained NIBSC value/expected NIBSC value) × 100. The results are expressed as percentages.

The response to diphtheria vaccination was assessed using 12 pre- and postvaccination sample pairs, assayed over two consecutive days. The samples were evaluated for a significant fold increase in titers between the prevaccination titer and postvaccination titer. A significant increase has been defined in literature as 4-fold (31).

For statistical analysis of the data, Wilcoxon's rank sum test and Freidman's analysis were applied using Analyse-it (Analyse-it Software, Ltd., Leeds, United Kingdom). A P value <0.05 was considered significant.

RESULTS

Intra- and interassay precision were calculated for each kit (Table 1). Three random samples were selected that could be read from all five calibration curves in the measuring range of 0.1 to 2 IU/ml. The intra- and interassay precisions for the three samples ranged from 3.4% to 17.4% and 4.6% to 27.3%, respectively, with the lowest values achieved with the Scimedx (4.6% to 5.8% and 4.6% to 7.5%) and BS (3.4% to 3.8% and 5.8% to 6.7%) assays. Poor interassay precision (>15%) was evident for the low-level sample in the Euroimmun assay and the high-level sample on both the Euroimmun and Serion assays. To further challenge precision at the bottom end of the curve on the three assays with measuring ranges lower than 0.1 IU/ml (Serion, 0.05 IU/ml; BS, 0.012 IU/ml; and Euroimmun, 0.01 IU/ml), precision with another sample was assessed. This sample was randomly selected and was similar to the three samples used above since it gave a measureable value <0.1 IU/ml in all three assays. The mean precision of the sample was 0.052 IU/ml (coefficient of variation [CV] = 15.2%) in the Serion assay, 0.09 IU/ml (CV = 4.2%) in the BS assay, and 0.031 IU/ml (CV = 10.3%) in the Euroimmun assay.

TABLE 1.

Intra- and interassay precision calculated in five commercial ELISAs

Assay	Intraassay or interassay precision	Low-level serum sample			Medium-level serum sample			High-level serum sample
		Mean (IU/ml)	SD	% CV	Mean (IU/ml)	SD	% CV	Mean (IU/ml)	SD	% CV
Euroimmun	Intra	0.44	0.06	13.6	0.96	0.1	10.4	0.46	0.08	17.4
	Inter	0.44	0.12	27.3	0.96	0.14	14.6	0.46	0.09	19.6
Scimedx	Intra	0.65	0.03	4.6	2.92	0.17	5.8	0.96	0.05	5.2
	Inter	0.65	0.03	4.6	2.92	0.22	7.5	0.96	0.07	7.3
Serion	Intra	0.48	0.03	6.3	1.5	0.11	7.3	0.45	0.03	6.7
	Inter	0.48	0.03	6.3	1.5	0.12	8.0	0.45	0.07	15.6
BS	Intra	0.52	0.02	3.8	0.89	0.03	3.4	1.04	0.04	3.8
	Inter	0.52	0.03	5.8	0.89	0.06	6.7	1.04	0.06	5.8
Virotech	Intra	0.52	0.03	5.8	1.36	0.06	4.4	0.81	0.05	6.2
	Inter	0.52	0.06	11.5	1.36	0.09	6.6	0.81	0.06	7.4

In order to assess the performance of the five ELISAs, levels of antibodies to diphtheria toxoid in 56 serum samples were determined and quantified (Fig. 1). On the Scimedx, Serion, and Virotech assays, not all samples could be measured due to the limitation of the lower end of the measuring range and were therefore not included in Fig. 1. All five ELISAs identified a number of samples with titers of anti-diphtheria IgG antibodies below the WHO-recommended level of protection (0.1 IU/ml): Euroimmun, 37/56 samples (66%); Scimedx, 3/56 samples (5.3%); Serion, 21/56 samples (38%); BS, 26/56 samples (46.4%); and Virotech, 9/56 samples (17%).

FIG. 1.

Box plot of normal samples with quantitative values assayed in the five ELISAs (Euroimmun, n = 52; BS, n = 56; Serion, n = 40; Scimedx, n = 53; Virotech, n = 47).

Figure 1 also shows that there were no statistical differences between the Euroimmun and BS assays (Wilcoxon test, P = 0.375) and the Scimedx, Serion, and Virotech assays (Friedman's test, P = 0.295) but there was a significant difference between the two groups of data (Wilcoxon test, P < 0.0001). Figure 1 also suggests that there was not an even distribution of sample values around the median in the Euroimmun and Scimedx assays. Using the samples that were quantifiable in all five assays, there was good agreement in regard to positivity (>0.1 IU/ml) and negativity (<0.1 IU/ml) between the five assays. A comparison of relative sensitivity between the five assays was 88.6% to 100%, and relative specificity between four of the assays was 72.7% to 90.5% with the exception of the Scimedx assay (45% to 52.6%).

To assess the accuracy of calibration, the recoveries of serially diluted NIBSC 00/496 reference material with known titers, read from the respective calibration material, were assessed and expressed as a percentage of the target value (Table 2). To allow fair comparison across all five assays, recovery across a measuring range of 0.11 to 3 IU/ml was compared. Two of the five assays (BS and Virotech) gave consistent and accurate recoveries of the reference material across the range tested, with percent CV <10%. When recoveries around the protective cutoff of 0.1 IU/ml were assessed, only the Scimedx (95.1%), BS (97.1%), and Virotech (91.5%) assays showed recoveries within 10% of the target value. When the diluted reference material at target values of 0.037 and 0.012 IU/ml were run on the two assays designed to measure that low, the values were 108.1% and 130.3%, respectively, in the BS assay but were undetectable in the Euroimmun assay.

TABLE 2.

Mean percentage recovery from three repeat assays of the NIBSC 00/496 standard in five commercial ELISAs^a

Target (IU/ml)b	Recovery (%) in:
	Euroimmun	Scimedxc	Seriond	BS	Virotechc
3.00	45.5	149.3	NA	85.3	107.6
1.00	98.0	127.7	104.9	89.7	114.1
0.33	83.6	86.1	94.5	94.8	100.2
0.11	37.5	95.1	82.3	97.1	91.5
0.037	0.00	N/A	N/A	108.1	N/A
0.012	0.00	N/A	N/A	130.3	N/A
Mean	66.1	114.7	93.9	91.7	103.3
CV	44.3	25.4	12.1	5.8	9.4

^aTo calculate values, values (IU/ml) for serially diluted NIBSC 00/496 were obtained from a calibration curve and compared to the expected value according to the following equation: (obtained NIBSC value/expected NIBSC value) × 100. NA, not applicable.

^bMean value and CV are for the recovery of samples of 0.11 to 3.00 IU/ml.

^cThe measuring range of this kit was 0.1 to 5 IU/ml (see Materials and Methods).

^dThe measuring range of this kit was 0.05 to 2.0 IU/ml (see Materials and Methods).

Levels of anti-diphtheria toxin IgG antibodies were determined in 12 paired pre- and postvaccination samples (Table 3). It was not possible to obtain quantitative values for all prevaccination samples by all five assays, as some samples were below the limit of detection in certain assays.

TABLE 3.

Pre- and postvaccination data from five commercial ELISAs^a

Sample ID	Euroimmun			Scimedx			Serion			BS			Virotech
	Pre	Post	Fold	Pre	Post	Fold	Pre	Post	Fold	Pre	Post	Fold	Pre	Post	Fold
9	0.093	1.089	11.7	0.186	1.547	8.3	0.113	1.286	11.4	0.129	0.914	7.1	0.129	1.351	10.4
10	0.018	0.174	9.7	0.103	0.333	3.2	<0.05	0.310		0.046	0.218	4.8	<0.1	0.246
14	0.018	0.822	46.3	0.103	1.257	12.2	<0.05	0.861		0.049	0.838	17.1	<0.1	0.891
15	0.232	1.271	5.5	0.246	2.445	10.0	0.458	>2.000		0.431	1.458	3.4	0.183	1.826	10.0
16	0.077	0.494	6.4	0.175	0.560	3.2	0.220	0.608	2.8	0.191	0.417	2.2	0.251	0.722	2.9
17	0.045	0.884	19.4	0.138	1.186	8.6	<0.05	0.776		0.065	0.567	8.8	<0.1	0.709
19	0.011	0.146	13.0	0.110	0.228	2.1	<0.05	0.278		0.044	0.266	6.0	<0.1	0.211
20	0.160	0.495	3.1	0.622	1.034	1.7	0.630	0.930	1.5	0.377	0.614	1.6	0.339	0.620	1.8
21	0.069	1.233	17.8	0.137	3.806	27.7	0.265	>2.000		0.216	1.476	6.8	0.164	1.744	10.7
22	<0.01	0.568		<0.1	0.907		<0.05	0.557		0.025	0.460	18.2	0.154	0.765	5.0
26	0.054	0.308	5.7	0.149	0.336	2.3	0.061	0.233	3.8	0.065	0.178	2.7	<0.1	0.252
28	0.014	0.018	1.3	<0.1	<0.1		<0.05	<0.05		0.021	0.035	1.7	<0.1	<0.1
Mean	0.072	0.625		0.197	1.240		0.291	0.585		0.138	0.620		0.203	0.849
SD	0.069	0.431		0.156	1.068		0.216	0.387		0.140	0.473		0.078	0.569
SEM	0.020	0.125		0.045	0.308		0.062	0.112		0.040	0.137		0.023	0.164

^aThe samples were assayed in duplicate wells at the same time in three separate assays. All assays were valid, and the data for the pre- and postvaccination samples represent the mean values derived from the three independent assays. Fold, fold change; SEM, standard error of the mean for all readable pre- or postvaccination samples.

It has been proposed that a 4-fold or greater increase in the specific antibody titer is a clinically significant response to diphtheria vaccine (16). It has also been suggested that a nonresponder would exhibit a <2.6-fold increase in antibody titers after vaccination or antibody titers less than the protective titer postvaccination (17, 27). The titers of anti-diphtheria IgG antibodies were <0.1 IU/ml for prevaccination samples 9 and 21 in the Euroimmun assay and >0.1 IU/ml for prevaccination samples 10, 14, 17, 19, and 26 in the Scimedx assay, which were out of consensus with titers measured in the other four assays. With the anti-diphtheria IgG titers recorded in samples 10, 16, and 19 in the Scimedx assay, a 4-fold increase in titer was not achieved postvaccination but the prevaccination levels indicated titers >0.1 IU/ml.

For both the Serion and Virotech assays, it was difficult to accurately measure an increase in titer for samples 10, 14, 17, and 19, as well as for sample 22 in the Serion assay and sample 26 in the Virotech assay, as the prevaccination anti-diphtheria IgG antibody titers were lower than the measuring range of the kit. Fourfold increases in antibody titer could not be measured in either assay for sample 16; however, a prevaccination level of >0.1 IU/ml and no 4-fold increase in titer were measured in sample 26 in the Serion assay, but the postvaccination level was >0.1 IU/ml. The value measured for prevaccination sample 22 in the Virotech assay was >0.1 IU/ml, which was out of consensus with values obtained in the other four assays, which either recorded values lower than the measuring range or falling in the range but <0.1 IU/ml.

In the BS assay, 4-fold increases in anti-diphtheria IgG titers were not obtained for samples 15, 16, and 26, but in the two former samples the prevaccination titer was >0.1 IU/ml and in sample 26 the postvaccination titer was >0.1 IU/ml. In all five assays, a 4-fold increase in titer could not be measured for samples 20 and 28. However, for sample 20 all prevaccination levels were >0.1 IU/ml, and both pre- and postvaccination titers were <0.1 IU/ml for sample 28.

DISCUSSION

Although reports have suggested that the anti-diphtheria IgG antibody titer required for adequate protection is >0.1 IU/ml (27), several reports have suggested that sensitive quantitation of specific IgG levels between 0.01 to 0.1 IU/ml are also important. Antibody levels within this range provide minimal protection, and quantification may provide important clinical information.

There is debate as to whether in vitro techniques for measuring anti-diphtheria IgG antibodies are as sensitive as the in vivo neutralization tests. It has been suggested that inaccuracies with ELISA may be due to the detection of nonneutralizing antibodies directed toward an immunogenic but nontoxic part of the toxoid (22, 23). In these studies, it is not stated whether these ELISAs were calibrated to any standard, such as NIBSC 00/496. When using standardized in vitro ELISA for the measurement of anti-diphtheria IgG antibodies, the correlation of sensitivities with an in vivo neutralization assay and in vitro tissue culture method is good at values as low as 0.01 IU/ml (17, 30). At titers <0.01 IU/ml, however, the correlation has been shown to be poor (14, 17).

In the present study, we have compared the assay performance of five different commercially available anti-diphtheria toxoid IgG ELISAs. The data suggest that considerable variation exists between the five assays. The ranges of percent CV for both intraassays and interassays were 3.4% to 17.4% and 4.6% to 27.3%, respectively, with samples of antibody titers >0.1 IU/ml and 4.2% to 15.2% for samples <0.1 IU/ml. The serum sample data readings suggest that the Euroimmun and BS assays give similar results, but the results are significantly different from the Scimedx, Serion, and Virotech results. The sensitivities between the five assays were comparable, but the Scimedx assay had lower specificity with respect to the other four assays.

Comparison of the accuracy of calibration to the international reference standard NIBSC 00/496 revealed interesting results. Only three of the assays possessed <15% mean variation between the assay calibrator material and reference standard at anti-diphtheria IgG antibody levels >0.1 IU/ml. The other two assays showed a large mean variation of 25.4% to 44.3%. At levels <0.1 IU/ml, one assay had a mean variation of 19% and the other failed to detect the low-level anti-diphtheria IgG antibodies.

Analysis of the anti-diphtheria IgG levels in pre- and postvaccination serum further highlighted variation. Depending on the assay chosen, a sample could be interpreted as possessing a protective or nonprotective level of anti-diphtheria IgG antibodies, and an accurate assessment of fold increases in titers could be difficult.

A recent independent study has also highlighted the differences that exist between commercial anti-diphtheria IgG ELISA kits. Di Giovine et al. (6) assessed the performance of five commercial anti-diphtheria IgG ELISAs using samples with values determined in the gold-standard VCA. The correlation coefficients between the individual assays and the VCA ranged from 0.6 to 0.85, and in three of the assays the majority of positive VCA samples were <0.1 IU/ml. The data clearly demonstrated that manufacture-dependent differences between anti-diphtheria IgG ELISAs exist.

Since all assays were performed by the same technician and the protocols for all commercial assays were similar, such variations that affect antibody-antigen interaction may arise at several points in assay manufacture. The variation in precision data suggests possible differences in antigen source, preparation, or plate coating techniques, and the differences in the data sets imply that different types of blocking agents may have been utilized or that plate blocking may have occurred to different extents. Some of the variation is undoubtedly due to differences in the setting of the calibration, and it is not disclosed by all manufacturers how the calibration has been set in regard to the use of an international standard. Potentially, the types of buffer systems used in the assays could also influence the antibody-antigen interaction. Almost certainly, the data suggest that there is a need for some level of agreed standardization in the preparation of these assays.

The sensitive measurements of anti-diphtheria IgG antibodies are important clinically for two reasons: assessment of immune status and measurement of immune response. The WHO has suggested that the level of anti-diphtheria toxoid IgG antibodies required for optimal protection from diphtheria infection is >0.1 IU/ml. However, the levels of circulating anti-diphtheria IgG antibodies required for basic immunity in humans has been estimated to be 0.01 IU/ml by the Schick skin test (12, 24, 25), and many studies have reported that levels in the general population are <0.1 IU/ml and in some cases <0.01 IU/ml (1, 3, 11, 19, 20, 21, 26, 28, 32). Maple and colleagues showed that from the age of 14 years up to 54 years of age, the percentage of the population (n = 3,088) with anti-diphtheria toxoid IgG levels >0.1 IU/ml decreased from ∼40% to ∼5%. Within the same groups, the percentage of the population with levels <0.01 IU/ml increased from ∼10% to ∼40% (15). A study by Edmunds and colleagues found similar results (7). In the present study, the standard deviation of the mean titer in 56 random serum samples and in the prevaccination samples suggested that a considerable percentage of samples have anti-diphtheria IgG antibody titers <0.1 IU/ml (between 5.3% and 66%, depending on the assay used).

The response of the immune system to the challenge of a vaccine can aid immunologists in their diagnosis of an immunodeficiency (5). Studies have suggested that responders, with an efficient immune response, produce a 4- to 31.1-fold increase in specific antibody titer to diphtheria vaccine (2, 16, 26), whereas nonresponders, possessing a defective immune response, only show an approximate 2.6-fold increase in titer (16) or an antibody titer postvaccination of <0.1 IU/ml (27). Furthermore, two studies have suggested that the percentage of nonresponders to the diphtheria vaccine could be between 4% and 14% depending on the age group and vaccination regime (11, 20).

In conclusion, we have provided a study to evaluate the performance by five different commercial ELISAs in the measurement of anti-diphtheria toxoid IgG levels. Given the clinical use of the information derived from the test, it is essential that sensitive and accurate ELISAs are utilized. We conclude that among the commercially available anti-diphtheria IgG ELISAs tested in this study, there are significant differences that exist in regard to precision, relative sensitivity, relative specificity, and calibration. Further studies are required to understand the clinical implications that arise from the use of different ELISAs with differing levels of sensitivity and these significant manufacture-dependent differences.