External Quality Assessment for the Determination of Diphtheria Antitoxin in Human Serum

Abstract

Accurate determination of diphtheria toxin antibodies is of value in determining the rates of immunity within broad populations or the immune status of individuals who may be at risk of infection, by assessing responses to vaccination and immunization schedule efficacy. Here we report the results of an external quality assessment (EQA) study for diphtheria serology, performed within the dedicated surveillance network DIPNET. Twelve national laboratories from 11 European countries participated by testing a standard panel of 150 sera using their current routine method: Vero cell neutralization test (NT), double-antigen enzyme-linked immunosorbent assay (ELISA; DAE), dual double-antigen time-resolved fluorescence immunoassay (dDA-DELFIA), passive hemagglutination assay (PHA), toxin binding inhibition assay (ToBI), and in-house or commercial ELISAs. The objective of the study was not to identify the best assay, as the advantages and drawbacks of methods used were known, but to verify if laboratories using their routine method would have categorized (as negative, equivocal, or positive) a serum sample in the same way. The performance of each laboratory was determined by comparing its results on a quantitative and qualitative basis to NT results from a single reference laboratory, as this test is considered the in vitro “gold standard.” The performance of laboratories using NT was generally very good, while the laboratories’ performance using other in vitro methods was variable. Laboratories using ELISA and PHA performed less well than those using DAE, dDA-DELFIA, or ToBI. EQA is important for both laboratories that use in vitro nonstandardized methods and those that use commercial ELISA kits.

Today diphtheria is a marginal problem in western countries, as only sporadic cases are reported (29). However, diphtheria is still present in Latvia, Russia, and Ukraine and is endemic in other parts of the world, including Asia (particularly in India, Indonesia, and Nepal), Africa (Angola), and South America (Brazil) (29).

Clinical diphtheria is caused by toxin-producing corynebacteria. Three species, Corynebacterium diphtheriae, Corynebacterium ulcerans, and Corynebacterium pseudotuberculosis, have the potential to produce diphtheria toxin and hence can cause classic respiratory diphtheria (6).

It is worth noting that C. ulcerans infections have been reported worldwide in recent years, and fatal infections have been recorded (5, 8, 15, 23, 25). As the morbidity of diphtheria is almost entirely due to diphtheria toxin, protection against disease is dependent on antibodies against the toxin (2, 21).

Accurate determination of anti-diphtheria toxin antibodies is essential to establish susceptibility of clinical laboratory workers, to obtain reliable information on the immune status of a person or a given population, and to evaluate the immunogenicity of diphtheria vaccines in clinical trials, as well as to monitor long-term immunity and thus provide recommendations for vaccination policy. Therefore, it is of critical importance to have serological methods that are accurate, reproducible, specific, and sensitive.

The in vivo toxin neutralization test using guinea pigs or rabbits is regarded as the “gold standard” method for determining protective levels of serum antitoxin (14). However, this test requires animals and specialized facilities, is labor intensive and expensive, and requires relatively large volumes of test serum. It is therefore not practical for routine use in serological diagnosis and seroepidemiological studies. Tests using cells in culture have been developed as reliable alternatives to the in vivo test for detection of diphtheria toxin and for toxin neutralization (18). The Vero cell toxin neutralization assay (NT) is also the recommended World Health Organization (WHO) and European Pharmacopeia in vitro alternative method, as it provides comparable results to guinea pig protection models for potency testing of vaccines (6, 10, 21, 28). However, because this assay is also time-consuming and requires cell culture facilities, diagnostic laboratories prefer to use simple-format indirect enzyme-linked immunosorbent assays (ELISAs), which offer significant advantages in terms of cost, speed, ease of use, and adaptability to automation. Other in vitro methods are available, such as the double-antigen ELISA (DAE) (16), the dual double-antigen, time-resolved fluorescence immunoassay (dDA-DELFIA) (1, 4), the passive hemagglutination assay (PHA) (27), the toxin binding inhibition assay (ToBI) (12), and the fluorescent bead-based multiplex assay (24), but none of these are as easy to perform as an indirect ELISA.

Due to the reemergence of diphtheria to epidemic levels in the Russian Federation and Newly Independent States during the 1990s, the European Laboratory Working Group on Diphtheria was established in 1993 (7, 11). In 2006, the Diphtheria Surveillance Network was expanded and officially recognized by the European Commission as a dedicated surveillance network, named DIPNET, that encompasses 25 European countries as participants. One of the main objectives of the network is to assess and standardize laboratory diagnostic methods for diphtheria as well as to perform external quality assurance (EQA) studies for laboratory diagnosis of diphtheria (identification and determination of isolate toxigenic status) with molecular typing and serology to strengthen laboratory assurance of all DIPNET participants, especially because diphtheria has become a rare disease in the majority of the participating countries.

In this study, we report the results of the EQA study for serology in which 12 national laboratories of 11 European countries participated using their current routine method for assaying human diphtheria toxin antibodies.

MATERIALS AND METHODS

Study design.

Each of the participating labs received from the coordinating center, the Istituto Superiore di Sanità (ISS), Rome, Italy, a panel of 150 human sera to be tested for diphtheria antitoxin antibodies using an assay of their choice (Table 1). Each lab used its own standard curve and included a positive and a negative control sample normally used in the assay. The standard panel was tested twice by each participant. The results from the specific diphtheria antitoxin concentration, expressed in IU/ml, were calculated by each center according to their standard operating procedures and sent by e-mail to the coordinating center.

TABLE 1.

Tests and reference preparations for participant laboratories

Laboratory	Assay	Lowest level of detection (IU/ml)	Diphtheria toxin or toxoid, producer (Lf)a	Diphtheria reference serum (antitoxin)b
I	dDA-DELFIA	0.0004	Toxoid, SSI (2,267/ml)	WHO,c batch DI98 (equine)
	ELISA (VaccZyme)	0.012	Toxoid	NIBSC, batch 00/496 (human)
	ELISA (Virotech)	0.1d	Toxoid	NIBSC, batch 00/496 (human)
II	PHA	0.01	Toxin	Control serum, 10 IU/ml
III	ELISA (in-house method)	0.001	Toxoid, NCIPD Sofia (490/ml)	In-house human serum calibrated against WHO standardb
IV	ELISA (VaccZyme)	0.012	Toxoid	NIBSC, batch 00/496 (human)
	ELISA (NovaLisa)	0.01	Toxoid	NIBSC, batch 91/534 (human)
V	ELISA (Virotech)	0.1d	Toxoid	NIBSC, batch 00/496 (human)
VI	ELISA (VaccZyme)	0.012	Toxoid	NIBSC, batch 00/496 (human)
	Vero cell (NT)	0.016	Toxin, RIVM batch 79/1 (1,000/ampoule)	NIBSC 3rd British standard, batch 66-153 (equine)
VII	ELISA (Euroimmun)	0.005	Toxoid	NIBSC, batch 91/534 (human)
VIII	Vero cell (NT)	0.004	Toxin; Japanese, lot M59 (0.25/ml)	JNSDAe (equine)
IXf	DAE	0.007	Toxoid	WHO,c batch DI05 (equine)
	ELISA (VaccZyme)	0.012	Toxoid	NIBSC, batch 00/496 (human)
	Vero cell (NT)	0.005	Toxin, KTL (650/ml)	WHO,c batch DI05 (equine)
X	ELISA (Serion)	0.05	Toxoid	SSIc
XI	ELISA (in-house method)	0.015	Toxoidg	NIBSC, batch 00/496 (human)
	Vero cell (NT)	0.0008	Toxin, EDQM, BRP batch 1	WHO,c batch DI07 (equine)
XII	ToBI	0.005	Toxin, RIVM batch 79/1 (1,000/ampoule)	WHO,c batch DI07 (equine)

^aLf, limit of flocculation.

^bIn the case of ELISA, the reference serum is the one against which the human sera used as control in the kit have been calibrated.

^cSince 1997, the diphtheria antitoxin WHO 1st international standard (Statens Serum Institut, Copenhagen, Denmark) (10 IU/ml) has been distributed by the National Institute for Biological Standards and Control (NIBSC) in different liquid lots with the prefix “DI.”

^dLimit of quantification.

^eJNSDA, Japanese National Standard Diphtheria.

^fReference laboratory.

^g2nd WHO International Standard for diphtheria toxoid (NIBSC code 02/176, 1,100 Lf/ampoule) for coating plates (at 0.5 Lf/ml).

Standard panel construction.

The standard panel was prepared using sera kindly donated by blood donors from a center in Rome. ISS tested the panel twice by dDA-DELFIA and used the average of the two values to reduce the interassay variability. The panel, containing 150 sera with 300 μl of each specimen, was sent frozen by courier post to each participant and stored at −20°C until testing. Labs X, XI, and XII tested only 146, 147, and 140 samples, respectively.

Assays.

In the present EQA study, various assays were used to measure specific human diphtheria toxin antibodies. These included the Vero cell NT, dDA-DELFIA, DAE, PHA, ToBI, and in-house and commercial ELISA methods. The commercial ELISA kits specific for the determination of diphtheria antitoxin IgG were the Serion ELISA classic (Serion), Virotech ELISA (Genzyme Virotech GmbH, Germany), Euroimmun ELISA (Euroimmun), Vacczyme ELISA, (Binding Site Ltd., United Kingdom), and NovaLisa (NovaTec Immunodiagnostica GmbH, Germany). Commercial ELISAs were performed according to manufacturer's instructions, using reagents that were supplied with the kits. Details of the assays used by each participating lab, including the toxin/toxoid and reference antitoxin used and the reported limit of detection are listed in Table 1.

Reference assay.

The assay selected as a reference to evaluate the performance of the other assays was the Vero cell NT from laboratory IX (Table 1).

Using this assay, diphtheria antitoxin levels in individual serum samples were classified as follows, based on WHO guidelines (6) and work performed by Ipsen (13): positive, i.e., the full protective level of circulating antitoxin (≥0.1 IU/ml); equivocal, partial protective levels of antitoxin (0.01 to 0.09 IU/ml), or negative, providing no protection (<0.01 IU/ml). Results obtained by the reference lab showed that of the panel of 150 sera, 77 samples were positive, 45 samples were equivocal, and 28 samples were negative.

Data analysis.

Participants’ results were compared on a quantitative and qualitative bases. Standard methods recommended for meta-analyses of diagnostic test evaluations were used (19). Data were analyzed using SPSS for Windows (version 14.0.1.366) and Microsoft Excel 2003. To assess the extent of quantitative diagnostic accuracy, the NT-derived values from the reference laboratory (lab IX) and the results obtained with all of the other tests were compared using a scatter plot of antibody measurements on a log₁₀ scale. The correlation coefficients (R), the slope of the regression (S) and the y-intercept (D) were calculated for each scatter plot. For serum samples with concentrations below the lower limit of detection (LOD), the imputed value was half the LOD value. For concentrations above the detection level, the imputed value was the highest detection value (26).

To assess the extent of qualitative diagnostic agreement, the sera were classified as negative, equivocal, and positive as described earlier.

In the case of in-house ELISA, sera were qualified as positive when the level of antitoxin was ≥0.1 IU/ml (9, 20, 21). In the case of commercial ELISA kits, the test results were interpreted according to the manufacturer's indications.

For each single test, except ELISA, diagnostic agreement was calculated with the formula (TP + TN)/(TP + FP + FN + TN), where TP represents true-positive samples, FP represents false-positive samples, FN represents false-negative samples, and TN represents true-negative samples.

RESULTS

Interlaboratory comparison of NT.

Four labs (VI, VIII, IX, and reference lab XI) performed the Vero cell NT assay. The interlaboratory comparison of this method showed a high correlation with regression line close to the line of identity (Fig. 1). However, the qualitative agreement between the four labs was not always satisfactory, ranging from a minimum of 76%, when lab XI was compared to lab VIII, to a maximum of 91% when lab VIII was compared to lab VI (data not shown). However, the best agreement with the reference lab IX was obtained by lab XI (agreement, 89%), where the same antitoxin reference serum was used (Table 1).

FIG. 1.

Interlaboratory comparison of the diphtheria antitoxin levels (IU/ml) of the standard panel tested by NT in lab XI (A), lab VIII (B), and lab VI (C) versus reference lab IX. The slope of the regression (S) and intercept (D), correlation coefficient (R) of the regression line (solid lines), and line of identity (dashed lines) are shown. Vertical and horizontal dotted lines indicate the cutoffs used by the reference laboratory to determine negative (<0.01 IU/ml), equivocal (0.01 to 0.09 IU/ml), and positive (≥0.1 IU/ml) sera.

The reasons for the qualitative differences between the NTs might be due to operating procedures, specifically the toxin dose level at which the assay was performed (highlighted by the fact that detection limits were not the same in all four labs) and the use of toxins from different manufacturers, different batches of the same reference serum, or different producers (Table 1). Nevertheless, although the qualitative correlation is not perfect, no negative sera were identified as positive and vice versa, and generally only borderline sera were identified differently by the various labs.

Lab XI tested the standard panel by NT using two different reference sera: the equine WHO 1st international standard serum (lot DI07) and the human NIBSC antitoxin 00/496. No significant differences in the concentrations of antibodies determined using the human or the equine antitoxin as reference serum were observed (R = 0.99).

Comparison of the PHA with NT.

The correlation between the lab II PHA and lab IX NT was R = 0.78, and the slope from the identity line was equal to 0.4 (Fig. 2A). The test identified correctly 49 samples out of 77 with antibody levels of ≥0.1 IU/ml, 37 out of 45 samples in the range 0.01 and 0.09 IU/ml, and only 5 out of 28 samples with antibody levels below 0.01 IU/ml. One negative serum was misidentified as positive, and 28 positive sera were identified as equivocal. The diagnostic agreement for lab II by PHA assay versus lab IX NT was 61% (Table 2).

FIG. 2.

Interassay comparison of the diphtheria antitoxin levels (IU/ml) of the standard panel tested by NT and PHA (A), dDA-DELFIA (B), DAE (C), and ToBI (D). The slope of the regression (S), intercept (D), correlation coefficient (R) of the regression line (solid lines), and line of identity (dashed lines) are shown. Vertical and horizontal dotted lines indicate the cutoffs used by the laboratories to determine negative (<0.01 IU/ml), equivocal (0.01 to 0.09 IU/ml), and positive (≥0.1 IU/ml) sera. dDA-DELFIA used a different cutoff for negative (<0.015 IU/ml).

TABLE 2.

Qualitative agreement between the reference NT and PHA, dDA-DELFIA, DAE, and ToBI

Lab and test result	% diagnostic agreement	No. of samples with NT result:
		Positive	Equivocal	Negative
Lab II PHA	61
Positive		49	7	1
Equivocal		28	37	22
Negative		0	1	5
Lab I dDA-DELFIA	79
Positive		68	12	0
Equivocal		9	30	8
Negative		0	3	20
Lab IX DAE	75
Positive		54	3	0
Equivocal		23	37	6
Negative		0	5	22
Lab XII ToBI	83
Positive		55	0	0
Equivocal		16	39	3
Negative		0	4	23

Comparison of the dDA-DELFIA with NT.

The correlation between the lab I dDA-DELFIA and lab IX NT was R = 0.88 (Fig. 2B); the regression line equation corresponded to log₁₀ dDA-DELFIA (IU/ml) = −0.17 + 0.71 × log₁₀ NT (IU/ml). From previous studies (25), lab I had set the cutoff for negative sera at <0.015 IU/ml; equivocal sera were therefore those included in the range 0.015 to 0.09 IU/ml. Over a total of 150 samples, the dDA-DELFIA test identified 68 out of 77 samples as positive, 30 out of 45 samples as equivocal, and 20 out of 28 samples as negative. Thus, some NT-negative sera were identified as equivocal and some equivocal sera as positive, but no negative sera were identified as positive and vice versa. Therefore, the estimated diagnostic agreement for the lab I dDA-DELFIA with respect to the lab IX NT was 79% (Table 2).

Comparison of the DAE with NT.

The correlation between the lab IX DAE and lab IX NT corresponded to R = 0.92 (Fig. 2C); the regression line equation corresponded to log₁₀ DAE (IU/ml) = −0.35 + 0.81 × log₁₀ NT (IU/ml). Over a total of 150 samples, the DAE test identified 54 samples out of 77 with an antibody level of ≥0.1 IU/ml, 37 out of 45 samples in the range between 0.01 and 0.09 IU/ml, and 22 out of 28 samples with antibody levels below 0.01 IU/ml. Thus, some NT-negative sera were identified as equivocal, and some equivocal sera were identified as positive, but no negative sera were identified as positive and vice versa. The estimated diagnostic agreement for lab IX DAE in relation to lab IX NT was 75% (Table 2).

Comparison of the ToBI with NT.

The correlation between the lab XII ToBI and lab IX NT was R = 0.92 (Fig. 2D); the regression line equation corresponded to log₁₀ ToBI (IU/ml) = −0.41 + 0.74 × log₁₀ NT (IU/ml). Over a total of 140 samples, the ToBI test identified 55 out of 71 samples with an antibody level of ≥0.1 IU/ml, 39 out of 43 samples in the range 0.01 to 0.09 IU/ml, and 23 out of 26 samples with antibody levels below 0.01 IU/ml. Therefore, the estimated diagnostic agreement for lab XII ToBI with respect to lab IX NT was 83% (Table 2).

Comparison of the ELISA with NT.

Two in-house and five different commercial ELISA kits were used (Table 1). The results obtained by the labs using the different ELISAs are shown in Fig. 3.

FIG. 3.

Interassay comparison of the diphtheria antitoxin levels (IU/ml) of the standard panel tested by NT and seven different ELISAs: lab III in-house ELISA (A), lab XI in-house ELISA (B), Serion ELISA (C), Euroimmun ELISA (D), Virotech ELISA (E), NovaLisa ELISA (F), and VaccZyme ELISA (G). The slope of the regression (S), intercept (D), correlation coefficient (R) of the regression line (solid lines), and line of identity (dashed lines) are shown. Vertical and horizontal dotted lines indicate the cutoffs used by the NT reference laboratory to determine negative (<0.01 IU/ml), equivocal (0.01 to 0.09 IU/ml), and positive (≥0.1 IU/ml) sera.

Lab III tested the standard panel serum with a recently developed in-house ELISA. As shown by the respective graph in Fig. 3A, there is no correlation between this ELISA and lab IX NT (R = 0.22).

Lab XI tested the standard panel also with an in-house ELISA and validated the in-house assay against the NT Vero cell assay using the same human reference as a calibrator serum (i.e., NIBSC 00/496). Testing the standard panel by the two methods, lab XI obtained R = 0.87 (results not shown), whereas for the ELISA plotted against the NT of reference lab IX, an R = 0.85 and slope and intercept values of 0.45 and 0.46, respectively, were obtained (Fig. 3B). For the two in-house ELISAs, the qualitative agreement with the NT was determined using a diagnostic threshold cutoff value of 0.1 IU/ml. Thus, the sera were divided into only two categories: negative (<0.1 IU/ml) and positive (≥0.1 IU/ml). The ELISA performed by lab III showed poor agreement with the reference NT assay, and 10 samples that were categorized as positive by NT were negative in the ELISA. More importantly, 20 samples that were negative in the NT assay were identified as positive in the ELISA (Table 3). The ELISA performed by lab XI categorized 6 samples that were positive in the NT assay as negative, but only 1 sample that was negative in the NT assay was categorized as positive in the ELISA (Table 3). Samples that were categorized as equivocal in the NT assay (i.e., likely to offer some degree of protection) were mostly reported as positive in the ELISA from lab III (40/45 samples) but as negative in the ELISA from lab XI (35/44 samples).

TABLE 3.

Qualitative agreement between the reference NT and in-house or commercial ELISA kits^a

Lab test result (IU/ml)	No. of samples with NT result:
	Positive (≥0.1 IU/ml)	Equivocal (0.01-09 IU/ml)	Negative (<0.01 IU/ml)
Lab III in-house ELISA
Positive (≥0.1)	67	40	20
Negative (<0.1)	10	5	8
Lab XI in-house ELISA
Positive (≥0.1)	69	9	1
Negative (<0.1)	6	35	27
Lab X Serion
Positive (>1.0)	5	0	0
Equivocal (0.1-1.0)	46	9	3
Negative (<0.1)	24	36	23
Lab IV NovaLisa
Positive (≥0.1)	49	7	1
Equivocal (0.01-0.09)	28	38	24
Negative (<0.01)	0	0	3
Lab VII Euroimmun
Positive (>1.0)	3	0	0
Equivocal (0.1-1.0)	29	4	1
Negative (<0.1)	45	41	27
Lab I Virotech
Positive (>1.0)	6	0	0
Equivocal (0.1-1.0)	64	14	4
Negative (<0.1)	7	31	24
Lab V Virotech
Positive (>1.0)	7	0	0
Equivocal (0.1-1.0)	58	7	2
Negative (<0.1)	12	38	26
Lab I VaccZyme
Positive (>0.149)	68	13	1
Retest (0.1-0.149)b	6	12	2
Negative (<0.1)	3	20	25
Lab IV VaccZyme
Positive (>0.149)	60	11	2
Retest (0.1-0.149)b	15	15	2
Negative (<0.1)	2	19	24
Lab VI VaccZyme
Positive (>0.149)	64	11	1
Retest (0.1-0.149)b	10	16	2
Negative (<0.1)	3	18	25
Lab IX VaccZyme
Positive (>0.149)	75	18	1
Retest (0.1-0.149)b	2	18	7
Negative (<0.1)	0	9	20

^aAccording to the reference lab IX NT, 77/150 samples were positive, 45 samples were equivocal, and 28 samples were negative. Labs X, XI, and XII tested only 146, 147, and 140 samples, respectively.

^b“Retest,” sample to be retested.

Lab X tested the panel using the Serion ELISA kit. Data analysis has been performed on 146/150 sera, as there was not sufficient material for four sera. The test showed a poor correlation with NT of R = 0.60 (Fig. 3C). Applying the criteria of interpretation of results reported in the kit instructions and shown in Table 3, 7% of the NT-positive sera were identified accordingly and 61% of them were identified as equivocal (Table 3). Eighty percent of the NT equivocal sera were categorized as negative.

Lab VII used the Euroimmun ELISA kit, which showed a correlation of R = 0.70 with respect to lab IX NT, with a slope from the identity line of 0.47 (Fig. 3D). Many of the NT-positive sera were classified as negative (58%) or equivocal (38%) (Table 3). Ninety-one percent of NT equivocal sera were categorized as negative.

Two labs (I and V) tested the standard panel serum with the Virotech ELISA kit. The interassay precision between the two laboratories, expressed as the mean coefficient of variation, was determined as 8%. The correlations with lab IX NT by lab I and lab V using this ELISA kit corresponded to R = 0.75 (Fig. 3E) and R = 0.74 (data not shown), respectively. Both labs classified the negative sera almost in agreement with NT (Table 3); the concordance for positive and equivocal sera was poor due to the majority of NT-positive sera being classified as equivocal and the NT equivocal sera as negative (Table 3).

Lab IV using the NovaLisa ELISA obtained a correlation of R = 0.77 with lab IX NT (Fig. 3F), with a slope from the identity line of 0.40. The cutoffs indicated by this kit to classify the sera in terms of diagnostic interpretation are equivalent to those used for NT (Table 3). The lab identified correctly only 3 samples out of 28 negative sera, and only 64% of the NT-positive samples were categorized as positive by ELISA (Table 3).

Four labs (I, IV, VI, and IX) tested the standard panel serum with the VaccZyme ELISA kit. The interassay precision between the four labs, expressed as the mean coefficient of variation, was 7%. The correlation coefficients obtained by comparing the results from each of the four labs with the lab IX NT ranged between 0.82 and 0.85. Data are shown for only lab I in Fig. 3G. The instructions for this kit, contrary to the others, do not give a precise indication of how to interpret the level of antibodies obtained for diagnostic purposes. The manufacturer recommends the use of an equivocal zone of 0.1 to 0.149 IU/ml and that samples falling within the zone should be repeated to confirm that protective levels of anti-diphtheria toxin antibodies are present or not. If the level of protection cannot be confirmed, the sample should be referred to a reference laboratory for further testing or a second sample requested. On this basis, therefore, the sera were classified as positive when the antibody level was >0.149 IU/ml and negative when the level was <0.1 IU/ml. All four labs reported sera within this range which were either NT equivocal sera or NT-positive sera; 80% of NT-negative sera were classified correctly by the ELISA.

DISCUSSION

EQA studies, proficiency testing studies, and interlaboratory comparisons are important studies that allow labs to identify testing problems, compare methods, and evaluate and eventually improve their performance. Qualified labs, according to ISO/IEC 17025 (12a), are required to participate in these kinds of studies on a regular basis. The results of this EQA study clearly demonstrate the relative performance of laboratories using their routine assay (in-house or commercial) and provide the participants with an opportunity to assess their own performance in comparative terms—something that is only possible when EQA studies such as this are performed. The goal of this study was not to identify the best assay, as the advantages and drawbacks of all methods used by the participants are well known, but rather to verify if the lab using a specific method would have categorized a serum sample in agreement with the NT. In fact, the capacity of the lab to reliably detect the specific anti-diphtheria toxin levels is crucial for case management.

To organize an EQA for serology requires precise and intense work. In order to achieve a panel of 150 sera, representing, according to WHO classifications (6) negative, equivocal, and positive sera for diphtheria antitoxin, at least 300 sera need to be tested. Possibly, it may be preferable to obtain sera from blood donors, as these are regularly checked for the absence of infectious agents. This safety aspect is relevant both for the analyst (although the samples still have to be handled as potentially infective) as well as for shipping purposes. In addition, when collecting human sera, the consensus of the donor must be acquired and all administrative, legal, and ethical issues must be respected. Furthermore, as pooling of sera is not recommended, in general, a large volume of blood is required from each donor, as the number of EQA participants will affect the number of serum aliquots that need to be prepared.

In this study, serology for diphtheria was performed by selected laboratories using different methods. The Vero cell NT is considered to be the in vitro gold standard assay, and this assay was chosen as the reference test. Because of some variability between four laboratories performing the NT assay, the values derived by the testing of the standard panel by lab IX have been used to compare the performance of all other participants.

Lab IX, as well as labs I and XII, participated in previous seroepidemiological studies for diphtheria and used the same assays (26). The correlations of lab I dDA-DELFIA with lab IX NT were 0.92 and 0.89 in ESEN 1 and in DIPNET (this study), respectively; the correlations of lab XII ToBI with lab IX NT were 0.96 and 0.92 in ESEN 1 and in DIPNET, respectively. Lab IX NT and DAE also remained constant over the years, with corrrelations of 0.95 in ESEN 1 and 0.92 in this study. The comparable correlations between lab I and lab XII with lab IX after several years indicate that changes in critical reagents, as well as analysts, appear to be consistently supporting the use of lab IX as the reference laboratory.

Our interlaboratory comparisons showed that the quantitative correlation between the NTs performed in the different labs is high, even when different protocols and key reference reagents, such as reference antiserum and toxin, are used. However, qualitative comparison showed that some borderline sera were classified differently by NT from each laboratory, but no sera were considered as false negative or as false positive. Lab VI could consider performing the assay at a different toxin dose level to obtain a lower limit of detection in order to be more precise in the classification of serum samples with low levels of diphtheria antibody.

PHA is a demanding test that requires high expertise and is known to be difficult to standardize (21, 22, 27). The degree of correlation between individual serum antitoxin titers obtained by lab II PHA and lab IX NT is low. The assay is sensitive (with a reported detection limit of 0.01 IU/ml), but the diagnostic accuracy is low, particularly at low levels of functional antibody, as determined by NT assay. This result is not uncommon, as a previous study by Walory et al. also reported a low correlation comparing PHA with NT (R = 0.34) (27). The results obtained by lab II using PHA disqualify its use, unless improved performance can be achieved.

Only a few laboratories use alternative in vitro tests such as DAE, dDA-DELFIA, or ToBI, which are not commercial methods and therefore require in-house installation and validation. These methods are complex, are not as easy as ELISA, and depend on critical reagents that are not necessarily readily available commercially, such as labeled toxoids, or special buffers in the case of dDA-DELFIA. It is more difficult to achieve reproducible results with these methods, and internal controls are very important. The results obtained by the three labs using these methods were satisfactory, even though lab I needed to adjust the cutoff value for negative sera, shifting it from 0.01 IU/ml to 0.015 IU/ml in order not to classify as an equivocal value a serum that is negative in the NT. This can be relevant in certain instances, because if a subject is considered negative for diphtheria toxin antibodies, it might be possible that he/she requires three doses of vaccine (complete immunization schedule), while an intermediate level of antibodies would require only a booster dose to restore protection.

The majority of labs (9/12) participating in the study performed an indirect ELISA. Three labs (lab I, IX, and XI) took the opportunity of participation in the EQA to also perform an ELISA in parallel with their in vitro more-demanding assay for testing of the serum panel. While the quality of an in-house ELISA is dependent on the validation performed by the individual lab, the commercial ELISA kits are sold as supposedly validated methods inclusive of all key reagents and reference sera to allow calculation of the diphtheria antitoxin concentration in human serum samples. Two ELISA kits, Virotech and VaccZyme, were used by more than one participant and were shown to provide reproducible results between labs, reflecting the ease of use and robustness of the kit. Thus, the results obtained by commercial ELISA can be considered not only a measure of the lab's performance but also of the ELISA kit itself.

ELISA kits are widely used. However, from this EQA, it is evident that some labs did not get sufficiently accurate results as correlation coefficients with the reference NT test from lab IX were below 0.85. The in-house ELISA performed by lab XI, using standardized reagents, was the one with highest correlation with NT. The results from this assay were used to analyze the correlation between ELISA and NT, separating NT sera with antibody levels of <0.01 IU/ml, 0.01 to 0.09 IU/ml, and ≥0.1. There is a greater deviation from the equality line between the two methods at lower functional antibody titers (NT, <0.1 and 0.01 to 0.09 IU/ml). Lack of correlation between ELISAs and NT for human serology has already been well reported in the literature (3, 9, 17, 21, 22), as in the NT range of 0.001 to 0.01 IU/ml, the determined ELISA value can be 10 to 100 times higher (17, 27). The lack of correlation with NT in terms of antibody levels detected is intrinsic to the ELISA that is measuring not only functional antibodies but also IgG binding to a variety of epitopes of the diphtheria toxin/toxoid. The differences in the correlations of ELISAs from different manufacturers with NT might be due to the different purities of the diphtheria toxoid used as the coating antigen.

Diagnostic agreement between ELISAs and NT is evaluated using generally different cutoffs, and these cutoffs are 10 times higher than those applied for NT. Selection of cutoffs has a direct influence in terms of diagnostic interpretations of the immune status of a person and consequently on the decision whether or not to reimmunize, as well as assessing potential deficiencies in humoral immunity. Usually, in ELISA, sera with antibody levels of <0.1 IU/ml are considered negative and those with antibody levels of ≥0.1 IU/ml are considered to be positive (9, 20). However, for some of the ELISA kits used by the participants in this EQA (Serion, Euroimmun, and Virotech), the manufacturers’ recommended division of test sera into the three categories was used for the NT (positive, equivocal, and negative) but with titers set 10-fold higher than those used in the NT. We have used in our study, for qualitative agreement analysis, only three categories (i.e., negative, equivocal, and positive), with no differentiation between positive sera containing antibody levels that confer short- or long-term protection.

On the basis of this qualitative classification, it is evident from the EQA study that labs using ELISA can underestimate the immune status of a subject. In fact, three labs using three different commercial ELISA kits (Serion, Euroimmun, and Virotech) and following the manufacturer's instructions, classified NT-positive sera as equivocal or negative. The application by NovaLisa ELISA of NT cutoffs for qualitative classification of sera led to an erroneous classification of both positive and negative sera. The clinical implication of over- or underestimation of diphtheria antibody titers would be that some subjects may be wrongly assumed to need or not need immunization.

Laboratories that participated in this serological EQA for diphtheria were generally satisfied. For some, this study provided an opportunity to compare the performance of their assay for the first time, while for others who had participated in similar studies previously, this study provided reassurance that their routine assay was still performing as expected, with no changes over time. The EQA results for some labs using ELISA kits were disappointing.

Furthermore, it is also evident that irrespective of the method used, it is important to include well-defined key reagents in the assay, such as an international reference antiserum, and it may also be of value to include a panel of control sera of defined activity (e.g., 1.0, 0.1, 0.01, and 0.001 IU/ml). This could alert the user when the method is not performing as expected, which may be due to analytical error or loss of stability of one or more key reagents. Because diphtheria is now a rare disease in Western Europe, the participation of national reference centers in EQA schemes for serology is very important, as it provides a good opportunity to compare and monitor assay performance, thus maintaining confidence in results generated for clinical purposes in the region.