Comparison of commercial enzyme-linked immunosorbent assays and agar gel immunodiffusion tests for the serodiagnosis of equine infectious anemia

Abstract

The purpose of this study was to estimate the performance characteristics (accuracy, detection limit, and precision) of commercially available enzyme-linked immunosorbent assay (ELISA) and agar gel immunodiffusion (AGID) kits in comparison with a reference AGID kit for the detection of equine infectious anemia (EIA) antibodies in horses for regulatory use in Canada. A total of 285 positive and 315 negative samples by the reference AGID were tested blindly on 2 other AGID and 4 ELISA kits. Commercially available AGID kits for the serodiagnosis of EIA were found equivalent. The 3 ELISAs directed against antibodies to the p26 core protein also performed relatively well in comparison with the reference AGID, with excellent relative accuracy and acceptable precision. The single ELISA directed against antibodies to the gp45 trans-membrane viral protein yielded a lower relative sensitivity. The performance characteristics of the ELISAs directed against antibodies to p26 are, therefore, adequate to support the implementation of ELISA for regulatory purposes in Canada.

Introduction

Equine infectious anemia (EIA) is a retroviral disease of all equidae, including horses, mules, and donkeys, that falls under a regulatory control program in many industrialized countries. In Canada, this reportable disease falls under the Health of Animals Regulations sponsored by the Canadian Food Inspection Agency (CFIA). In this country, horse EIA monitoring is encouraged for attendance at races, shows, fairs, sales, breeding farms, etc. Following a positive status, an infected animal is usually euthanized and contact horses are put under a federally-imposed quarantine until confirmation of their negative status.

The diagnosis of EIA in horses is done serologically. Seropositivity is a good indication of infection because horses infected with the EIA virus carry it for life (1), with the development of a sustained antibody response appearing usually within 45 d of viral infection (2). The agar gel immunodiffusion test (AGID) (2), prescribed by the Office International des Épizooties (OIE) for international trade (3), is currently in use in Canada for its EIA control program. This test, specific to the p26 core viral protein, is relatively rapid, inexpensive, simple, and highly specific to identify animals infected with the EIA virus, although it is interpreted subjectively by visual reading of precipitation line curvature.

During the last few years, the detection of EIA antibodies by ELISA has been described and used in some countries where this test is commercialized under various formats. In the United States (USA), a few ELISAs have been approved by the US Department of Agriculture Animal and Plant Health Inspection Service (USDA:APHIS) in the 1990’s as equivalent test methods for the diagnosis of EIA (4,5). Validation studies have indicated excellent agreement between these ELISAs and the AGID assay (4,5), with the ELISA being found, in some cases, to be even more sensitive than the AGID (6–9).

The objective of this study was to estimate the performance characteristics (accuracy, detection limit, and precision) of commercially available ELISA and AGID kits from the USA in relation to the officially approved reference AGID assay in Canada, and to determine if, in the current Canadian context, the ELISA could also be adopted as an official test method for the diagnosis and control of EIA in this country.

Materials and methods

Commercial assays tested

The following tests and methods are consistent for the accuracy, detection limit, and precision testing and, therefore, are described first. Two AGID and 4 ELISA kits commercially available in the USA for the serodiagnosis of EIA were compared to the reference EIA-AGID kit currently in use in Canada (DiaSystems EIA AGID; IDEXX Laboratories, Maine, USA). All 3 AGID kits were directed against the p26 core viral protein. The AGID # 1 and the AGID # 2 were, respectively, the EIA antibody test kit (VMRD, Washington, USA) and the LAB-EZ/EIA (Synbiotics Corporation, California, USA). Technical specifications for the ELISA # 1 (DiaSystems EIA CELISA;IDEXX Laboratories), ELISA # 2 (ViraCHECK/EIA; Synbiotics Corporation), ELISA # 3 (SA-ELISA; Centaur, Kansas, USA), and ELISA # 4 (SA-ELISA-II; Centaur) are presented in Table I. All testing was performed at the Retrovirology Centre of Expertise CFIA laboratories in Charlottetown, Prince Edward Island and in Saint-Hyacinthe, Quebec.

Table I.

Specifications of the enzyme-linked immunosorbent assay (ELISA) kits

	ELISA # 1	ELISA # 2	ELISA # 3	ELISA # 4
ELISA type	Competitive	Indirect	Indirect	Indirect
Antigen(s) targeted	p26	p26	gp45	p26 + gp45
Wave length (nm)	650	630	450	450
Cutoff	< = PC	> = PC	> NC	> NC
Incubation temperature	37°C	RT	RT	RT
Test time	< 1 h	< 30 min	< 1 h	< 1 h

PC — positive control; NC — negative control; RT — room temperature

Kits were stored as per the manufacturer’s recommendations. All tests were performed blindly by technicians according to the instructions on the insert provided by each manufacturer (Table I). For the accuracy study of the ELISA # 4, however, the length of the serum incubation time was increased from 15 to 20 min in our laboratory in order to obtain the optical density (OD) required for the positive control to be valid.

All of the AGID tests were read and interpreted independently by at least 2 EIA certified analysts, both after 24 and 48 h of incubation, as suggested by the manufacturers. The AGID tests were declared valid only if the negative and positive controls (provided by the CFIA Retrovirology Centre of Expertise) included on each test plate yielded expected results. In case of a disagreement in interpretation between analysts, the sample was retested. Samples for which readers disagreed on both tests were declared suspicious.

All ELISA tests were interpreted visually by 3 independent analysts and were also read using a spectrophotometer at the suggested wavelength. The ELISA tests were declared valid only if the negative and positive controls (provided by the manufacturers) included on each test plate yielded expected results. For visual interpretations, the samples on which the analysts disagreed were declared suspicious.

Accuracy testing

An accuracy study of 6 AGID and ELISA tests was undertaken from 1999 until 2002 to estimate the sensitivity and specificity of each kit relative to the reference AGID test. In total, 600 distinct horse serum samples available from the diagnostic laboratory of the CFIA Retrovirology Centre of Expertise were used. Prior to the comparison study, all samples were tested twice with the reference AGID and yielded unequivocal test results. From those results, 285 samples were positive by AGID whereas 315 samples were AGID negative. Samples were homogeneized, aliquoted in many replicates, and stored at –20°C ± 5°C until use. To avoid alteration due to freeze-thaw cycles, a new aliquot was used while conducting each test.

Because of a limited number of kits of ELISA # 3 were purchased, only 216 positive and 244 negative samples were tested for this assay. The ELISA # 3 was no longer available commercially after the summer of 2001. Also, due to sample depletion at the time the ELISA # 4 was evaluated, 270 EIA-positive and 313 EIA-negative samples were used in its evaluation. All kits from the same manufacturer were of the same lot number.

The relative sensitivity and specificity compared to the reference AGID were estimated for each kit and a 95% confidence interval was computed around those estimates, assuming a normal distribution (10). The ‘relative’ sensitivity and specificity were estimated because we compared these immunological test kits to an immunological reference kit without other confirmation of animal infection status. A kappa value was also computed between results for AGID/ELISA assays and the reference AGID. To obtain conservative estimates for kappa and relative sensitivity computations, all reference positive samples that yielded suspicious results on AGID or ELISA were considered negative. Similarly, all reference negative samples that yielded suspicious results on AGID or ELISA were considered positive for kappa and relative specificity computations.

Detection limit testing

In 1999, the detection limits of the AGID # 1, AGID # 2, ELISA # 1, and ELISA # 2 were compared with the reference AGID by testing serial dilutions of positive and negative samples. Fifteen representative samples of various AGID positivity levels (from weak to strong) and 3 AGID negative samples were tested undiluted and diluted, with EIA negative control serum, from 1:5 to 1:1600. The frequencies of endpoint dilutions for each kit were tabulated. Again, all kits from the same manufacturer were of the same lot number.

Precision testing

In 2002, a precision study was undertaken to estimate the intraplate and interlot variability of the ELISA kits. Because ELISA # 3 was no longer available at the time of testing, it was not included in this part of the study.

The repeatability (intra-plate variability) was assessed by testing 10 samples (4 reference negative and 6 reference positive samples) 6 times on the same plate, on the same day, by the same analyst. This was performed on 2 different lots of each kit. The median, minimum, and maximum coefficients of variation (CVs) were computed for each lot. Percent correct classification was also computed.

The reproducibility (interplate variability) of the test results was assessed by testing 30 samples (11 reference negative and 19 reference positive samples) on 2 (ELISAs # 1 and # 2) or 3 (ELISA # 4) distinct lots, on different days, by different analysts. The median, minimum, and maximum coefficients of variation (CVs) were computed. Percent correct classification was also computed.

Results

Accuracy testing

The 2 AGID test results were highly correlated with the reference AGID (Table II). All results were identical for all AGID kits except for 3 reference positive samples (as determined by the reference AGID). For these 3 samples, AGID # 1 yielded a suspicious result at 24 and 48 h for one sample, a negative result at 24 h and a suspicious result at 48 h for the 2nd sample, and a positive result at 24 h and a suspicious result at 48 h for the 3rd sample.

Table II.

Relative sensitivity (se) and specificity (sp) (95% confidence interval) of the equine infectious anemia (EIA) agar gel immunodiffusion (AGID) kits relative to the reference AGID. Results presented were obtained after 24 h and 48 h of incubation

	24 h incubation			48 h incubation
	Relative se	Relative sp	kappa	Relative se	Relative sp	kappa
AGID # 1	99.3% (97.2% to 99.9%)	100% (99.1% to 100%)	0.993	98.9% (96.7% to 99.7%)	100% (99.1% to 100%)	0.990
AGID # 2	100% (99.0% to 100%)	100% (99.1% to 100%)	1.000	100% (99.1% to 100%)	100% (99.1% to 100%)	1.000

The distribution of the OD read by a spectrophotometer for the ELISAs are presented in Figure 1. Among the indirect assays, ELISA # 2 presented no overlap between the reference positive and negative samples. The ELISAs # 3 and # 4 presented much higher OD than the 2 other kits for reference positive samples. However, there was a clear overlap between the reference positive and negative samples in the low OD region, particularly for the ELISA # 3. The ELISAs # 1, # 2, and # 4, all directed against antibodies to the p26 viral protein, were in excellent agreement with the reference AGID test when read spectrophotometrically (Table III). The relative sensitivity of ELISA # 3, targetting only antibodies to the gp45 protein, was significantly lower than that of the other ELISAs.

Figure 1.

Distribution of optical densities (OD) of equine infectious anemia (EIA) antibodies for samples positive and negative by reference agar gel immunodiffusion(AGID), for enyme-linked immunodiffusion assay (ELISA) # 1, ELISA # 2, ELISA # 3, and ELISA # 4. Note that the OD scale ranges from 0 to 0.8 for ELISAs # 1 and # 2, from 0 to 2.5 for ELISAs # 3, and from 0 to 4.0 for ELISA #4.

Table III.

Relative sensitivity (se) and specificity (sp) (95% confidence interval) of the equine infectious anemia (EIA) enyme-linked immunosorbent assay (ELISA) kits relative to the reference agar gel immunodiffusion (AGID). Results presented were obtained by spectrophotometric reading and visual interpretation

	Spectrophotometric reading			Visual interpretation
	Relative se	Relative sp	Kappa	Relative se	Relative sp	Kappa
ELISA # 1	100% (99.0% to 100%)	93.3%(89.9% to 95.7%)	0.930	99.7% (97.8% to 100%)	99.7% (98.0% to 100%)	0.993
ELISA # 2	100% (99.0% to 100%)	100% (99.1% to 100%)	1.000	99.7% (97.8% to 100%)	100% (99.1% to 100%)	0.997
ELISA # 3	86.6% (81.1% to 90.7%)	100% (98.8% to 100%)	0.872	87.0% (81.6% to 91.1%)	100% (98.8% to 100%)	0.877
ELISA # 4	97.4% (94.5% to 98.9%)	97.8% (95.2% to 99.0%)	0.952	94.4% (90.8% to 96.7%)	99.7% (98.0% to 100%)	0.945

When the ELISA reaction was interpreted visually (Table III), high agreements were obtained between the reference AGID and ELISAs # 1 and # 2, beyond that expected by chance alone. For ELISA # 1, 1 reference positive and 1 reference negative sample were declared suspicious. Similarly, 1 reference positive sample was declared suspicious on ELISA # 2. The relative specificity of the ELISA # 1 interpreted visually was higher than when a spectrophotometer was used. For ELISA # 3 interpreted visually, 27 reference positive samples were declared negative and 1 reference positive was declared suspicious, reflecting the lower sensitivity of this test. For ELISA # 4 interpreted visually, 9 reference positive samples were declared negative, 6 reference positive samples were declared suspicious, and 1 reference negative was declared positive, yielding a lower relative sensitivity than ELISAs # 1 and # 2.

Detection limit testing

The detection limits were different for ELISAs # 1 and # 2 (Table IV). For each positive sample, ELISA # 1 was able to detect antibodies at dilutions at least as high, or higher than the reference AGID. In most cases (87%), ELISA # 2 detected antibodies at dilutions within one dilution of what was detected by the reference AGID. Detection limits were similar for AGID # 2 and the reference AGID. The AGID # 1 appeared slightly less sensitive than the reference AGID. All of the serial dilutions of the reference negative samples were also found negative by all of the assays.

Table IV.

Comparison of number of equine infectious anemia (EIA) positive samples for serial dilutions from neat to 1/1600 tested by reference agar gel immunodiffusion (AGID), AGID # 1, AGID # 2, enzyme-linked immunosorbent assay (ELISA) # 1, and ELISA # 2

Sample dilution	Reference AGID	AGID # 1	AGID # 2	ELISA # 1	ELISA # 2
Neat, 1/5, 1/10	15	15	15	15	15
1/25	15	8	15	15	12
1/50	12	4	8	15	7
1/100	6	1	3	13	3
1/200	0	0	0	9	2
1/400	ND	ND	ND	5	0
1/800	ND	ND	ND	2	0
1/1600	ND	ND	ND	2	0

ND — Not done, expected negative

Precision testing

For the repeatability trials (intraplate variability), a 100% correct classifications were obtained for each lot of the ELISA kits that were tested (# 1, # 2, # 4), for both visual and spectrophotometric interpretations. For ELISA # 1, the median CVs were 9% and 12% for lot 1 and 2, respectively, and individual CVs varied from 1% to 18%. For ELISA # 2, the median CVs were 12% and 13% for lot 1 and 2, respectively, and individual CVs varied from 2% to 23%. For ELISA # 4, the median CVs were 7% and 12% for lot 1 and 2, respectively, and individual CVs varied from 1% to 41%. All median CVs were therefore less than 15% and higher individual CVs were, as expected, associated with low OD serum, because even small variations in OD can produce higher CVs when divided by a low mean OD (CV = variance/mean).

For the reproducibility assessment, ELISA # 2 had the lowest median coefficient of variation (CV = 6%, min 0%, max 29%). The ELISAs # 1 and # 4 had similar CVs with a median of 16% (min 0%, max 36%) and 15% (min 1%, max 33%), respectively. All 30 samples were correctly classified on each lot for every ELISA tested using visual and spectrophotometric interpretations.

Discussion

According to the validation study presented here, the commercially available AGID kits for the serodiagnosis of EIA that were tested are apparently equivalent. The ELISA kits directed against antibodies to the p26 core protein (# 1, # 2, and # 4) also performed relatively well in comparison with the reference AGID. As previously reported (4–9), their relative sensitivity and specificity compared to the reference AGID was excellent. In contrast to a previous report (11), ELISAs # 1, # 2, and # 4 had acceptable interplate variability and provided valid results.

The p26 derived competitive ELISA (ELISA # 1), had a higher relative specificity by visual interpretation compared to spectrophotometric readings. This may be due in part to an inadequately pre-determined cut-off used for the spectrophotometric interpretations suggested by the manufacturer. Further evidence of this inadequate cut-off may be seen from the detection limit trial, where ELISA # 1 was found to have a greater analytical sensitivity (lower detection limit) than the AGID assays and the ELISA # 2. This enhanced sensitivity may be obtained at the cost of a reduction of specificity of the spectrophotometric reading (5,12). This aspect remains to be examined.

The ELISA # 2 provided nearly perfect agreement with the reference AGID and presented the least interplate variability. Also, in the detection limit trial, positive sera had nearly the same endpoint dilutions for the ELISA # 2 and the reference AGID. These results suggest that ELISA # 2 may be used interchangeably with the AGID test.

The ELISA # 4, although it targets antibodies to both the trans-membrane (gp45) and core (p26) antigens, did not perform better than the other ELISAs evaluated. Similarly to other kits, it provided an acceptable relative sensitivity, but this was lower than for ELISAs # 1 and # 2. However, the relatively high OD obtained by the positives on this ELISA made it the easiest test to read visually.

The gp45 derived ELISA (ELISA # 3) was very specific but, likely because it was lacking the detection capabilities of the antibodies directed against the p26 core protein, had the lowest relative sensitivity in comparison with the reference AGID and the other ELISAs. Perhaps because of inconsistent antibody response against gp45 (4) or the antigenic variations observed in the gp45 protein (13,14), this kit could not provide a reliable and acceptable relative sensitivity. The ELISA # 3 was discontinued by the manufacturer.

The ELISA formats have the advantages of allowing efficient and accurate testing of large numbers of serum samples within a relatively short time and of providing the potential for an objective interpretation by the use of a spectrophotometer. Because our data indicated excellent performances of the p26-based ELISAs available commercially in USA in comparison with the reference AGID test in use in Canada, the implementation of these ELISAs for the EIA control program in this country is recommended. As suggested by the OIE (3), the AGID could remain a confirmatory test for horses with a positive ELISA status.