Validation of a Competitive Enzyme-Linked Immunosorbent Assay for Detection of Antibodies against Babesia bovis

Abstract

A previously developed competitive enzyme-linked immunosorbent assay (cELISA) based on a species-specific, broadly conserved, and tandemly repeated B-cell epitope within the C terminus of rhoptry-associated protein 1 of Babesia bovis was refined and validated for use internationally. Receiver operating characteristic analysis revealed an assay with a specificity and positive predictive value of 100% and a sensitivity of 91.1%, with various negative predictive values depending on the level of disease prevalence. The cELISA was distributed to four different laboratories, along with a reference set of 100 defined bovine sera, including known-positive, known-negative, and field samples. Pairwise concordances among the four laboratories ranged from 94% to 88%. Analysis of variance of the resulting optical densities and a test of homogeneity indicated no significant difference among the laboratories. Overall, the cELISA appears to have the attributes necessary for international application.

Bovine babesiosis is a tick-transmitted disease caused by one or more species of intraerythrocytic protozoa. While definitive diagnosis can be made by detecting infected erythrocytes in stained blood films, the parasitemia in peripheral blood is often too low to use this method reliably for diagnostic purposes. Serological testing is used as a tool to determine the babesial infection statuses of individuals and herds in control programs and, in the absence of treatment, is the best indicator of persistently infected hosts. Although a number of serologic assays have been developed and used for several years, they all suffer to some extent in sensitivity, specificity, or objectivity or are cumbersome for testing large numbers of samples. For these reasons, we developed a competitive enzyme-linked immunosorbent assay (cELISA) (8) based on the ability of serum antibody to inhibit a monoclonal antibody (MAb) directed against a Babesia bovis-specific repetitive epitope within rhoptry-associated protein 1 (RAP-1) (14-16). In a previous study, we demonstrated the high specificity afforded by use of a purified recombinant RAP-1 C-terminal construct as an antigen and inhibition of binding of MAb to a B. bovis RAP-1-specific epitope by serum antibodies. In addition, we established the ability of the assay to detect both relatively early and long-term-carrier infections. However, we did not fully validate the assay or attempt to determine the assay's reliability in laboratories outside of ours. In the study reported here, we further validated the assay and provided evidence for its utility for global use through an international interlaboratory comparison.

MATERIALS AND METHODS

Sera.

For the extended bench validation, 135 known-positive and 141 known-negative sera were evaluated. The known-positive sera were from animals that had experienced an experimental infection (119 samples) or that were from regions of endemicity in Puerto Rico and were determined to be positive by a B. bovis-specific nested PCR (nPCR) (16 samples). All nPCR-positive results were confirmed by sequence analysis. Known-negative samples (141 samples) were from the northwest United States, where neither B. bovis nor Boophilus microplus tick vectors exist. For interlaboratory comparison, a set of 100 defined sera were distributed. The sera represented 24 known-negative sera from Tasmania, where B. bovis and its tick vector are known to be absent; 26 known-positive sera from cattle experimentally infected at the Tick Fever Research Center in Australia and confirmed infected by microscopic detection in stained blood films; 32 sera of unknown status from nonvaccinated cattle raised in the region of endemicity in northern Australia; and 18 sera from cattle vaccinated once with the Dixie strain of B. bovis (4). The sera from Australia were preserved with sodium azide and sent to our ARS laboratory in the United States. Whole blood was collected in Puerto Rico, allowed to clot, and then sent to our laboratory, where the serum from each sample was separated.

IIF.

The indirect immunofluorescence (IIF) assay was performed as previously described (6) using 10 μl of a 1/100 dilution of serum. A positive result was defined as fluorescence equal to or greater than that of a weak positive control sample.

PCR samples and reaction conditions.

External and nested primers were previously described (5) and produced a nested amplification product of approximately 297 base pairs from within the gene coding for RAP-1. The sensitivity of the nPCR was determined on dilutions of blood containing a known number of infected erythrocytes confirmed by real-time PCR (rtPCR). SYBR green-based rtPCR (Bio-Rad, Hercules, CA) was performed with the following conditions: 5 min at 95°C; 45 cycles of 95°C for 15 s, 58°C for 30 s, and 72°C for 45 s; and 72°C final extension for 7 min with a hold at 10°C. The PCR mixture contained 20 mM Tris (pH 8.4); 50 mM KCl; 3 mM MgCl; 200 μM (each) dATP, dGTP, dCTP, and dTTP; a 0.25 μM concentration of each primer; 25 U/ml iTaq DNA polymerase; and 10 nM fluorescein stabilizer and SYBR green dye. Each reaction was performed with the iCycler iQ real-time PCR detection system (Bio-Rad). Initially, rtPCR was standardized on dilutions of pCR2.1 (Invitrogen, Carlsbad, CA) containing the full-length rap-1 gene (TOPO-rap-1) and using the same internal primers as for nPCR (see below) (Fig. 1A). These primers amplified a fragment of 297 bp from nucleotides 690 to 987. rtPCR was then performed with genomic DNA (gDNA) from known numbers of cultured B. bovis-infected erythrocytes (MO7 reference biological clone) (10) and compared with the rap-1-containing plasmid to establish a genomic-control standard curve (Fig. 1B). For the nPCR, the external-reaction primers were forward, 5′-CACGAGGAAGGAACTACCGATGTTGA-3′, and reverse, 5′-CCAAGGAGCTTCAACGTACGAGGTCA-3′, and the nested primers were forward, 5′-TCAACAAGGTACTCTATATGGCTACC-3′, and reverse, 5′-CTACCGAGCAGAACCTTCTTCACCAT-3′. The external reaction was performed in a 50-μl volume containing 2.0 μl of sample gDNA in Tris buffer, 1.5 μl of 50 mM MgCl₂, 5.0 μl 10× reaction buffer, 1.0 μl of 10 mM deoxynucleoside triphosphate mix, 1.0 μl of external primers (50 pmol/μl of both forward and reverse primers), 0.4 μl Taq polymerase (5 units/μl), and 39.1 μl of ultrapure water. The primary amplification was carried out with 25 cycles of 95°C for 1 min, 60°C for 1 min, and 72°C for 1.5 min, with a final extension time of 7 min at 72°C. For the nested reaction, 1 or 2 μl from the primary reaction was used under the same conditions but increased to 35 cycles. The nPCR products were examined following 2% agarose gel electrophoresis (Fig. 1C).

FIG. 1.

Validation of the Babesia bovis-specific nested PCR. (A) Real-time PCR detection of the B. bovis rap-1 gene (297 bp). The standard curve was generated using TOPO-RAP-1. On the x axis, the log starting quantity of template is equal to the copy number. Correlation coefficient, 0.996; slope, −3.362; intercept, 33.778; y = −3.362x + 33.778. (B) Real-time PCR detection of the B. bovis rap-1 gene from dilutions of MO7 B. bovis genomic DNA extracted from cultured infected erythrocytes. On the x axis, the log starting quantity of template is equal to genomic units. Correlation coefficient, 0.992; slope, −3.597; intercept, 33.418; y = −3.597x + 33.418. (C) Nested PCR of MO7 B. bovis genomic DNA extracted from 10-μl volumes of 50% suspensions of erythrocytes spiked with 10-fold dilutions of infected erythrocytes beginning with 10⁸/ml. big, Babesia bigemina genomic DNA control.

The B. bovis-specific nPCR was then used to evaluate DNA from 10 μl of washed, packed erythrocytes from EDTA blood samples collected in Puerto Rico, along with uninfected erythrocytes and Babesia bigemina-infected erythrocytes as controls. nPCR-positive samples were sequenced to verify that the amplification was B. bovis specific. Briefly, the PCR product was cloned into pCR2.1 (Invitrogen) and plated on LB-plus-ampicillin plates containing X-Gal (5-bromo-4-chloro-3-indolyl-β-d-galactopyranoside). Colonies containing inserts were picked and screened with the B. bovis nested primer set (45 cycles). Those whose PCR products were of the correct size (∼297 bp) were grown overnight in LB plus ampicillin, and the plasmid isolated using a miniprep kit (Wizard-Plus SV; Promega Inc., Madison, WI) was sequenced by conventional methods (Amplicon Express, Pullman, WA).

cELISA.

The format of the cELISA was as previously described using the same C terminus of RAP-1 antigen expressed as a histidine-tagged thioredoxin fusion protein purified on a ProBond resin column (Invitrogen, Carlsbad, Calif.) and dried on the wells of Immulon II plates (8). Optimal concentrations of antigen and MAb were determined by block titration as previously described (8), and a plate washer (BioTek Instruments Inc., Winooski, VT) was used at each step of the protocol. Other slight modifications were made after additional experience in using the assay. Prior to use, 200 μl of a blocking buffer (phosphate-buffered saline containing 0.2% Tween 20 and 20% nonfat dry milk) was added to each well, and the plates were incubated at room temperature for 1 h on a rotating platform. After the blocking buffer was aspirated off, 100 μl of undiluted serum was added to each well (principal test sera and control sera), and the plates were incubated at room temperature for 30 min. After the serum from each well was aspirated off, 100 μl (50 ng/well) of BABB75A4 MAb (7) was added, and the plates were incubated at room temperature for 15 min. Each plate was then washed three times with 200 μl of wash buffer (blocking buffer minus the nonfat dry milk), followed by the addition of 100 μl of wash buffer containing an appropriate concentration of conjugate (horseradish peroxidase-labeled goat anti-mouse immunoglobulin G; KPL, Gaithersburg, MD) (each lot of new conjugate was titrated). After incubation at room temperature for 15 min, each plate was washed three times, as before, and then allowed to set for 30 to 60 s in wash buffer before a final three washes. After equal volumes of 3,3′,5,5′-tetramethylbenzidine and H₂O₂ were combined according to the manufacturer's instructions (KPL), 100 μl of the substrate was added to each well and the plates were incubated at room temperature in the dark for 15 min, followed by the addition of 50 μl of stop buffer (2 N H₂SO₄). The mean optical density (OD) at 450 nm was determined for all test wells and for duplicate wells of a positive control serum and negative control sera pooled from five known-negative animals using a microtiter plate reader (Dynex Technologies, Chantilly, VA). The percent inhibition for each test sample was determined using the mean of each duplicate well compared to the mean of duplicate control wells using the following formula: percent inhibition = [1 − (OD of sample - OD of buffer/OD of negative control - OD of buffer)] × 100.

Interlaboratory comparison.

Aliquots of 100 sera (all from Australia, as described above) were sent to each of four laboratories in different countries, along with antigen plates from the same lot and conjugates, substrates, and other reagents from the same lot and/or shipment from the same vendors. All laboratories used the same protocol, described above, and recorded ODs at 450 nm with plate readers. All samples were coded so that the assay in each laboratory was run in a blind fashion.

Statistical analysis.

To accurately assess the assay for diagnostic specificity, sensitivity, and predictive values, the results from the 100 known-positive and 100 known-negative samples were subjected to receiver operating characteristic (ROC) analysis performed using MedCalc statistical software (version 8.1.1), and a frequency distribution graph was generated. Concordance among laboratories was established using Cohen's kappa values (2), Hartley's test for homogeneity (9), and a one-way analysis of variance (ANOVA) of ODs among the four laboratories.

RESULTS

Sensitivity of nPCR.

An rtPCR standard curve of 1 to 10⁷ rap-1 copies was generated by amplifying recombinant rap-1 plasmid (Fig. 1A). gDNA extracted from known numbers of cultured MO7 B. bovis organisms was compared to the plasmid standard demonstrating an equivalent sensitivity (∼10 copies = ∼10 genomic units) (Fig. 1B). Genomic DNA was then extracted from similar cultures and subjected to nPCR, where the same level of detection was demonstrated, indicating that the nested PCR can detect at least 10 infected erythrocytes in the PCR sample (Fig. 1C). Given this level of sensitivity, the nPCR was used to define known-positive samples from Puerto Rico as part of the defined sera used to establish specificity and sensitivity. Sequence analysis of PCR-positive results confirmed the presence of B. bovis DNA in these samples (data not shown).

Specificity, sensitivity, and predictive value.

Figure 2A shows the frequency distribution graph for the cELISA using the 135 known-positive and 141 known-negative samples. Based on ROC analysis (Fig. 2B), 21% inhibition was chosen as the threshold value to define a positive or negative sample, yielding a specificity of 100%, a sensitivity of 91.1%, and a positive predictive value of 100%. Using the 21% inhibition cutoff, the negative predictive value varied, depending on the prevalence of disease in a particular area, ranging from 99.0% at a prevalence of 10% to 55.6% at a prevalence of 90% (Fig. 3).

FIG. 2.

(A) Frequency distribution of known-negative (left cluster) and known-positive (right cluster) sera. A sample was considered positive with an inhibition equal to or greater than 21%. (B) ROC plot (solid line) of sensitivity and specificity with 95% confidence levels (broken lines) calculated from the 135 known-positive and 141 known-negative serum samples, establishing 21% inhibition as the optimal cutoff value (the percent inhibition that corresponds to the maximum diagnostic sensitivity and specificity). A random, no-discrimination line is shown as the 45° broken line.

FIG. 3.

Predictive value of a negative cELISA test result when applied to different levels of disease prevalence. The values were determined by ROC analysis based on a 21% cutoff.

Comparison with indirect immunofluorescence.

The same set of known-positive and -negative sera were tested using a standard IIF assay (6) to determine concordance with the cELISA. As for the cELISA, the IIF correctly identified all known-negative samples. However, 21 known-positive samples were negative by the IIF assay compared to 12 cELISA false-negative results, although not necessarily the same samples (Table 1).

TABLE 1.

Validation of cELISA with IIF

Assay result	Known-positive no. (n = 135)a	Known-negative no. (n = 141)b
cELISA positive	123	0
cELISA negative	12	141
IIF positive	114	0
IIF negative	21	141

^aKnown-positive samples as defined in Materials and Methods.

^bKnown-negative samples as defined in Materials and Methods.

Comparison among laboratories.

Aliquots of a defined set of bovine sera from Australia and other cELISA reagents from the same lot or batch were sent to four different laboratories, each of which used the same protocol and recording methods. A pairwise comparison was done to determine the interlaboratory repeatability of the assay. Concordances with kappa values are shown in Table 2. The results ranged between 88% and 94% agreement, with kappa values all above 0.7, indicating that the assay was reliable. In addition, a one-way ANOVA of the ODs and Hartley's test for homogeneity demonstrated that there was no significant difference among the four laboratories.

TABLE 2.

Pairwise concordance among laboratories^a

Laboratories	Concordance (%)	Kappa valueb
1 vs 2	90	0.7961
1 vs 3	94	0.8796
1 vs 4	88	0.7577
2 vs 3	90	0.7981
2 vs 4	91	0.8148
3 vs 4	90	0.7990

^aHartley's test of homogeneity indicated no significant difference among the laboratories. One-way ANOVA of the ODs among the laboratories with an F value of 0.179 also indicated no significant difference (P < 0.05).

^bKappa values above 7.0 denote satisfactory reliability.

The mean OD among the positive control sera used in all four laboratories, representing 12 antigen plates and 24 wells, was 0.405 ± 0.122, ranging from 0.672 to 0.212. The mean OD among the pooled negative controls used in all four laboratories, representing 12 antigen plates and 48 wells, was 2.262 ± 0.196, ranging from 2.600 to 1.710. Finally, the tightest range in ODs among the 100 test samples used in all four laboratories, representing 12 antigen plates and 800 wells, was 0.292 to 0.265 (a positive sample), and the greatest range was 2.572 to 1.609 (sample 12 in Table 3; three laboratories reported it as negative, and a single laboratory recorded it as positive but near the cutoff inhibition).

TABLE 3.

Percent inhibition from 18 samples resulting in discordant status among the four laboratories

Sample no.	% Inhibition for laboratory no.:
	1	2	3	4
1	16	11	21	6
2	25	19	17	27
3	22	7	17	12
4	24	23	19	25
5	13	14	21	1
6	25	8	21	26
7	13	0	20	31
8	19	14	27	27
9	26	19	34	31
10	32	4	26	19
11	40	5	31	20
12	9	−15	24	11
13	50	19	44	45
14	30	−4	27	18
15	34	23	33	20
16	11	10	21	16
17	48	1	5	10
18	25	14	18	24

A different status (positive or negative) was recorded for 18 of the 100 samples after the single analysis performed in each of the four laboratories (Table 3). All but six of these sample results were near the cutoff inhibition, either a positive result not exceeding 30% inhibition or a negative result not less than 17% inhibition. Five of them (numbers 7, 10, 11, 14, and 17) gave positive results in one or two laboratories with inhibitions above 30%, and a single sample (number 6) gave a negative result of 8% inhibition in a single laboratory compared to positive results in the other three laboratories. There were five samples for which two laboratories recorded a negative and two recorded a positive for a single sample; again, most of them were near the percent inhibition cutoff (Table 3).

DISCUSSION

A standardized diagnostic assay that can be used globally to detect B. bovis-infected cattle would be valuable. Other serological methods that have been developed and utilized in the past include an IIF assay (6) and a number of ELISAs (1, 3, 12, 13, 17). Although these assays allow the detection of infected cattle, they all have limitations to some degree in specificity, sensitivity, reliability, or, in the case of IIF, subjective interpretation and low throughput. Our previous report (8) described the development of a competitive ELISA using a species-specific epitope in the C terminus of RAP-1 (15). RAP-1 from B. bovis was chosen because a structurally similar RAP-1 from Babesia caballi had recently been used in the development of an assay for B. caballi-infected horses (11). The usefulness of this diagnostic format is evidenced by the recent recognition of cELISAs for both B. caballi and Babesia equi as the official USDA diagnostic tests and by the Office International des Epizooties as the international standard assay for these babesial species. In our previous report, we described the specificity and potential usefulness of this B. bovis-specific cELISA but did not include enough information to fully validate its use internationally. In order to comply with Office International des Epizooties validation standards, we have now defined the optimized format for the B. bovis cELISA; determined the specificity, sensitivity and predictive values; and demonstrated its reliability.

As previously reported the antigen is based on a species-specific, broadly conserved, and tandemly repeated B-cell epitope within the C terminus of RAP-1 of B. bovis (14). This antigen is well recognized by antibody from infected cattle within 2 weeks following infection. The response to this epitope persists in experimentally infected cattle for at least several months, and the antigen is recognized by antibodies from infected cattle in diverse geographic locations (8).

After determining the optimal concentration of antigen and MAb, drying the antigen on the plates, and slightly altering the washing procedure and conjugate/substrate, we determined that bovine sera could be used undiluted to minimize false negatives without increasing false positives or background. The current protocol has now been used to test several hundred known-negative samples, with 141 randomly selected for this report, along with the 135 known positives. At a cutoff of 21% inhibition, the assay has a specificity of 100% and a sensitivity of 91.1%. This translates into a positive predictive value of 100% under any conditions. However, at 21% inhibition, the negative predictive value decreases as the prevalence increases and drops off dramatically as the prevalence increases from 80% to 90%. To our knowledge, 90% prevalence has not been reported for any region surveyed, but enzootic stability, where the estimated prevalence is near 70%, does occur in many regions. At this prevalence, the predictive value of a negative test result would be 83%.

Based on this and our previous study, the cELISA has the attributes (particularly the use of a dried-antigen plate format) necessary for worldwide diagnostic application. The overall accuracy is good, and the reliability is excellent based on concordance, kappa values, test of homogeneity, and ANOVA results from the four laboratories independently performing the assay. The few samples that were scored differently between laboratories had, for the most part, inhibition levels near the cutoff of 21%. This suggests that the cELISA, as is the case for other assays, should be repeated with samples near the cutoff using either a repeat of the same sample or another sample collected several days later. Our experience with the assay using a limited number of field samples, where PCR and the cELISA have been compared, suggests that resampling when results are negative but near the cutoff is also important, since there is a chance that very early infections would be PCR positive prior to detectable antibody being produced (data not shown). The accuracy and reliability of the cELISA need to be further determined by application to large numbers of sera collected from well-defined enzootic regions, and such studies are under way. However, the assay appears to meet regulatory stringency for use as an international standard and is formatted for ease of distribution and use under a variety of laboratory conditions.