Improved Enzyme-Linked Immunosorbent Assay To Reveal Mycoplasma agassizii Exposure: a Valuable Tool in the Management of Environmentally Sensitive Tortoise Populations

Abstract

The precarious status of desert (Gopherus agassizii) and gopher (Gopherus polyphemus) tortoises has resulted in research and conservation efforts that include health assessments as a substantial component of management decision-making. Therefore, it is critical that available diagnostic tests for diseases impacting these species undergo rigorous standardization and validation. Since 1992, analysis of exposure of tortoises to Mycoplasma agassizii, an etiological agent of upper respiratory tract disease, has relied on the detection of specific M. agassizii antibody by enzyme-linked immunosorbent assay (ELISA). We report here substantive refinements in the diagnostic assay and discuss the implications of its use in wildlife conservation and management. The ELISA has been refined to include more stringent quality control measures and has been converted to a clinically more meaningful titer reporting system, consistent with other diagnostic serologic tests. The ELISA results for 5,954 desert and gopher tortoises were plotted, and a subset of these serum samples (n = 90) was used to determine end-point titers, to establish an optimum serum dilution for analyzing samples, and to construct a standard curve. The relationship between titer and A₄₀₅ was validated using 77 serum samples from known positive (n = 48) and negative (n = 29) control tortoises from prior transmission studies. The Youden index, J, and the optimal cut point, c, were estimated using ELISA results from the 77 control sera. Based on this evaluation, the refinement has substantially improved the performance of the assay (sensitivity of 0.98, specificity of 0.99, and J of 0.98), thus providing a clinically more reliable diagnostic test for this important infection of tortoises.

Mycoplasma agassizii is a causative agent of upper respiratory tract disease (URTD) in chelonians, and Koch's postulates have been fulfilled through experimental infection studies of both desert (Gopherus agassizii) and gopher (G. polyphemus) tortoises (11, 12). This disease is believed to be a contributing factor in the morbidity and mortality of both species (2, 5, 16, 18). Because of the precarious status of these chelonian species, conservation efforts now include health assessments as an important component of management decision-making. Therefore, it is critical that available diagnostic tests undergo rigorous standardization and validation.

Mycoplasmal URTD is one of very few diseases in chelonians for which comprehensive diagnostic tests exist. Current diagnostic methods include culture, PCR, and enzyme-linked immunosorbent assay (ELISA) serology (7, 10, 12, 27). Culture is not a convenient diagnostic tool given the fastidious growth requirements and very slow growth rates for this organism. Up to 6 weeks are required for the primary isolation of M. agassizii. Furthermore, culture and PCR do not always have high sensitivity. Obtaining an adequate nasal flush or swab sample is difficult and further complicated by the fact that culture and PCR are significantly less sensitive when animals are not exhibiting overt clinical signs (8, 22).

Because standard culture techniques are not of practical value for rapid diagnostics, a monoclonal antibody-based ELISA for the detection of specific antibody to M. agassizii was developed in 1992 (27). This assay was validated on the basis of experimental infection studies of both desert and gopher tortoises (11, 12). Seroconversion was demonstrated within 6 weeks following infection. In addition, samples from known healthy and naturally infected ill tortoises were tested, and the presence of specific antibody was correlated with the occurrence of histopathological lesions (11, 12, 17, 23).

The original assay conditions were described previously (27). Since the test was developed in 1992, an immense database of ELISA results from more than 20,000 serum samples has been generated. This database afforded a unique opportunity to refine the existing ELISA in order to provide a clinically more meaningful and reliable diagnostic test. Results of the original ELISA were reported as an enzyme immunoassay (EIA) ratio, defined as the absorbance of a sample measured at a wavelength of 405 nm (A₄₀₅) divided by the A₄₀₅ of the negative control, with an EIA ratio of ≥3 considered to be a positive result. The purposes of the current study were to determine the distribution of antibody levels in desert and gopher tortoises and to refine the assay by converting the reporting system from an EIA ratio to a titer-based system in order to make the assay more consistent with other serologic assays. Cutoff points were optimized, and the corresponding Youden index was determined as a measure of the assay's diagnostic effectiveness (15, 26). Furthermore, more stringent quality assurance measures were incorporated to ensure optimum performance of the assay at all times. An adaptation of the Youden plot, which provides information pertaining to within-batch imprecision and drift as well as long-term between-batch reproducibility, was used for internal quality control (19).

MATERIALS AND METHODS

Chelonian plasma samples (clinical and reference sera).

For the past decade, samples from researchers, veterinarians, zoological parks, and land management agencies have been submitted to our laboratory for mycoplasma testing. ELISA results from desert (n = 4,830) and gopher (n = 1,124) tortoises were used in this study to plot the distribution of antibody levels in tortoises. For this analysis, only samples obtained from free-ranging gopher and desert tortoises were used. Ninety gopher and desert tortoise plasma samples from our plasma banks were used for assay refinement. Although the true exposure status of these tortoises could not be confirmed with a “gold standard” assay, they were specifically chosen for use in developing the standard curve because they provided a cross section of the entire continuous range of A₄₀₅ values obtained with the ELISA. Another 77 reference plasma samples that originated from prior infection studies (i.e., the true exposure status is known) were used to verify cutoff points and estimate assay sensitivity, specificity, and the Youden index, J (15, 26). All plasma samples were stored in polypropylene cryovials at −20°C in a manual defrost freezer.

ELISA.

A whole-cell M. agassizii lysate antigen was prepared as previously described (27). The antigen was stored in aliquots at a concentration of 200 μg/ml at −80°C. Biotinylated mouse monoclonal antibody HL673 prepared against desert tortoise immunoglobulin Y light chain with documented cross-reactivity to gopher tortoise immunoglobulin Y light chain was used as the secondary antibody (27).

The M. agassizii ELISA was performed as previously described (27). M. agassizii antigen was used at 30 μg/ml, monoclonal antibody HL673 was used at 1 μg/ml, and alkaline phosphatase-conjugated streptavidin (Roche Diagnostics GmbH, Penzberg, Germany) was diluted 1:7,000. Dilutions of tortoise serum are described in detail below. P-nitrophenyl phosphate disodium was used at 1 mg/ml in 0.01 M sodium bicarbonate with 2 mM MgCl₂ (pH 9.6) (pNPP; Sigma Aldrich, Inc., St. Louis, MO). A₄₀₅ values were obtained by using an ELISA microplate reader (Biotek EL 403; Bio-Tek Instruments, Winooski, VT). The mean of wells coated with antigen and incubated with secondary antibody, conjugate, and substrate were used as the blank. The A₄₀₅ values were corrected for background by subtracting the blank, and A₄₀₅ values were converted to titers as described below.

EPT analysis and conversion to a titer-based reporting system.

Ninety sera with prior ELISA results spanning the entire range of A₄₀₅ values (from 0 to 2.4) were used to determine end-point titers (EPT), to establish an optimum serum dilution for analyzing samples, and to construct a standard curve relating A₄₀₅ values to EPT. The original ELISA used serum samples diluted 1:10 (27). For the refinement experiments, the ELISA was run on samples at serial twofold dilutions ranging from 1:8 to 1:1,024. The EPT was arbitrarily defined as the reciprocal of the last dilution with an A₄₀₅ value of ≥0.15. Samples with the same EPT were pooled using equal aliquots, and the pools were run at 1:50 and 1:100 dilutions to establish an appropriate single dilution for sample analysis. Based on the desired capability to distinguish between positive and negative samples, the 1:50 dilution was chosen for all subsequent assays. A standard curve was established to determine the relationship between EPT and the A₄₀₅ value. Sample classifications were then established using sera from known positive and negative control animals from prior transmission studies.

Because serum volumes from the 90 free-ranging tortoises were limiting, we replicated the standard reference curve using sera that originated from tortoises used in a transmission study. Specifically, a negative control (plasma from a known uninfected tortoise) and positive sera of different titers (plasma from known infected gopher tortoises at different time frames postinfection) were included as controls and to monitor assay performance. These reference standards were included in duplicate on every plate of every assay performed.

Establishment of quality control parameters.

An adaptation of the Youden plot was used for internal quality control (19). The intraplate, intra-assay, and interassay values for the reference standards were compared for 400 assays. In order for assays to be within acceptable limits, we required standard values to fall within 1 standard deviation of the mean A₄₀₅ of the first 75 replicates of the reference standards, which closely approximated the arbitrary 20% limit recommended previously by Jeffcoate (19). These limits were used to monitor within-batch imprecision and drift as well as long-term between-batch reproducibility.

Assessment of new ELISA protocol.

Data and banked sera from three infection studies (11, 12, 22) were used to estimate the Youden index and optimal cutoff point as described in detail previously (15, 26). Negative controls (n = 29) were healthy, mycoplasma-free tortoises defined by negative ELISA, culture, and PCR results. A subset of these tortoises (n = 12) was submitted for diagnostic necropsy to confirm the absence of histopathological lesions. Forty-eight positive control samples were available for use and included sera from tortoises given an intranasal inoculum that contained M. agassizii at various doses. All of these tortoises had one or more of the following characteristics supporting infection: development of clinical signs of URTD, histopathological lesions found at necropsy, a positive PCR evaluation of nasal flush samples collected from 4 to 12 weeks postinfection, or at least a twofold increase in M. agassizii antibody levels. The ELISA results for the negative control tortoises were not normally distributed, and therefore, all ELISA A₄₀₅ values were transformed by the addition of a constant value to eliminate negative or zero values and then calculation of the square root. Transformation resulted in normal distributions for both positive and negative control data; normal quantile plots and Shapiro Wilk goodness-of-fit tests were used to confirm normality (JMP IN 5.1, 2005). The sensitivity, the specificity, the optimal cut point, and the corresponding Youden index (J) were estimated for normally distributed data, with positive and negative controls having unequal variance, as described previously by Schisterman et al. (26). The positive predictive value (PPV) and negative predictive value (NPV) for the refined ELISA were determined using Bayes' rule for the estimated sensitivity and specificity (24, 31). The PPV and NPV for a range of seroprevalence levels, from 0 to 100, were plotted to demonstrate the impact that seroprevalence has on these parameters.

Statistical analysis.

Details of data transformation are described above. Equations for the Youden index and Bayes' rule (15, 24, 26, 31) were entered into the SAS programming language (SAS 9.1.3, 2002 to 2004) to estimate optimal cutoff points, calculate the Youden statistic, and calculate PPV and NPV.

RESULTS

Distribution of serologic results in desert and gopher tortoises.

The ELISA results for 5,954 desert and gopher tortoises were plotted to evaluate the distribution of seropositive results within our database (Fig. 1). Tortoises that had an EIA ratio of <2.0 (left side of Fig. 1) were negative (73%). Five percent of the test results fell within the suspect range, with EIA values between 2.0 and 3.0. EIA ratios of ≥3.0 were considered to be positive. Only 22% of all tortoises tested had positive results, suggesting that mycoplasmal URTD may be less pervasive in Gopherus spp. than previously thought. There were relatively few outlier points where the A₄₀₅ did not correlate with surrounding ratio values. Of the 5,954 samples tested, fewer than 50 samples (<0.8% of all samples tested) had values that were inconsistent with the previously established definitions of positive and negative tests, and thus, their true status may have been misclassified previously (27).

FIG. 1.

Distribution of M. agassizii ELISA results from desert (n = 4,830) and gopher (n = 1,124) tortoises. Each diamond represents the result from an individual serum sample, and the white line is a running average trend line. The EIA ratio is the optical density at 405 nm (A₄₀₅) of the sample/A₄₀₅ of the negative control. An EIA ratio of <2 was considered to be negative, EIA ratios of ≥2 and <3 were deemed suspect, and an EIA ratio of ≥3 was considered to be positive. The gray area indicates the suspect range; 5% of all test results fell within this range. Less than 25% of ELISA results were positive, and >70% were negative.

EPT.

In order to make the ELISA consistent with other serologic assays and to minimize interassay variation, the assay was converted from an EIA ratio to a titer-based system. Based on the EPT results for 90 samples, we classified samples as being negative, suspect, low positive (EPT of 64), medium positive (EPT of 128), and high positive (EPT of ≥256). The median EPT of sera categorized as being negative, suspect, and positive were 16, 32, and 256, respectively. The mean A₄₀₅ values and corresponding titer for all sera within the negative, suspect, and three positive ranges are shown in Fig. 2. The negative, suspect, and low-positive samples had shallow slopes relative to those of the medium- and high-positive samples. A 1:50 dilution was selected for all future assays because the variability of the A₄₀₅ values for the pooled samples was reduced, a low-positive serum could still be distinguished from negative serum, and the suspect range was decreased at that dilution.

FIG. 2.

EPT of 90 gopher and desert tortoise plasma samples grouped by test result. Values are expressed as the mean A₄₀₅ values ± standard deviations for each group. The EPT was defined as the reciprocal of the greatest plasma dilution to give an ELISA value of ≥0.15. The ELISA is presently being run at a dilution of 1:50 because of the observed reduction in variability of A₄₀₅ values and the ability to clearly discern between positive and negative results at that range.

Standard curve and quality assurance.

A standard curve was constructed with plasma pools composed of samples with the same EPT. Control samples obtained during a transmission study from known infected and noninfected tortoises were run in conjunction with the pools and plotted on the standard curve to test the established cutoff points. Experimentally infected animals and negative control animals were appropriately classified using the EPT interpretation of the ELISA (data not shown). Because sera were limited for the 90 samples, the standard curve was replicated using sera from transmission studies, for which large sample volumes are available in our serum bank. In order to determine the estimated EPT of a test sample, standard reference sera (EPT of 8, 32, 64, 128, and 256) were included on each plate of every assay. Based on this standard curve, a linear regression was performed in order to estimate the titer of an unknown serum sample (Fig. 3). Upper and lower limits for the controls were set at 1 standard deviation from the mean of the A₄₀₅ values for the first 75 replicates of the five standard reference sera. The control limits of the assay were further refined based on data from a 3-year period and including results from over 400 assays, as shown in Fig. 3.

FIG. 3.

Standard curve established for the M. agassizii ELISA. The circles represent mean A₄₀₅ values ± standard deviations for the five controls (titers of 8, 32, 64, 128, and 256). Upper and lower control limits for each of the five control samples were set at 1 standard deviation from the mean and based on over 400 assay replications. The standard deviation for the control samples with titers of 8 and 32 was so small that the bars were obscured by the symbol.

Assessment of cut points.

In order to obtain a statistically rigorous cut point for the ELISA, positive and negative control samples were evaluated using methods described previously by Schisterman et al. (26). The receiver operating characteristic (ROC) curve for these data is shown in Fig. 4. The calculated optimal cut point (c) to maximize sensitivity and specificity was 0.096, with an estimated sensitivity, specificity, and J statistic of 0.994, 0.998, and 0.996, respectively. ELISA assays in general are not precise enough to have a single cut point, and therefore, a range that maximizes both sensitivity and specificity is necessary. For the M. agassizii ELISA, a cutoff range that minimizes the false-positive rate was selected. Although a c value of 0.096 falls within the suspect range, a cut point range of 0.13 to 0.15 was selected based on the distribution of the standard curve and EPT of the 90 test samples. This range provides a J value of 0.985 to 0.983, a sensitivity of 0.985 to 0.983, and a specificity of 0.999 to 1.0. Four tortoises in the negative control group had suspect ELISA results using the previous method but were classified as being negative when retested using the new protocol. One tortoise in the infected group (i.e., was inoculated with M. agassizii) was classified as being suspect 12 weeks postinfection by the prior ELISA but was reclassified as being positive using the refined method.

FIG. 4.

ROC curve for the M. agassizii ELISA (n = 77). A positive cut point range of 0.13 to 0.15 was selected based on the distribution of the standard curve and EPT of the 90 test samples. This range provides a J value of 0.985 to 0.983, a sensitivity of 0.985 to 0.983, and a specificity of 0.999 to 1.0.

Bayes' rule was used to demonstrate how seroprevalence can influence PPV and NPV for the ELISA (Fig. 5) (24, 31). Figure 5 depicts a simulation of the assay performance over the entire range of seroprevalence values (i.e., 0 to 100%), with a fixed sensitivity of 98% and specificity of 100%. The PPV and NPV of the ELISA at a disease prevalence of 62%, the prevalence of the combined infection studies, were 99% and 97%, respectively. The example provided illustrates what those same parameters would be if the seroprevalence was 25% (PPV of 97% and NPV of 99%). Twenty-five percent was chosen as an example based on the approximate distribution of seropositive tests in the tortoise serum banks. For the tortoise M. agassizii ELISA, the PPV drops below 90% only at a seroprevalence of <9%, and the NPV drops below 90% at a >85% seroprevalence.

FIG. 5.

PPV and NPV as a function of seroprevalence in the population. Curves were generated using Bayes' formula, with test sensitivity and specificity of 98% and 100%, respectively. The solid line represents the PPV, and the dashed line represents the NPV. The two examples show the PPV and NPV of the ELISA at different seroprevalence levels (25% and 62%). For the M. agassizii ELISA, PPV and NPV were significantly affected only at extreme seroprevalence levels.

DISCUSSION

The M. agassizii ELISA has become a valuable diagnostic tool that has been widely utilized for a variety of studies pertaining to chelonians (1, 2, 9, 13, 17, 21, 23, 28-30). Importantly, some state and federal wildlife regulatory agencies have mandated serologic testing for M. agassizii exposure in order to limit the spread of this pathogen by tortoise relocation practices. Management decisions for tortoises impacted by development may be made based on the results of this assay. It is therefore critical that the test function optimally with adequate and defendable quality control measures.

The standard curve combined with the established control limitations provide a rigorous mechanism for closely monitoring inter- and intra-assay variance to ensure optimal quality assurance. The limits for the five controls used on each plate of every ELISA run were set at 1 standard deviation from the mean to minimize variability in the assay. Minor drifts in the absorbance values of the control samples can signal alterations in reagent quality and facilitate rapid recognition of problems within the assay.

In assessing the cut point, the overall goal was to establish a point that was not only statistically valid but also clinically meaningful. The Youden index provides a convenient mechanism to assess the effectiveness of a diagnostic assay and simultaneously to estimate the optimal cut point, c. This method has recently been suggested to be preferred over the ROC method (25) for the estimation of c. Values for the Youden index range from 0 to 1, with a J value of 1 providing perfect differentiating capacity and a J value of 0 having no ability to distinguish between cases and controls. The optimal cut point, c, is the cutoff point where sensitivity and specificity are maximized, and equal weights have been assigned to both parameters. However, depending on the goals of the user, it may be appropriate to shift cut points based on a desired sensitivity or specificity. The optimal c for the M. agassizii ELISA (c = 0.096; J = 0.996) falls in the range of a titer of 32 and corresponds to a suspect test result under the current assay conditions. When the assay was first developed, the cut points were specifically placed lower in order to minimize the chances of missing any infected tortoises (i.e., to reduce the number of false-negative results). Recent regulatory policies established by state and federal agencies have mandated serologic testing of tortoises impacted by development for M. agassizii exposure. These policies have resulted in management decisions based only on M. agassizii ELISA results, including euthanasia of tortoises testing positive without regard for the overall seroprevalence of the population and appropriate use of the assay. Given the potentially grave implications to tortoise populations with individuals that test positive by ELISA, we opted to take an alternative approach and maximize the specificity of the assay to reduce the probability of false-positive results. Furthermore, because of the inherent variability of every ELISA, having a single cut point is neither repeatable nor clinically relevant. Therefore, a range of A₄₀₅ values for the positive cut point has been established (0.13 to 0.15, which corresponds to the minimum value for a titer of 64). The actual cut point for each plate may vary slightly depending on the specific plate conditions as identified by the five controls run simultaneously and the resultant standard curve. This cut point range was based on the variability of the control samples run in over 400 replications. This range still provides very high J indices, sensitivity values, and specificity values of 0.982 to 0.985, 0.983 to 0.985, and 0.999 to 1.0, respectively. These values represent a significant improvement over previously reported values for M. agassizii ELISA sensitivity (0.94) and specificity (0.86) (8). Combined, these data provide strong evidence that the M. agassizii ELISA is highly effective at differentiating between exposed and nonexposed tortoises.

Because the M. agassizii ELISA is so widely utilized by individuals with various biological and infectious disease backgrounds, there has recently been significant confusion over the accuracy of the assay and appropriate application and interpretation of results. When making management decisions on the basis of the assay, it is critical to establish goals for the tortoise population of interest, to determine a necessary sample size to meet the goals for detection, and to consider the PPV and NPV of the test before implementing any policy. For some diagnostic assays, PPV and NPV are functions that may be greatly affected by the prevalence of the disease in the population of interest. The example provided in Fig. 5 shows that the PPV and NPV for the M. agassizii ELISA are significantly impacted by seroprevalence only at levels of >85% or <9%. Thus, a single positive result from an adequately sampled population with no or very low seroprevalence should be interpreted with caution, as it has a greater risk of being a false-positive result. Having an occasional false-negative result from a population with very high seroprevalence will likely not impact management decisions significantly. The goals established for the tortoise population can help managers decide whether the potential error should impact decision-making or not.

Disease has become an increasingly important issue for wildlife management considerations over the past two decades. More recently, the emergence of zoonotic diseases with wildlife reservoirs has brought this issue to the forefront. Disease surveillance is fundamental for disease prevention and control, and thus, there will be an increased need for the development of diagnostic assays not only for wildlife management but also for public health concerns. Diseases in free-ranging populations are often managed by isolation or culling, predominantly because treatment of individuals is impractical and vaccination programs can be instituted only in limited situations. It is critical that diagnostic tests be appropriately validated and have quality control mechanisms established. This consideration is of even greater importance when there are potentially severe consequences for individual animals or for the introduction of infectious agents into environmentally sensitive populations. Furthermore, the interpretation of test results and subsequent decision-making should be goal oriented and based on a sound understanding of assay limitations.