Application of an Improved Method for the Recombinant K39 Enzyme-Linked Immunosorbent Assay To Detect Visceral Leishmaniasis Disease and Infection in Bangladesh

Abstract

Several serology-based immunoassays are used to diagnose visceral leishmaniasis (VL), a chronic protozoan parasitic disease caused by the Leishmania donovani complex. These tests are primarily designed to diagnose the most severe clinical form of VL, known as kala-azar. However, leishmanial infection is frequently asymptomatic and may manifest only as a positive serologic response or positive leishmanin skin test. We modified a previously described enzyme-linked immunosorbent assay (ELISA) that detects patient antibodies reactive with the recombinant Leishmania protein K39 (rK39) to confirm suspected kala-azar and to detect asymptomatic infection in a community study in Bangladesh. With the inclusion of a standard curve on each ELISA plate, the rK39 ELISA was more repeatable (kappa coefficient of agreement = 0.970) and more reliable compared to the original method (kappa = 0.587, P < 0.001). The cutoff point for a positive antibody response was chosen based on the 99th percentile of the ELISA distribution for the negative-control sera. However, we found that sera from all patients with active kala-azar yielded values more than twice the magnitude of this cutoff. Using receiver-operator characteristic curves, we determined a second cutoff value predictive of kala-azar. Using these criteria, the sensitivity and specificity of the modified ELISA for kala-azar were 97.0% and 98.9%, respectively, for sera from our study population. We hypothesize that individuals with antibody levels greater than the 99th percentile of the negative controls but less than the cutoff point for kala-azar have asymptomatic leishmanial infections.

Visceral leishmaniasis (VL) is caused by protozoa in the Leishmania donovani complex and is transmitted by the bite of infected female phlebotomine sand flies (8). Bangladesh, Brazil, India, and Sudan account for approximately 90% of the estimated global burden of leishmaniasis (16). VL may be present as an asymptomatic infection or as kala-azar, a chronically progressive disease characterized by weight loss, fever, hepatosplenomegaly, and, typically, death if left untreated (13). Of the estimated 59,000 deaths caused by leishmaniasis in 2001, 73% occurred in south Asia (15). South Asia is currently the focus of a planned elimination program, the strategy of which depends on early diagnosis and treatment of VL, combined with intensified vector control efforts.

A variety of serologic tests, including the immunofluorescence antibody test, the direct agglutination test, and enzyme-linked immunosorbent assays (ELISA), have been used to confirm suspected kala-azar and to detect subclinical infection in field settings (1, 12, 18). Evaluations of serologic tests using parasitological diagnosis in bone marrow or splenic aspirates as the “gold standard” generally demonstrate excellent sensitivity and good specificity for detection of kala-azar (3, 5). The tests are also positive in some proportion of other residents of VL-endemic communities (7, 14), a finding presumed to reflect the background level of subclinical infection. However, it is difficult to evaluate the use of serologic tests to detect subclinical infection because there is no independent definitive test for this condition. We utilized an ELISA to detect antibodies specific for the recombinant Leishmania protein k39 (rK39) for an epidemiologic investigation in a field setting in Bangladesh (2). We had two major objectives in using the ELISA: (i) in combination with clinical evaluation, to identify past and current kala-azar patients and (ii) to ascertain asymptomatic leishmanial infection, in order to better understand the transmission dynamics of the infection in the community. This article reports a modification of the previously published method to address repeatability problems encountered during the first year of fieldwork and the evaluation of the assay to determine the optimal cutoff for confirmation of kala-azar and for detection of subclinical infection.

MATERIALS AND METHODS

Patients and blood collection.

The serological work was performed as part of an epidemiologic study (2) conducted from January 2002 to April 2004 in a village in Fulbaria Thana in Mymensingh district, an area with a high reported incidence of VL. Surveys in 2002, 2003, and 2004, including serologic testing on capillary blood specimens, were used to screen for kala-azar and subclinical Leishmania infection. The study physician evaluated all participants who reported symptoms and those with high ELISA readings. We defined a case of kala-azar as an illness with ≥2 weeks of fever that included a history of one or more of the following symptoms: weight loss, abdominal fullness, abdominal pain, and skin darkening and that resolved after 20 days of intramuscular injections with sodium antimony gluconate (Glaxo Wellcome-Bangladesh). All adult participants provided written informed consent. The parent or guardian provided consent for children, and children 7 years or older also provided assent. The Research and Ethical Review Committees of the International Centre for Diarrhoeal Disease Research, Bangladesh (ICDDR,B), Dhaka, Bangladesh, and the Institutional Review Board of the Centers for Disease Control and Prevention approved the procedures of this project.

Specimens were categorized according to the status of the participant at the time the specimen was drawn, although, in some cases, this categorization depended on data collected in subsequent years. For example, a specimen from a participant with no symptoms during the 2002 survey was retrospectively categorized as “preclinical” if the participant developed kala-azar within the following 12-month period. For analyses that focused on the 2002 serum specimens, kala-azar patients whose treatment ended prior to 1999 were categorized as “past kala-azar,” while “recent kala-azar” patients were those whose treatment ended between 1999 and 2002. Individuals who did not have a history of kala-azar and did not develop signs consistent with kala-azar throughout the 3-year study were designated as “no kala-azar.”

Finger stick blood specimens (200 μl) were collected in capillary tubes without anticoagulant (Microvette CB 300; Sarstedt, Newton, NC). Samples were stored on ice and transported to the ICDDR,B laboratory in Dhaka. Following separation, sera were placed in new collection tubes and kept frozen at −20°C until testing.

Initial methods for rK39 ELISA.

We based our initial ELISA methods on a published protocol (9). Briefly, ELISA plates (Thermo Labsystems, Inc., Beverly, MA) were coated with 25 ng/well of rK39, provided as a gift from the Corixa Corporation (Seattle, WA), in coating buffer (50 mM carbonate-bicarbonate buffer, pH 9.6). Following overnight incubation at 4°C and subsequent washes with phosphate-buffered saline (PBS) containing 0.05% Tween 20, excess protein binding sites were blocked at room temperature for 3 h with 250 μl/well of PBS containing 1% bovine serum albumin. After five washes, 50 μl of 1:100 patient sera in serum diluent (PBS containing 0.1% bovine serum albumin and 0.05%Tween 20) was added to duplicate wells. The plate was placed on an orbital shaker at room temperature for 45 min. After five washes, the plate was incubated with 50 μl/well of human antibody-reactive conjugate for 45 min. In 2002, horseradish peroxidase (HRP)-protein A (Zymed Laboratories, Inc., South San Francisco, CA) was used as the antibody detection reagent as in the published method. The concentration used was based on a conjugate titration performed with each lot. To increase binding specificity in ELISAs performed in 2003 and 2004, we replaced the HRP-protein A with peroxidase-conjugated goat anti-human immunoglobulin G (IgG), IgA, and IgM (IgGAM; Zymed Laboratories, Inc., South San Francisco, CA) at a 1:4,000 dilution. Following six washes, 100 μl of 3,3′,5,5′tetramethylbenzidine substrate solution (Kirkegaard & Perry, Gaithersburg, MD) was added to each well. To stop the enzymatic reaction, 100 μl/well of 1 N H₂SO₄ was added. Optical density (OD) at 450 nm was measured using a microplate reader (Molecular Devices, Sunnyvale, CA). Pooled sera from nine Bangladeshis living in nonendemic areas and two North Americans were run in duplicate on each plate to establish that plate's negative cutoff value.

Modification of ELISA method to include standard curve.

To address repeatability problems encountered with the initial methods, the ELISA protocol was modified to include a standard curve on each plate. The pool was prepared by combining 20 μl of sera from 23 different individuals who were strongly positive in the original assay protocol. The pool was prepared at a 1:100 dilution in serum diluent with subsequent 1:2 serial dilutions. Fifty microliters of each dilution was tested in duplicate. The highest concentration in the standard curve was assigned a value of 1,000 concentration units (CU), with each subsequent 1:2 dilution expressed in CU (500 CU, 250 CU, etc.). A four-parameter standard curve was constructed using Softmax Pro 4.0 software. The OD for each test serum specimen was compared to the standard curve and reported in CU values. The negative antibody cutoff value was established from the CU of 38 individuals living in an urban, non-VL-endemic area of Bangladesh (the city of Dhaka). The mean age was 20 years old, with 18 females and 17 males.

Discordance rates and repeatability by ELISA method.

To compare the test repeatability of the two ELISA methods, 36 specimens from participants representing different clinical categories and with a range in rK39 reactivity were processed in duplicate on two ELISA plates (plates A and B) on each of 5 days. Thus, a total of 10 assays per specimen were run, and the results were analyzed by both the original (OD) and the modified (CU) methods. For each method, the result was categorized into one of three categories based on the respective established cutoff points.

Statistical analyses.

Data analyses were performed using Microsoft Excel 2000, Epi Info version 6, and SAS System for Windows, version 9. To derive the cutoff point for kala-azar, IgGAM serology data from years 2002 to 2004 were compiled for those with no history of disease (n = 4,708) and for those who had kala-azar at the time of specimen collection (n = 33). The probability of having kala-azar at the time of specimen collection was modeled using logistic regression. The log (CU + 1) was the only predictor included in this model. The generated predicted probabilities were used to construct a receiver-operator characteristic (ROC) curve that plots the true-positive rate against the false-positive rate for different potential cutoff points. Data from the repeatability experiment were used to estimate the kappa value for the original and the modified ELISA method. The kappa coefficient of agreement quantifies reproducibility, correcting for agreement expected by chance. A kappa value of 1 represents perfect repeatability.

RESULTS

ELISA results based on published methods.

In 2002, 1,707 serum specimens were tested according to the published rK39 ELISA method (9). In the course of the testing, we observed day-to-day variation in the mean (Fig. 1A) and standard deviation (SD; Fig. 1B) of the negative-control wells, although the same person performed the assays and the same control specimens were used. The mean OD of the 11 negative controls ranged from 0.059 to 0.880 (median, 0.196; mean, 0.230), and the standard deviation ranged from 0.002 to 0.042 (median, 0.013; mean, 0.015). Consequently, the initial OD cutoff point (mean of the negative controls + 3 standard deviations, subsequently referred to as 3-SD cutoff) ranged from 0.066 to 1.007. Because of this variability, we questioned the reliability of the assay to accurately assess which patients had anti-rK39 antibody levels indicative of active kala-azar. In particular, on plates with low mean negative-control OD values and a small SD, patients with no history of kala-azar were commonly classified as positive. Therefore, we added a second cutoff point (mean of the negative controls + 10 SDs, subsequently referred to as 10-SD cutoff) to increase the specificity for confirmation of kala-azar. However, the OD value for the 10-SD cutoff also varied widely, from 0.083 to 1.303.

FIG. 1.

Based on 47 ELISA plates tested between 27 January and 16 March 2002 for anti-K39 antibodies, the OD mean reading at 450 nm for 11 negative controls (9 from a nonendemic area in Bangladesh and 2 nonexposed North Americans) varied from 0.059 to 0.880 (median, 0.196; mean, 0.230) (A). The standard deviation was also inconsistent (range, 0.0024 to 0.0423; median, 0.013; mean, 0.015) (B). Consequently, these deviations resulted in variable cutoff points.

Results by the modified ELISA methods.

To address the problem of plate-to-plate variation, we added a standard curve on each plate. We retested 1,634 serum specimens collected during the 2002 survey by this method. This method was also employed in Bangladesh during the 2003 and 2004 surveys. The cutoff point to differentiate seronegative from seropositive individuals was established by calculating the 99th percentile of the mean CU from 38 negative-control individuals living in the city of Dhaka. Sera with values below this level (20 CU) were considered negative. However, we noted that all sera from patients with active kala-azar by clinical evaluation yielded values more than twice this level. Thus, we sought to establish a second cutoff value to optimize the sensitivity and specificity for the confirmation of kala-azar and to distinguish these patients from individuals with probable asymptomatic leishmanial infection.

We determined the second CU cutoff by constructing ROC curves comparing ELISA readings for specimens from all 33 active kala-azar patients sampled from 2002 to 2004 and all 4,708 specimens collected from participants who never had kala-azar, either before or during the study period (Fig. 2). The ROC curve demonstrated a distinct inflection point at 61, indicating that this was the cutoff that optimized sensitivity (97.0%) and specificity (98.9%) for kala-azar diagnosis. Of 33 kala-azar patients, only one had an ELISA value below 61 CU; the reading for this patient was 57.0 CU, suggesting that a slightly lower cutoff could be used if high sensitivity for kala-azar is imperative. For example, the use of 50 CU as the “kala-azar” cutoff in our data yields 100% sensitivity and a specificity of 98.7% and is still slightly more than twice the “subclinical” cutoff value based on the 99th percentile of the negative-control sera.

FIG. 2.

For the ROC curve, 4,708 individuals with no previous history of kala-azar and 33 individuals with active kala-azar were modeled using logistic regression with the log (CU + 1) as the only predictor. Maximum sensitivity (97.0%) and specificity (98.9%) occur with a CU value of 61 CU.

Table 1 compares the distribution of ELISA results by the initial method and the modified standard curve-based method. All active kala-azar patients had serology results above the “kala-azar” cutoff by both methods. The major differences were seen for specimens from participants with no history of disease throughout the study period and for treated kala-azar patients. By the original ELISA method, 116/1,514 (7.7%) of the individuals with no history of kala-azar had antibody values between the two cutoff points (3 to 9.9 SD) and 83 (5.5%) had values above the 10-SD cutoff point. By the modified method, 197/1,472 (13.4%) had readings between 20 CU and 60.9 CU and 20 (1.4%) had levels ≥61 CU. rK39 ELISA readings from treated kala-azar patients were less likely to surpass the modified method's 61-CU cutoff point compared to the original method's 10-SD cutoff point.

TABLE 1.

Distribution of original and modified ELISA results by clinical category at the time of specimen collection

Clinical category	No. (%) of specimens
	Within indicated no. of SDs of mean ODa of negative controls by original ELISA method			With indicated CU by modified ELISA method
	<3	3-9.9	≥10	<20	20-60.9	≥61
No kala-azar	1,315 (87)	116 (8)	83 (5)	1,255 (85)	197 (13)	20 (1.4)
Subsequent kala-azarb	18 (64)	2 (7)	8 (29)	16 (59)	4 (15)	7 (26)
Current kala-azar	0 (0)	0 (0)	15 (100)	0 (0)	0 (0)	15 (100)
Treated kala-azar in previous 3 yr	13 (15)	2 (2)	73 (83)	18 (22)	18 (22)	47 (57)
Treated kala-azar >3 yr ago	17 (46)	9 (24)	11 (30)	26 (70)	9 (24)	2 (5)
Total	1,363 (81)	129 (8)	190 (11)	1,315 (80)	228 (14)	91 (6)

^aOptical density read at 450 nm.

^bKala-azar with onset of illness during the 12 months after specimen collection.

Discordance rates and repeatability by ELISA method.

Table 2 compares the results for 1,643 specimens analyzed by both the original and the modified ELISA methods. The overall discordance rate between the methods was 24.4% (401/1,643). However, the disagreement in ELISA results varied by clinical category: 0% for specimens from current kala-azar patients (n = 15), 23.4% for those with no history of disease (n = 1,467), 19.2% for preclinical individuals (n = 26), 31.7% for specimens from recently treated (≤3 years) kala-azar patients (n = 82), and 57.8% for specimens from past kala-azar patients (>3 years; n = 37).

TABLE 2.

Discordance of result classification for 1,643 specimens tested by the original and by modified ELISA methods

Classification by modified method	No. of specimens classified by original ELISA method as:
	Mean + 3 SD	Mean + 3-9.9 SD	Mean + >10 SD
<20 CU	1,144	116	61
20-60.9 CU	180	11	39
≥61 CU	3	2	87

Table 3 compares the repeatability values for the assay when serum from each patient was tested on two different plates (A and B) by both the original and the modified ELISA methods. Out of 180 test results for plate set A and plate set B, the modified method had fewer contradictory classifications (3/180 or 1.7%) compared to the original method (42/180 or 23.3%). The overall repeatability of each method produced a kappa value of 0.970 for the modified method and 0.587 for the original method, indicating that the repeatability of the modified method was much higher than that of the original method (P < 0.001).

TABLE 3.

Repeatability of the original ELISA method using mean of the negative pool (in OD) plus SD and the modified ELISA method

Plate B category	No. of plate A specimensa in indicated category by:
	Original method				Modified method
	Mean + 3 SD	Mean + 3-9.9 SD	Mean + >10 SD	Total	<20 CU	20-60.9 CU	≥61 CU	Total
Original method
Mean + 3 SD	35	4	11	50
Mean + 3-9.9 SD	8	29	8	45
Mean + >10 SD	5	6	74	85
Total	48	39	93	180
Modified method
<20 CU					114	2	0	116
20-60.9 CU					1	3	0	4
≥61 CU					0	0	60	60
Total					115	5	60	180

^aValues in boldface are numbers of patients with results that concur on both plates.

DISCUSSION

During the first year of study, we encountered several problems with interpretation of our rK39 ELISA results. The protocol on which our assay was based, as well as other serodiagnostic assays for VL, utilized cutoff points derived from the mean OD value of negative control sera plus 3 standard deviations (9, 11, 17) or a multiple of the “normal” endemic serum OD value (6). However, in our study, the OD values and the SDs for the negative controls varied among runs by almost 15- and 20-fold, respectively, resulting in a wide range of cutoff points. It is unclear why the variability problem was so marked in this particular setting. There was no apparent trend over time in the variability of the mean negative ODs or SDs. We attempted to address the problem both mathematically and by using secondary reagents that we thought would have greater specificity. However, these changes did not remedy the problem of run-to-run variability. We therefore revised the assay to incorporate a standard curve on each plate. When we compared the repeatability of the original and the modified assays, the modified method was strikingly more consistent than the original method. Among the different clinical groups, there were shifts in the results for all categories except current kala-azar cases. Using the modified ELISA, sera from persons with no history of disease were less likely to score in the highest serology category (i.e., the one consistent with having active kala-azar) than sera for which the original method was used. Previously treated kala-azar patients were more likely to be classified as seronegative in the modified method compared to the original ELISA. Thus, with the standard curve adaptation, the rK39 ELISA produced more repeatable results and greater specificity for detection of active kala-azar.

We should emphasize that the primary value of this analysis is to demonstrate how a cutoff value for active kala-azar using a standard curve may be established for future studies. The precise numerical values presented here as cutoff points for current disease or positive antibody in this setting should not be directly applied to other study populations, but the methods by which they were determined are applicable. First, a positive pool serum bank should be established. These sera should be defined as positive by some other independent test result (e.g., parasitological diagnosis or, as in this study, clinical diagnosis of kala-azar and response to antileishmanial therapy). The pool should be based on equal volumes of serum from active patients. For consistency, the total volume of the pool should be adequate for performing all assays for the entire study. A standard curve based on these positive pooled sera should be included with each ELISA plate to correct for any day-to-day variation.

When validating a diagnostic assay for sensitivity and specificity, the sample size of infected and uninfected patients needed to determine sensitivity and specificity should be calculated (4, 10). However, frequently the generated reference size may not be feasible. In this situation, one can test the reference serum samples as they become available with the understanding that the derived cutoff value may change and the diagnostic assay will become more accurate as the reference sample size increases. One limitation of our study population was the relatively small number of current kala-azar patients. However, with the accumulation and merging of data from each year of our study, as well as the demonstrated repeatability of the method, our confidence in this assay grew. Similarly, the limited number of negative-control serum samples posed another logistical dilemma. For the initial ELISA testing, our negative pool consisted of only 11 individuals, 2 of whom lived in North America. The negative pool was subsequently expanded to improve accuracy. We also decided to limit the final negative pool to those living in a nonendemic area of Bangladesh in an attempt to assemble a control group more genetically and environmentally similar to the study participants.

Jacobson describes advantages and disadvantages of methods used for establishing cutoff points for infected and uninfected individuals (10). The simplest method is to plot frequency distributions of infected and uninfected individuals to determine at which point the two distributions overlap. Alternatively, a ROC curve that plots the true-positive rate versus false-positive rate for different cutoff points may be constructed using serology results from individuals with no history of disease and patients with current disease (19). The inflection point on the ROC curve that maximizes the area under the plot indicates the cutoff yielding the best balance of sensitivity and specificity. Modified ROC curves, plotting the true-positive rate versus true-negative rate for each cutoff point, may also be employed to determine sensitivity and specificity of the diagnostic test. A third method commonly employed for establishing an ELISA cutoff point uses the mean value of negative controls plus a multiple of the standard deviation of the negative controls. The major disadvantage of this method is the assumption of a normal distribution among the uninfected individuals. By taking a percentile (e.g., the 99th percentile), one can eliminate the error associated with assumptions of normality. Based on our nonnormally distributed serology data from all 3 years, we believe that using either the 99th percentile of the negative-control CU values or the ROC analysis is more reliable than using frequency distributions or the mean of the negative controls plus a multiple of the standard deviation.

In this study, the significance of the cutoff point that distinguishes individuals with “negative” and “positive” antibody levels remains somewhat unresolved. By employing two different cutoff points, we placed serology results into one of three categories. A patient in our study population with an assay result of less than 20 CU, or less than the 99th percentile of our negative controls, was considered negative. If a study participant had clinical signs consistent with kala-azar and an rK39 reactivity level greater than or equal to 61 CU, we would presume that this person has kala-azar and we would recommend antileishmanial treatment. However, this decision is also based on the knowledge that kala-azar is frequent in Mymensingh, Bangladesh, and that other infectious agents that may cross-react with leishmanial antigens, such as trypanosomiasis, are not endemic.

Application of the modified method yielded an intermediate zone of 20 to 60.9 CU between the two ELISA cutoff points. Although we cannot confirm that all individuals with ELISA results in this category had “subclinical” or asymptomatic infection, there is a strong chance their test results reflected an immune response to Leishmania, especially considering the highly specific antigen being used in this assay. Unfortunately, this study was limited to specimen collection once per year. With more-frequent data collection and additional laboratory evaluation of immune responses, the significance of the intermediate level rK39 ELISA results could be better characterized. Additional investigations using this assay in Bangladesh as well as other regions where VL is endemic will assist the evaluation of how robust our methods are for identifying persons with asymptomatic infection. Further study of the “subclinical” group could be informative to identify immunologic, genetic, nutritional, or environmental factors that influence whether exposed individuals progress to severe disease or asymptomatically resolve infection.

In summary, for future studies that might adopt this ELISA standardization, several issues must be considered. First, the purpose of the diagnostic test must be clear. Second, the ELISA must be repeatable. For the purpose of establishing cutoff points, the quality and quantity of the negative and positive controls are crucial. Although cutoff point values derived in different studies will vary depending on the laboratory, location, and disease prevalence in the region, instituting a two-point cutoff system does appear to be useful to differentiate among persons who are not infected, those who have been exposed, and those with active leishmanial disease.