Abstract
The Leptospira immunoglobulin M enzyme-linked immunosorbent assay (IgM ELISA) has been recommended for the rapid diagnosis of leptospirosis in endemic areas. We conducted a retrospective case-control study of 218 patients (109 leptospirosis cases confirmed by Leptospira culture and/or microscopic agglutination test and 109 control patients with acute febrile illness) to evaluate the diagnostic accuracy of a commercial IgM ELISA (Panbio) in northeast Thailand. Paired serum samples taken on admission and at least 10 days after the onset of symptoms were tested. Using the cutoff value recommended by the manufacturer (11 Panbio units), sensitivity and specificity of IgM ELISA on paired sera were 90.8% and 55.1%. A receiver operating characteristic curve was used to determine the optimal cutoff value. This was 20 Panbio units, which gave a sensitivity and specificity of 76.1% and 82.6%, respectively, on paired sera. We conclude that using either cutoff value, the accuracy of IgM ELISA is limited in our setting.
Introduction
Leptospirosis is considered an emerging infectious disease in Thailand.1–3 Regular contact with the environment is a major risk factor, with a study in Thailand reporting that leptospirosis accounted for around one-third of undifferentiated fevers in agricultural workers.3 Such infection usually results from contact with water or soil contaminated with pathogenic Leptospira originating from the urine of maintenance (carrier) hosts such as rodents. The clinical features of leptospirosis are often nonspecific and severity ranges from a mild flu-like illness to severe sepsis and septic shock. Signs and symptoms of leptospirosis may resemble those of other diseases in the tropics including dengue, scrub typhus, and malaria, resulting in an inaccurate clinical diagnosis.4 Diagnostic confirmation is not attempted across much of the world, however, leading to sub-optimal patient management and a poor understanding of disease epidemiology.
In Thailand, the definitive diagnosis of leptospirosis is made by culture of Leptospira spp. from clinical samples such as blood or urine, or by the reference serological assay, the microscopic agglutination test (MAT). These tests require specialist expertise, are labor-intensive, and provide a retrospective diagnosis.5 Several rapid serological diagnostic tests have been developed as alternatives to MAT, of which a commercial immunoglobulin M enzyme-linked immunosorbent assay (IgM ELISA) is promising because it can be performed in a greater number of laboratories throughout the tropics, and is inexpensive compared with MAT. The IgM ELISA has been recommended by the World Health Organization (WHO) as a diagnostic test for the serodiagnosis of leptospirosis where healthcare resources are limited,4 although its reported accuracy is variable. A number of studies have reported that IgM ELISA has high sensitivity and specificity for the diagnosis of acute leptospirosis.6–8 However, recent studies evaluating this test in Laos, Vietnam, and Hawaii found that its diagnostic accuracy was very low (sensitivity ranged from 36.0% to 60.9% and specificity ranged from 65.1% to 98.0%).9–11 The reason for poor performance in the latter studies may be that in countries where leptospirosis is highly endemic, a variable proportion of the apparently healthy population may be seropositive. This would result in variability in the optimal cutoff value in different geographic regions. In this study, we tested the hypothesis that the optimal cutoff for this assay in Thailand is higher than that stated by the manufacturer, and determined the optimal cutoff value and diagnostic accuracy of the IgM ELISA in this setting.
Materials and Methods
The Standards for the Reporting of Diagnostic accuracy testing (STARD) were followed in this study.
Study patients
Patients with laboratory-confirmed leptospirosis (cases) or without leptospirosis (controls) were identified from a prospective cohort study of consecutive patients presenting to Udon Thani hospital, northeast Thailand with an acute febrile illness between 2001 and 2002.12 In brief, patients were recruited into the study during twice daily ward rounds. Inclusion criteria were patients who were ≥ 15 years of age with fever (> 37.8°C) of unknown cause who agreed to participate and to attend outpatient follow-up for a convalescent serum sample. Exclusion criteria were patients with a definable source of infection on admission. Blood samples for Leptospira culture and serological testing were taken on admission, and a second serum sample taken 10 or more days after the onset of symptoms. Leptospira culture and MAT were performed as described previously.5,12 Serum was stored at −80°C until analysis for IgM ELISA and MAT.
Cases of leptospirosis were defined by isolation of pathogenic Leptospira spp., and/or a 4-fold or greater rise or a single serum MAT titer of ≥ 1:400. Controls (ratio 1:1) were randomly selected from those patients who had convalescent sample and did not meet the diagnostic criteria for leptospirosis. Overall, 109 cases and 109 controls were recruited in our study. A database was constructed in which cases and controls were entered, randomized, and blinded to the investigators before performing IgM ELISA.
Leptospira IgM ELISA
Detection of IgM antibodies to Leptospira species was determined using a commercially available Leptospira IgM ELISA (Panbio Pty., Ltd., Queensland, Australia). The assay was performed according to the manufacturer’s instructions. Briefly, test sera, cutoff calibrator, and positive and negative control sera were diluted 1:100 in serum diluent, and 100 μL added to Leptospira antigen-coated microwells and incubated for 30 minutes at 37°C. After washing with phosphate-buffered saline containing 0.05% Tween 20, 100 μL of HRP conjugated anti-human IgM was added and incubated for another 30 minutes at 37°C. After further washing, 100 μL of tetramethylbenzidine substrate was added and incubated at room temperature for 10 minutes, after which the reaction was stopped with 100 μL of 1 M phosphoric acid. The absorbance of each well was read at a wavelength of 450 nm with a Bio-Tek ELX 808 plate reader (Bio-Tek Instruments, Winooski, VT). The results were expressed as Panbio units calculated by the ratio of sample absorbance to the mean cutoff absorbance multiplied by 10. The recommended cutoff for a positive result is a value of ≥ 11 Panbio units, and is interpreted by the manufacturer to indicate recent infection of leptospirosis.
Evaluation of IgM ELISA in healthy individuals
Serum samples were obtained from 218 healthy blood donors residing in northeast Thailand and tested using the IgM ELISA to evaluate background seropositivity. To determine the reference range for this population, IgM Panbio units were log transformed so that data was normally distributed, and the reference range then calculated using the geometric mean and range values (geometric mean ± 2 SD).
Statistical analysis
Statistical analyses were performed using STATA/SE version 11.2 (College Station, TX). Diagnostic sensitivity and specificity of the IgM ELISA was defined against the combined result for culture and MAT (a positive result of either or both being interpreted as diagnostic for leptospirosis), and expressed as a proportion with exact 95% confidence interval (CI). The Fisher’s exact test was used to compare categorical variables, and the Mann-Whitney test was used to compare continuous variables. The optimal cutoff value of IgM ELISA was determined using receiver operating characteristic (ROC) curves.13 Median time to positivity of IgM ELISA in leptospirosis cases was estimated by survival analysis. This analysis was censored for loss to follow-up.
Ethical statement
The study protocol was approved by the Ethics Committee of the Ministry of Public Health, Royal Government of Thailand. Each patient was recruited to the study after written informed consent was obtained.
Results
A case-control study consisting of 109 leptospirosis cases and 109 controls was used to evaluate the accuracy of the IgM ELISA. Of 109 leptospirosis cases, laboratory confirmation was made on the basis of being positive by culture and MAT in 19 patients (17%), culture positive/MAT negative in 15 patients (14%), and culture negative/MAT positive in 75 patients (69%). The proportion of study patients who were male was 82% and 57% for cases and controls, respectively (P < 0.001). Three cases (2%) and four controls (4%) died during hospital admission (P > 0.1). All patients in the control group had a convalescent serum sample taken a median of 17 days (interquartile range [IQR] 13–21 days, range 10–43 days) after the onset of symptoms. The discharge diagnoses of controls were as follows: scrub typhus (N = 51), bacterial septicemia (N = 8) (Escherichia coli [N = 2], Klebsiella pneumoniae [N = 2], Acinetobacter baumannii [N = 1], Corynebacterium jeikeium [N = 1], Enterococcus spp. [N = 1], and Streptococcus pneumoniae [N = 1]), dengue fever (N = 4), murine typhus (N = 4), melioidosis (N = 2), HIV-related infections (N = 2), other diagnoses (N = 6), and unknown diagnosis (N = 32).
The sensitivity and specificity of IgM ELISA were determined using culture and/or MAT positivity as the gold standard. The diagnostic accuracy of the ELISA based on testing paired (acute and convalescent) sera and using a positive result on admission and/or during follow-up was determined using the manufacturer’s cutoff of > 11 Panbio units. The sensitivity and specificity of paired serum samples were 90.8% (95% CI = 83.8–95.5) and 55.1% (95% CI = 45.2–64.6), respectively (Table 1). Recalculation of these using results from the admission sample only gave a sensitivity and specificity of 52.3% (95% CI = 42.5–62.1) and 66.4% (95% CI = 56.6–75.2), respectively.
Table 1.
Sensitivities and specificities of Panbio IgM ELISA for the diagnosis of leptospirosis*
| Cutoff value | Sample timing | Positive (+) or negative (−) result | Gold standard result† | % (95% CI) | ||
|---|---|---|---|---|---|---|
| + | − | Sensitivity | Specificity | |||
| ≥ 11 Panbio units | On admission‡ | + | 56 | 36 | 52.3 (42.5–62.1) | 66.4 (56.6–75.2) |
| − | 51 | 71 | ||||
| Paired sera | + | 99 | 49 | 90.8 (83.8–95.5) | 55.1 (45.2–64.6) | |
| − | 10 | 60 | ||||
| ≥ 20 Panbio units | On admission‡ | + | 33 | 10 | 30.8 (22.2–40.5) | 90.7 (83.5–95.4) |
| − | 74 | 97 | ||||
| Paired sera | + | 83 | 19 | 76.1 (67.0–83.8) | 82.6 (74.1–89.2) | |
| − | 26 | 90 | ||||
IgM ELISA = immunoglobulin M enzyme-linked immunosorbent assay; CI = confidence interval.
Positive gold standard result was isolation of pathogenic Leptospira spp., and/or a 4-fold or greater rise or a single serum MAT titer of ≥ 1:400.
Sera on admission of two leptospirosis cases and two controls were not available to test by the IgM ELISA.
A ROC curve analysis was performed using results based on paired sera, which demonstrated an area under the curve of 0.85 (95% CI = 0.79–0.89). From this, an optimal cutoff was estimated to be ≥ 20 Panbio units. This gave an accurate classification in 79.4% of patients compared with 72.5% if the low cutoff value was used (Figure 1). Using this higher cutoff value, the sensitivity and specificity of the ELISA on paired sera were 76.1% (95% CI = 67.0–83.8) and 82.6% (95% CI = 74.1–89.2), respectively, and those on admission were 30.8% (95% CI = 22.2–40.5) and 90.7% (95% CI = 83.5–95.4), respectively. Median duration between onset of symptoms and primary specimen collection were 7 days (IQR 4–10 days) and 10 days (IQR 7–16 days) for cases and controls, respectively. Using survival analysis, the median time from onset of symptoms to positivity of IgM ELISA in leptospirosis cases was estimated to be 7 and 14 days for the low cutoff value (≥ 11 Panbio units) and high cutoff value (> 20 Panbio units), respectively.
Figure 1.
Receiver-operator characteristic curve of the Leptospira immunoglobulin M enzyme-linked immunosorbent assay (IgM ELISA) versus gold standard test (a combination of culture and MAT) on paired sera.
The optimal cutoff value defined here was almost twice the cutoff value recommended by the manufacturer, suggesting high background seropositivity. This was evaluated by testing 218 serum samples from healthy individuals residing in northeast Thailand. The ELISA results for these samples demonstrated a geometric mean value of 5.7 Panbio units, with a normal range (geometric mean ± 2 SD) of 1.7 to 19.3 Panbio units. Of 218 healthy individuals, 28 (12.8%) had an IgM ELISA level equal to or more than 11 Panbio units, and 6 (2.8%) had an IgM ELISA level equal to or more than 20 Panbio units.
Discussion
Reassessment of diagnostic sensitivity and specificity of the IgM ELISA showed that using either the low or high cutoff value, the accuracy of this test is poor in Thailand. Using the cutoff value recommended by the manufacturer (11 Panbio units), the sensitivity on paired sera is high (90.8%) and may be used to rule in patients with suspected leptospirosis. However, the poor specificity of the test (55.1%) suggests that using this test in the clinical setting could lead to over diagnosis of leptospirosis and a high frequency of false positivity. Using the high cutoff value generated by ROC curve analysis (20 Panbio units), the overall accuracy of the test was improved. However, both sensitivity and specificity on paired sera were 76.1% and 82.6%, respectively, indicating that the utility of the test using the higher cutoff value is also limited. In addition, using the test on admission is neither sensitive nor specific enough for the diagnosis of leptospirosis.
Our finding that IgM ELISA had a poor diagnostic accuracy is consistent with previous reports from Laos, Vietnam, and Hawaii.9–11 Plausible explanations for poor sensitivity include short duration of follow-up and inadequacy of an IgM response in some leptospirosis patients. Taking a second convalescent sample or sampling after a longer duration of follow-up may be required. IgM may also fail to be elevated in patients presenting with a second or subsequent episode of infection, when the antibody response would be predicted to be largely IgG. Poor specificity of the IgM ELISA is also likely to be caused by a high background seropositivity rate in the general population in our setting. Positivity of IgM ELISA could also represent past exposure of leptospirosis. This is strongly supported by the high seropositivity rate in healthy individuals if the low cutoff value is chosen.
We conclude that the Leptospira IgM ELISA is not sufficiently accurate in Thailand. The recommendation by WHO to use such an assay for the diagnosis of leptospirosis if financial resources permit requires country-specific evaluation to determine its clinical use before implementation.
ACKNOWLEDGMENTS
We are grateful to the director and staff of the Medical Department of Udon Thani General Hospital (Udon Thani Province, Thailand). We thank Lee D. Smythe (WHO/FAO/OIE Collaborating Centre for Reference & Research on Leptospirosis, Western Pacific Region, Communicable Disease Unit, Queensland Health Scientific Services, Brisbane, Australia) who undertook the MAT tests. We thank Boongong Noochan, Amornwadee Sungkakam, Premjit Amornchai, and Sayan Langla (Mahidol-Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol University), for technical assistance. We also thank Associate Professor Pratap Singhasiwanon (Dean of the faculty of Tropical Medicine, Mahidol University) and Professor Punnee Pitisuttithum (Head of the Department of Clinical Tropical Medicine) for their support.
Disclaimer: The authors have no conflict of interest. The Panbio Leptospira IgM ELISA kits were provided without charge by Panbio Pty, Ltd., Queensland, Australia. The company has no role in the study design, data collection and analysis, interpretation of results, and writing or submission of this manuscript.
Footnotes
Financial support: The study was funded by the Wellcome Trust.
Authors’ addresses: Varunee Desakorn, Vipa Thanachartwet, Duangjai Sahassananda, and Wirongrong Chierakul, Department of Clinical Tropical Medicine, Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand, E-mails: tmvds@mahidol.ac.th, tmvtn@mahidol.ac.th, tmdsh@mahidol.ac.th, and tmwcr@mahidol.ac.th. Vanaporn Wuthiekanun and Nicholas P. Day, Mahidol-Oxford Tropical Medicine Research Unit (MORU), Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand, E-mails: lek@tropmedres.ac and nickd@tropmedres.ac. Apichat Apiwattanaporn, Medical Department, Udon Thani General Hospital, Udon Thani, Thailand, E-mail: a-apiwat@hotmail.com. Direk Limmathurotsakul, Department of Tropical Hygiene, Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand, E-mail: direk@tropmedres.ac. Sharon J. Peacock, Department of Microbiology and Immunology, Faculty of Tropical Medicine, Mahidol University, Bangkok, Thailand, E-mail: sharon@tropmedres.ac.
References
- 1.Levett PN. Leptospirosis. Clin Microbiol Rev. 2001;14:296–326. doi: 10.1128/CMR.14.2.296-326.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Suttinont C, Losuwanaluk K, Niwatayakul K, Hoontrakul S, Intaranongpai W, Silpasakorn S, Suwancharoen D, Panlar P, Saisongkorh W, Rolain JM, Raoult D, Suputtamongkol Y. Causes of acute, undifferentiated, febrile illness in rural Thailand: results of a prospective observational study. Ann Trop Med Parasitol. 2006;100:363–370. doi: 10.1179/136485906X112158. [DOI] [PubMed] [Google Scholar]
- 3.Tangkanakul W, Smits HL, Jatanasen S, Ashford DA. Leptospirosis: an emerging health problem in Thailand. Southeast Asian J Trop Med Public Health. 2005;36:281–288. [PubMed] [Google Scholar]
- 4.World Health Organization/International Leptospirosis Society Human leptospirosis: guidance for diagnosis, surveillance and control. 2003. http://whqlibdoc.who.int/hq/2003/WHO_CDS_CSR_EPH_2002.23.pdf Available at. Accessed September 1, 2010.
- 5.Wuthiekanun V, Chierakul W, Limmathurotsakul D, Smythe LD, Symonds ML, Dohnt MF, Slack AT, Limpaiboon R, Suputtamongkol Y, White NJ, Day NP, Peacock SJ. Optimization of culture of Leptospira from humans with leptospirosis. J Clin Microbiol. 2007;45:1363–1365. doi: 10.1128/JCM.02430-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ooteman MC, Vago AR, Koury MC. Evaluation of MAT, IgM ELISA and PCR methods for the diagnosis of human leptospirosis. J Microbiol Methods. 2006;65:247–257. doi: 10.1016/j.mimet.2005.07.015. [DOI] [PubMed] [Google Scholar]
- 7.Winslow WE, Merry DJ, Pirc ML, Devine PL. Evaluation of a commercial enzyme-linked immunosorbent assay for detection of immunoglobulin M antibody in diagnosis of human leptospiral infection. J Clin Microbiol. 1997;35:1938–1942. doi: 10.1128/jcm.35.8.1938-1942.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Zochowski WJ, Palmer MF, Coleman TJ. An evaluation of three commercial kits for use as screening methods for the detection of leptospiral antibodies in the UK. J Clin Pathol. 2001;54:25–30. doi: 10.1136/jcp.54.1.25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Blacksell SD, Smythe L, Phetsouvanh R, Dohnt M, Hartskeerl R, Symonds M, Slack A, Vongsouvath M, Davong V, Lattana O, Phongmany S, Keolouangkot V, White NJ, Day NP, Newton PN. Limited diagnostic capacities of two commercial assays for the detection of Leptospira immunoglobulin M antibodies in Laos. Clin Vaccine Immunol. 2006;13:1166–1169. doi: 10.1128/CVI.00219-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wagenaar JF, Falke TH, Nam NV, Binh TQ, Smits HL, Cobelens FG, de Vries PJ. Rapid serological assays for leptospirosis are of limited value in southern Vietnam. Ann Trop Med Parasitol. 2004;98:843–850. doi: 10.1179/000349804X3207. [DOI] [PubMed] [Google Scholar]
- 11.Effler PV, Bogard AK, Domen HY, Katz AR, Higa HY, Sasaki DM. Evaluation of eight rapid screening tests for acute leptospirosis in Hawaii. J Clin Microbiol. 2002;40:1464–1469. doi: 10.1128/JCM.40.4.1464-1469.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Thaipadungpanit J, Chierakul W, Wuthiekanun V, Limmathurotsakul D, Amornchai P, Boonslip S, Smythe LD, Limpaiboon R, Hoffmaster AR, Day NP, Peacock SJ. Diagnostic accuracy of real-time PCR assays targeting 16S rRNA and lipl32 genes for human leptospirosis in Thailand: a case-control study. PLoS ONE. 2011;6:e16236. doi: 10.1371/journal.pone.0016236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993;39:561–577. [PubMed] [Google Scholar]