Abstract
Samples from 160 prospectively recruited febrile patients with typhus-like illness in an area of Thailand (Chiang Rai, northern Thailand) where scrub typhus is endemic were used to evaluate the diagnostic capabilities of four rapid immunochromatographic tests (ICTs) for the detection of Orientia tsutsugamushi IgM and total antibodies during acute scrub typhus infection. Of the 160 cases, 54 (34%) had been confirmed to have scrub typhus using the reference scrub typhus infection criteria (STIC), i.e., positive cell culture isolation, an admission IgM antibody titer of ≥1:12,800, a 4-fold rising IgM antibody titer, and/or positivity for ≥2 out of 3 PCR gene targets). The ICTs gave the following sensitivities and specificities: the Panbio IgM ICT, 46% (95% confidence interval [CI], 33 to 60) and 95% (95% CI, 89 to 98), respectively; the Standard Diagnostics IgM ICT, 68% (95% CI, 60 to 75) and 73% (95% CI, 68 to 78), respectively; the AccessBio IgM ICT, 56% (95% CI, 48 to 63) and 90% (95% CI, 87 to 94), respectively; and the AccessBio total antibody ABt ICT, 61% (95% CI, 53 to 68) and 68% (95% CI, 63 to 73), respectively. An isothermal loop amplification (LAMP) PCR assay for scrub typhus demonstrated a sensitivity of 52% (95% CI, 38 to 66) and a specificity of 94% (95% CI, 88 to 98). This study has revealed the diagnostic limitations of antibody-based assays in an acute care setting. However, the combination of ICTs with LAMP usually increased sensitivity with a minimal reduction in specificity. The best combination, the Panbio IgM ICT and LAMP, resulted in a sensitivity of 67% (95% CI, 53 to 79) and a specificity of 91% (95% CI, 83 to 95). The combination of antibody-based assays with DNA- or antigen-based tests shows promise for improved diagnostic sensitivity.
INTRODUCTION
Scrub typhus, caused by Orientia tsutsugamushi, is an important acute febrile illness in the Asia-Pacific region and is endemic in Thailand and Laos (10, 12). The clinical discrimination of scrub typhus from other undifferentiated fevers, such as dengue, malaria, and leptospirosis, is often very difficult because the clinical symptoms are similar on acute presentation. The diagnosis of acute scrub typhus infection is important for patient management, to guide appropriate therapy, and, hence, to prevent complications. Because of the absence of rapid, sensitive, and affordable diagnostics in settings where scrub typhus is endemic, clinical suspicion usually guides empirical treatment. There is a clear and urgent need for cheap, accurate, and easy-to-use point-of-care scrub typhus diagnostics for application in low-resource, primary health care settings to guide clinical therapy.
The four commercial antibody-based immunochromatographic tests (ICTs) designed for early rapid diagnosis at the time of hospital admission of the febrile patient (acute care setting) were evaluated in this study. These included the Panbio IgM (PBm), the Standard Diagnostics IgM (SDm), the AccessBio IgM (ABm), and the AccessBio total antibody (ABt) ICTs, which were used to test samples from a prospective study of fever in the environment of Chiang Rai, northern Thailand, where scrub typhus is endemic.
MATERIALS AND METHODS
Samples.
Over one calendar year (August 2007-August 2008), the study recruited 161 inpatients over 15 years old who had acute fever of less than 2 weeks' duration, no evidence of a primary focus of infection, and three negative malaria blood smears and who provided written informed consent. These cases were termed “typhus-like illness,” and samples consisted of admission serum and full blood samples collected in EDTA. Convalescent-phase specimens were tested for serology only and were available for 138/161 (86%) patients. Rapid test evaluation was performed in 160/161 patients, due to an insufficient sample volume from 1 patient with which to perform all four ICT assays. Ethical approval for this study was granted by the local ethics committee of Chiang Rai Hospital, the Thai Ministry of Public Health, and the Oxford Tropical Research Ethics Committee, United Kingdom.
Scrub typhus immunochromatographic tests.
Immunochromatographic tests for the detection of scrub typhus IgM antibodies from Panbio (PBm ICT; Panbio, Australia), Standard Diagnostics (SDm ICT; Standard Diagnostics, South Korea), and AccessBio CareStart (ABm ICT; AccessBio) and for the detection of scrub typhus total antibodies from AccessBio CareStart (ABt ICT; AccessBio) were performed according to the manufacturers' instructions. The PBm ICT was performed by trained operators under the direction of the study supervisor in a routine hospital laboratory (Prachanukhao Hospital, Chiang Rai, northern Thailand) with normal staff rotation. The SDm ICT, ABm ICT, and ABt ICT were retrospectively performed at the Mahidol-Oxford Tropical Medicine Research Unit (MORU), Bangkok, Thailand, by three experienced operators who generated individual results without conferring.
LAMP assay.
All isothermal loop amplification (LAMP) assay reactions were performed in triplicate using the methods previously described (8).
Definition of scrub typhus infection criteria (STIC).
The following reference diagnostic assays were performed to determine the final scrub typhus infection status of the patients included in the study: in vitro isolation of O. tsutsugamushi from buffy coat samples performed using a previously described method (6); PCR assays, including the nested 56-kDa PCR assay (4), 47-kDa-based real-time PCR assay (5), and GroEL-based real-time PCR assay (7); and indirect immunofluorescence assays (IFAs) based on O. tsutsugamushi pooled Karp, Kato, and Gilliam antigens for scrub typhus and Rickettsia typhi Wilmington strain antigens for murine typhus. IgM antibodies were detected using IFA slides produced by the Australian Rickettsial Reference Laboratory (Geelong, Australia). Briefly, patient sera were serially 2-fold diluted from 1:100 to 1:25,600, and the endpoint was determined to be the highest titer displaying specific fluorescence (11).
One or more of the following STIC had to be fulfilled for a confirmed diagnosis of scrub typhus: (i) positive cell culture isolation of O. tsutsugamushi, (ii) an admission IgM titer of ≥1:12,800, (iii) a 4-fold rising IgM titer in paired serum samples, and (iv) a positive result in at least two out of the three PCR assays described above (8a). While isolation and dynamic serology criteria are retrospective by nature, they provide the strongest evidence of active acute disease on admission. Similarly, the second-highest possible IgM titer measured upon admission (which corresponds to ≥1:12,800 in the Mahidol-Oxford Research unit laboratory) and positive results by two PCRs targeting two different genes provide high specificity and confidence in the diagnosis. Because of the risk of background moderately raised IgM titers in areas of endemicity, we opted to choose a no-compromise titer for this criterion to achieve maximal confidence in the diagnosis. The second-highest and highest measured titers (1:12,800 and 1:25,600) cannot lead to a 4-fold rise, so a cutoff titer of 1:12,800 was chosen as the single-titer cutoff for positivity on admission.
Non-scrub typhus reference testing. (i) Dengue.
The Panbio Dengue Early enzyme-linked immunosorbent assay (ELISA; catalog no. E-DEN01P, lot no. 08140; Panbio, Brisbane, Australia) was used to detect NS1 antigen in the acute-phase plasma specimens only, following the manufacturer's instructions. A positive specimen was defined as one with >11 Panbio units, <9 Panbio units was defined as negative, and 9 to 11 Panbio units was equivocal and the specimen was retested to confirm the result. The dengue IgM (catalog no. E-DEN01M, lot no. 08316; Panbio, Brisbane, Australia) antibody-capture ELISA was used to detect IgM antibodies in both the acute- and convalescent-phase specimens, following the manufacturer's instructions. Results were calculated as Panbio units as per the manufacturer's instructions. Patients were considered to be positive for acute dengue virus infection if the NS1 antigen was positive and/or there was at least a 2-fold increase in the number of Panbio units between the acute- and convalescent-phase specimens.
(ii) Murine typhus.
Murine typhus was diagnosed using the IFA with R. typhi Wilmington strain antigen IgM antibodies using slides produced by the Australian Rickettsial Reference Laboratory (Geelong, Australia). Patient sera were serially 2-fold diluted from 1:100 to 1:25,600, and the endpoint was determined to be the highest titer displaying specific fluorescence. Patients were considered positive if they had an admission IgM antibody titer of ≥1:12,800 and/or a 4-fold rise in IgM antibody titer.
Data analysis.
Diagnostic accuracy was calculated by comparison of ICT assay results for all three readers with the STIC by 2-by-2 cross-tabulation. Standard diagnostic accuracy indices of sensitivity, specificity, and positive and negative predictive values (the proportion of subjects with positive or negative test results who were correctly diagnosed, respectively) were calculated using Stata/SE (version 10.0) software (Stata Corp., College Station, TX). Kappa values testing for a significant difference between the readers (P ≤ 0.05) and McNemar's test were used to make pairwise comparisons between assays. Fisher's exact test was used to compare positivity rates using the ICT rapid tests and the IgM IFA titers.
RESULTS
Reference assay results.
In this cohort of patients with typhus-like illness, 54/160 (34%) cases fulfilled the robust STIC. In vitro isolation was successful in 5% (8/160), a ≥4-fold rise of IgM antibody titer in paired sera was seen in 19% (26/138), an admission IgM antibody titer of ≥1:12,800 was present in 12% (19/160), and ≥2 of the 3 PCR assays were positive in 16% (26/160) of all patients. The median number of fever days prior to admission for the acute-phase admission samples was 5 (interquartile ratio [IQR], 3 to 7). Four (2.5%; 4/160) patients were positive using the murine typhus reference test, and of these, 1 was also positive using the STIC. Using the dengue reference methods, 18 (11%; 18/160) patients were positive, and 2 of these were also positive using the STIC.
Absolute diagnostic accuracy.
Using the STIC as the reference comparator, the scrub typhus LAMP gave a sensitivity of 52% (95% CI, 38 to 66) and specificity of 94% (95% CI, 88 to 98). The ICTs gave the following sensitivities and specificities: the Panbio IgM ICT, 46% (95% CI, 33 to 60) and 95% (95% CI, 89 to 98), respectively; the SDm ICT, 68% (95% CI, 60 to 75) and 73% (95% CI, 68 to 78), respectively; the ABm ICT, 56% (95% CI, 48 to 63) and 90% (95% CI, 87 to 94), respectively; and the ABt ICT, 61% (95% CI, 53 to 68) and 68% (95% CI, 63 to 73), respectively. The kappa values for SDm ICT, ABm ICT, and ABt ICT were 0.90, 0.88, and 0.91 (all P < 0.0001), respectively (detailed in Table 1).
Table 1.
Absolute diagnostic accuracy for assays that are used for the admission diagnosis of scrub typhus infection
| Diagnostic assay(s) | Result compared to STICa (95% confidence interval) |
||||
|---|---|---|---|---|---|
| Sensitivity (%) | Specificity (%) | Positive predictive value (%) | Negative predictive value (%) | Kappa (3 readers) | |
| Individual assays | |||||
| Panbio IgM ICT | 46.3 (33–60) | 95.1 (89–98) | 83.3 (65–94) | 77.2 (69–84) | NDb |
| SD IgM ICTc | 67.9 (60–75) | 73.0 (68–78) | 56.1 (49–63) | 81.7 (77–86) | 0.90 (0.85–0.93) |
| AccessBio IgM ICTc | 55.6 (48–63) | 90.0 (87–94) | 75.0 (66–83) | 80.0 (76–84) | 0.88 (0.82–0.90) |
| AccessBio Total antibody ICTc | 60.5 (53–68) | 67.9 (63–73) | 49.0 (42–56) | 77.1 (72–82) | 0.91 (0.90–0.93) |
| LAMP | 52.9 (38–66) | 94.3 (88–98) | 82.6 (66–93) | 79.4 (71–86) | ND |
| Combined assays | |||||
| Panbio IgM ICT and LAMP | 66.7 (53–79) | 90.6 (83–95) | 78.3 (64–89) | 84.2 (76–90) | |
| SD IgM ICT and LAMPc | 77.2 (70–84) | 68.2 (63–73) | 55.3 (49–62) | 85.4 (81–90) | |
| AccessBio IgM ICT and LAMPc | 68.5 (61–76) | 84.9 (81–89) | 69.8 (62–77) | 84.1 (80–88) | |
| AccessBio total antibody ICT and LAMPc | 71.6 (64–78) | 63.2 (58–69) | 49.8 (43–56) | 81.4 (76–86) | |
Positive cell culture isolation, admission IgM titer of ≥1:12,800, 4-fold rising IgM titer, and/or ≥2 PCR gene target positives.
ND, not done, as assays were performed in singular.
Data represent combined results of three readers.
Diagnostic accuracy of a diagnostic approach combining ICT results with LAMP results.
Combining the ICT and LAMP results in an AND/OR Boolean manner increased the sensitivities with a minimal reduction of specificity. The PBm ICT and LAMP gave a sensitivity of 67% (95% CI, 53 to 79) and a specificity of 91% (95% CI, 83 to 95), the SDm ICT and LAMP gave a sensitivity of 77% (95% CI, 70 to 84) and a specificity of 68% (95% CI, 63 to 73), the ABm ICT combined with LAMP gave a sensitivity of 69% (95% CI, 61 to 76) and a specificity of 85% (95% CI, 81 to 89), and the ABt ICT combined with LAMP gave a sensitivity of 72% (95% CI, 64 to 78) and specificity of 63% (95% CI, 58 to 69) (summarized in Table 1).
Effect of sample timing.
The ICT results were evaluated for their diagnostic accuracies over time. This was based on the percent positivity change of the ICTs in the reference STIC-positive sample group over time, defined by the clinical parameter number of days of fever/illness of the patient prior to providing the admission sample used for the ICT and LAMP assays. All the ICTs demonstrated a gradual increase in positivity over sample collection time (Fig. 1), with PBm ICT, SDm ICT, and ABm ICT having similar rates of improvement in positivity, although some of the initial positivity in the SDm ICT (>50%) must be balanced with the high false-positivity rate of this test. With the exception of the PBm ICT (Fisher's exact test, p = 0.042), the rapid tests evaluated did not demonstrate a significant positive association (P < 0.05) between increasing sample collection time (days 1 to 10 postonset of fever/illness) and the increasing proportion of STIC-positive patients (SDm ICT, P = 0.199; ABm ICT, P = 0.0.73; ABt ICT, P = 0.0192; and LAMP, P = 0.76). Combining the results of the ICTs with those of the LAMP (Fig. 2) improved the positivity range of the more accurate assays such as the PBm and ABm ICTs; however, the sensitivity of detection remained low (25%, with the exception of that for the SDm ICT) in the early days of illness.
Fig 1.
Effect of acute-phase sample collection time (expressed as number of days of fever) on the percent positivity of STIC-positive results for different individual scrub typhus immunochromatographic and LAMP diagnostic assays.
Fig 2.
Effect of acute-phase sample collection time (expressed as number of days of fever) on the percent positivity of STIC-positive results for the different immunochromatographic tests combined with LAMP for scrub typhus diagnosis.
Agreement with IFA.
There was a significant association between the antibody-based PBm, SDm, ACBm, and ACBt ICT positivity rates (for STIC-positive samples) and increasing IFA IgM titers (Fisher's exact test, all P < 0.0001) (Fig. 3). The DNA-based LAMP assay also demonstrated a significant association (Fisher's exact test, P < 0.0001). A gradual increase in ICT positivity was demonstrated concomitantly with increasing IFA IgM titers. Large increases in ICT positivity were noted with IFA IgM titers greater than 1:800 for the SDm, ACBm, and ACBt ICTs and titers greater than 1:3,200 for the PBm ICT.
Fig 3.
Percent positivity of scrub typhus immunochromatographic and LAMP rapid test for detection of IgM positivity (STIC-positive samples only) compared with the gold standard IgM indirect immunofluorescent-antibody titers.
Dual positivity with murine typhus and dengue.
Of the 18 dengue patients, 2 (11%) were STIC positive, and of the 4 murine typhus patients, 1 (25%) was STIC positive. The LAMP and the PBm ICT demonstrated the same level of positivity for dengue as STIC and demonstrated no reactivity with murine typhus patients. The ABm ICT and ABt ICT were positive for 1 (25%) murine typhus patient (the same patient that was STIC positive) and gave positive results for 3 (17%) and 5 (28%) dengue patients, respectively. The least specific assay was the SDm ICT, which gave positive scrub typhus results for 2 (50%) murine typhus and 6 (33%) dengue patients.
DISCUSSION
This study evaluated four commercial rapid point-of-care ICT assays for the diagnosis of scrub typhus infection in the acute phase on admission of the patient. Using a prospectively collected cohort of patients in Chiang Rai, located in northern Thailand, a setting in which scrub typhus is endemic, no individual assay was sufficiently sensitive in detecting anti-O. tsutsugamushi IgM and total antibodies for the accurate diagnosis of acute scrub typhus infection. When results from a DNA-based LAMP assay were combined with the ICTs, an increase of sensitivity of up to 20% could be achieved with a marginal loss of specificity, demonstrating the improved diagnostic utility of this combination of tests.
Previous studies (2) using panels of stored sera from Southeast Asian patients with a variety of acute tropical fever diagnoses demonstrated better accuracy of the AccessBio scrub typhus IgM ICT for the detection of O. tsutsugamushi IgM antibodies, with a sensitivity and specificity of 96.8% and 93.3%, respectively. Expanding the detection to total antibodies increased the sensitivity to 97.6% but at the cost of specificity, which was lowered to 71.4%. In the present study, the poorer performances of the AccessBio IgM and total antibody ICTs, compared to those in the previous study, could be attributed to the shorter median duration of illness (this study, 5 days of fever [IQR, 3 to 7]; earlier study, 8 days), less stringent reference comparator criteria, and a higher proportion of positive versus negative samples (54% versus 34%). Results presented here for the PBm ICT are similar to those presented elsewhere for a prospectively collected patient cohort in Laos (1).
Combining the O. tsutsugamushi LAMP and the ICTs increased sensitivity by 10 to 20%, depending on the ICT, with decreases in specificity of about 5% for most ICTs (Fig. 2). The concept of using a combination of diagnostic assays detecting two different targets has been demonstrated elsewhere, most notably, in the use of NS1 antigen and IgM antibody detection for dengue diagnosis, where NS1 antigen is abundant in the first 7 days of infection and the IgM antibody is detectable onwards from 5 days postinfection. In this study, the combination of LAMP detecting O. tsutsugamushi DNA and the ICTs detecting IgM or total antibodies resulted in a definite diagnostic advantage. Although the detection dynamics of PBm and LAMP were similar with increasing time postinfection (Fig. 3), the DNA-based assay contributed with specific detection of antibody-negative cases in the early stages of disease, while the ICTs demonstrated a higher false-positivity rate. The Panbio IgM ICT and the AccessBio IgM ICT provided the best compromise of sensitivity and specificity when combined with the LAMP, but with higher specificity for the Panbio ICT-LAMP combination.
Two of the assays, the Standard Diagnostics IgM ICT and the AccessBio total antibody ICT, demonstrated poor specificity. The discrepancy between the ICTs and STIC may be caused by a number of factors, including dual positivity of antibodies against O. tsutsugamushi and related pathogens such as R. typhi (1) or detection of persisting antibodies following recovery from previous scrub typhus infections, resulting in false positivity. The total antibody format of the ABt assay gave a lower specificity (68%), most likely because of detection of O. tsutsugamushi IgG antibodies in STIC-negative fever patients from an area of endemicity where repetitive scrub typhus infections are common. The positivity cutoff titer of the “gold standard” O. tsutsugamushi IFA will also influence the accuracy of the index test. In this study, this factor did not influence the study outcomes, as the rigorous STIC bypass this issue. The Karp and Gilliam strains of O. tsutsugamushi dominate in Laos (9) and Thailand (3), and the Karp, Gilliam, and Kato antigens were used for the IFA in this study. The Standard Diagnostics ICT is based on the Boryong strain (South Korean), the AccessBio assays used recombinant 56-kDa Karp, Kato, and Gilliam strain antigens, and the Panbio ICT used only a single recombinant Karp antigen. Additional studies are required to assess the accuracy of the assays in other geographical areas.
This prospective study in a setting where scrub typhus is endemic has highlighted the limitations of the true diagnostic utility of antibody-based assays when used individually for diagnosis in the early admission setting. However, combinations of assays that detect molecular and possibly antigenic targets show promise for improved diagnostic sensitivity. Further developments in O. tsutsugamushi antigen detection for diagnostic purposes are encouraged, as are quality control studies to determine between- and within-lot variation in the performance of ICTs.
ACKNOWLEDGMENTS
D.H.P. holds a Wellcome Trust Clinical Research Training Fellowship (grant no. 078990/Z/06/Z). This study was funded by the Wellcome Trust of Great Britain as part of the Wellcome Trust-Mahidol University-Oxford Tropical Medicine Research Programme.
We thank Pruksa Nawtaisong, Kemajittra Jenjaroen, and Siriphan Boonsilp for excellent technical assistance. We are grateful to Stephen Graves and John Stenos of the Australian Rickettsial Reference Laboratory for providing reference spotted fever group strains and negative-control DNA.
The manufacturers provided the commercial assays used in this study free of charge and did not play a role in the design of the study, nor did they have access to the manuscript prior to submission for publication. The authors have no personal conflicts of interest.
Footnotes
Published ahead of print 4 January 2012
REFERENCES
- 1. Blacksell SD, et al. 2010. Accuracy of rapid IgM-based immunochromatographic and immunoblot assays for diagnosis of acute scrub typhus and murine typhus infections in Laos. Am. J. Trop. Med. Hyg. 83: 365–369 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Blacksell SD, et al. 2010. Accuracy of AccessBio immunoglobulin M and total antibody rapid immunochromatographic assays for the diagnosis of acute scrub typhus infection. Clin. Vaccine Immunol. 17: 263–266 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Blacksell SD, et al. 2008. Genetic typing of the 56-kDa type-specific antigen gene of contemporary Orientia tsutsugamushi isolates causing human scrub typhus at two sites in north-eastern and western Thailand. FEMS Immunol. Med. Microbiol. 52: 335–342 [DOI] [PubMed] [Google Scholar]
- 4. Horinouchi H, et al. 1996. Genotypic identification of Rickettsia tsutsugamushi by restriction fragment length polymorphism analysis of DNA amplified by the polymerase chain reaction. Am. J. Trop. Med. Hyg. 54: 647–651 [DOI] [PubMed] [Google Scholar]
- 5. Jiang J, et al. 2004. Development of a quantitative real-time polymerase chain reaction assay specific for Orientia tsutsugamushi. Am. J. Trop. Med. Hyg. 70: 351–356 [PubMed] [Google Scholar]
- 6. Luksameetanasan R, et al. 2007. Patient and sample-related factors that effect the success of in vitro isolation of Orientia tsutsugamushi. Southeast Asian J. Trop. Med. Public Health 38: 91–96 [PubMed] [Google Scholar]
- 7. Paris DH, Aukkanit N, Jenjaroen K, Blacksell SD, Day NP. 2009. A highly sensitive quantitative real-time PCR assay based on the groEL gene of contemporary Thai strains of Orientia tsutsugamushi. Clin. Microbiol. Infect. 15: 488–495 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Paris DH, Blacksell SD, Newton PN, Day NP. 2008. Simple, rapid and sensitive detection of Orientia tsutsugamushi by loop-isothermal DNA amplification. Trans. R. Soc. Trop. Med. Hyg. 102: 1239–1246 [DOI] [PubMed] [Google Scholar]
- 8a. Paris DH, et al. 2011. Diagnostic accuracy of a loop-mediated isothermal PCR assay for detection of Orientia tsutsugamushi during acute scrub typhus infection. PLoS Negl. Trop. Dis. 5(9): e1307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Parola P, et al. 2008. Genotyping of Orientia tsutsugamushi from humans with scrub typhus, Laos. Emerg. Infect. Dis. 14: 1483–1485 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Phongmany S, et al. 2006. Rickettsial infections and fever, Vientiane, Laos. Emerg. Infect. Dis. 12: 256–262 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Robinson DM, Brown G, Gan E, Huxsoll DL. 1976. Adaptation of a microimmunofluorescence test to the study of human Rickettsia tsutsugamushi antibody. Am. J. Trop. Med. Hyg. 25: 900–905 [DOI] [PubMed] [Google Scholar]
- 12. Watt G, Parola P. 2003. Scrub typhus and tropical rickettsioses. Curr. Opin. Infect. Dis. 16: 429–436 [DOI] [PubMed] [Google Scholar]