Evaluation of Serological Diagnostic Tests for Typhoid Fever in Papua New Guinea Using a Composite Reference Standard

Abstract

Typhoid fever remains a major global health problem. A major impediment to improving outcomes is the lack of appropriate diagnostic tools, which have not significantly improved in low-income settings for 100 years. We evaluated two commercially available rapid diagnostic tests (Tubex and TyphiDot), a prototype (TyphiRapid TR-02), and the commonly used single-serum Widal test in a previously reported high-burden area of Papua New Guinea. Samples were collected from 530 outpatients with axillary temperatures of ≥37.5°C, and analysis was conducted on all malaria-negative samples (n = 500). A composite reference standard of blood culture and PCR was used, by which 47 participants (9.4%) were considered typhoid fever positive. The sensitivity and specificity of the Tubex (51.1% and 88.3%, respectively) and TyphiDot (70.0% and 80.1%, respectively) tests were not high enough to warrant their ongoing use in this setting; however, the sensitivity and specificity for the TR-02 prototype were promising (89.4% and 85.0%, respectively). An axillary temperature of ≥38.5°C correlated with typhoid fever (P = 0.014). With an appropriate diagnostic test, conducting typhoid fever diagnosis only on patients with high-grade fever could dramatically decrease the costs associated with diagnosis while having no detrimental impact on the ability to accurately diagnose the illness.

INTRODUCTION

Typhoid fever, caused by the bacterium Salmonella enterica serovar Typhi (S. Typhi), remains an important health problem throughout the developing world. There are an estimated 21.6 million cases annually worldwide, resulting in approximately 200,000 deaths (8). While typhoid fever endemicity is geographically widespread, the burden is particularly high in the southern Asia region extending down to Papua New Guinea (PNG), with an incidence rate of 100 to 1,000 per 100,000 (8). The most recent epidemiological study on typhoid fever in PNG was conducted approximately 20 years ago in Goroka, Eastern Highlands Province. The annual incidence rate of 1,208 cases per 100,000 was, at that time, among the highest in the world (27). Up-to-date incidence data are unavailable; however, the disease remains a common diagnosis in febrile patients in the PNG highlands.

Despite the high burden of typhoid fever, the disease has been much neglected in recent years, in part due to the lack of suitable diagnostic tools (19). The mainstay of laboratory diagnosis for typhoid fever is blood culture, although the gold standard is bone marrow culture (28). In low-income countries where the majority of typhoid fever cases occur, bacterial culture is not routinely conducted. Where culture is conducted, bone marrow culture remains uncommon due to the invasiveness and technical difficulty of the procedure. As such, there is a dependence on alternative forms of diagnosis. The Widal test has been widely used in low-income countries for over a century. It detects patient serum antibodies to S. Typhi O and H antigens. However, it has numerous limitations (24, 27, 28), including poor specificity (other conditions can lead to an increased antibody titer, including malaria infection) and a cutoff titer that differs according to the endemicity of the disease. In 1987, the recommended cutoff titer of ≥40 for a single-serum O antigen test resulted in a test that was 98% sensitive and 98% specific in the PNG highlands (29), but within 5 years, the recommended cutoff titer had risen to ≥160 (27). The use of the Widal test in PNG has not been reevaluated since, but it remains the primary method of laboratory diagnosis in this country.

In recent years, various alternative diagnostic tests for typhoid fever have been developed and evaluated. Previous studies have shown that various prototypes and commercially available diagnostic tests may have advantages over the Widal test (13–15, 25). However, there remains no single test that has proven to be sufficiently sensitive, specific, and practical for use in low-income countries, and as such, these diagnostic kits are seldom used on a large scale.

Rapid diagnostic test (RDT) evaluations to date have used blood culture primarily as the reference standard for typhoid fever diagnosis. Culture provides definitive evidence of infection, but it fails to detect all cases, due to low numbers of the pathogen in the bloodstream and/or prior exposure to antibiotics. There is no evidence to date to suggest that automated systems overcome this shortcoming; indeed, in one recent study manual and automated blood cultures were equally sensitive for the detection of S. Typhi (13). The use of a composite reference standard for evaluation of diagnostic tests has been proposed when there is no single test that is adequate as the reference standard (1). The use of multiple highly specific tests may increase the ability to detect a true-positive sample. This approach has been endorsed by the TDR Diagnostics Evaluation Expert Panel (3).

In recent years, molecular biology-based assays have been evaluated in trials for the detection of S. Typhi in blood (4, 11, 18, 22). The application of such methods in routine diagnosis for typhoid fever is yet to be established; however, nucleic acid diagnostic assays can offer high sensitivity and excellent specificity. Given the previously reported high incidence of typhoid in the highlands of PNG, the ongoing transmission of the disease, and the need to determine the best diagnostic strategy in PNG, we evaluated two commercially available typhoid serodiagnostic test kits (TubexTF, manufactured by IDL Biotech AB, Sweden, and TyphiDot, manufactured by Reszon Diagnostics International Sdn. Bhd, Malaysia); one prototype kit, TR-02 (Reszon Diagnostics International Sdn. Bhd, Malaysia); and the currently used Widal test. The TubexTF test detects serum anti-Salmonella O9 (lipopolysaccharide) IgM and IgG antibodies in an enzyme-linked immunosorbent assay (ELISA)-based assay (16). The TyphiDot assay is a dot enzyme immunoassay which detects serum anti-Salmonella IgM and IgG antibodies specific for a 50-kDa outer membrane protein (OMP) antigen (6, 12). The TR-02 prototype is an immunochromatography assay which detects serum IgM (but not IgG) antibodies to the same Salmonella OMP antigen used in the TyphiDot assay. We used a composite reference standard of blood culture and real-time PCR to evaluate the diagnostic kits.

MATERIALS AND METHODS

Participant recruitment and specimen collection.

Patient recruitment took place at the Goroka General Hospital outpatient center and the Lopi Urban Clinic in Goroka, Eastern Highlands Province, Papua New Guinea. Febrile patients (axillary temperature, ≥37.5°C) who reported having fever for at least 2 days were invited to participate in the study. In addition, patients with an absence of fever at the time of consultation, or self-reported fever for less than 2 days prior to consultation, were included in the study if there was clinical suspicion of typhoid fever by the study staff or senior clinical staff at the study sites. All patients were informed of the study aims, and following informed consent, a brief questionnaire was administered. The questionnaire sought information regarding clinical symptoms and prior antimicrobial treatment. Up to 25 ml of blood was collected from adults, and 3 to 5 ml was collected from children ≤5 years old. A blood film was prepared for malaria microscopy. Blood, inoculated blood culture bottles (see below), and the blood smear were transported to the laboratory within 2 h of collection. Upon receipt of samples at the laboratory, serum separator tubes were centrifuged at 3,000 × g for 10 min and aliquots of serum were stored at −20°C for later serological testing. Aliquots of whole blood were also stored at −20°C for later DNA extraction.

Malaria diagnosis.

Thick and thin blood films were fixed, Giemsa stained, and read according to standard methods (5) by experienced microscopists. Malaria-positive samples were excluded from further analysis on the assumption that malaria was the primary cause of fever.

Blood culture.

Immediately following blood collection, 5 ml of blood was added to 45 ml of tryptic soy broth with sodium polyanethol sulfonate, as described by Gratten (10). This methodology has been successfully used in our laboratory to culture a variety of organisms (9). Blood cultures were incubated at 37°C, and after 24 h of incubation, approximately 100 μl of blood culture broth was plated onto blood agar, chocolate agar, MacConkey agar, and xylose-lysine desoxycholate agar. Plates were incubated at 37°C for 24 h and examined for growth. Blood agar and chocolate agar plates were incubated in a 5% carbon dioxide-enriched atmosphere and incubated for 24 to 48 h to detect other causes of bacteremia. Subculture of the blood culture broth onto solid medium was repeated after 3 days and 7 days of blood culture bottle incubation. Blood cultures were considered negative and discarded if there was no growth on solid medium inoculated on day 7. This is largely consistent with WHO guidelines for S. Typhi culture (30).

Serological tests.

Batches of sera were removed from the freezer, and all serological tests were conducted on sera within 24 h of thawing. The Widal test was conducted using S. Typhi O antigen (Remel Europe Ltd.). Using a 6-well plate, patient serum was diluted in S. Typhi O antigen to produce serum titers of 40, 80, 160, and 320. The serum and antigen were allowed 1 min to agglutinate before reading. This method is standard procedure in PNG. The TubexTF, TyphiDot, and prototype TR-02 diagnostic tests were conducted according to the manufacturer's instructions. Results for all serological tests were double read.

Molecular detection.

DNA extraction was conducted from 200 μl of whole blood using the Qiagen DNeasy blood and tissue extraction kit, according to the manufacturer's instructions for extraction from blood samples.

Real-time PCR analysis was conducted on all malaria-negative samples using a previously described hydrolysis probe assay for S. Typhi (18). This assay targets a 73-bp region of the S. Typhi H1-d flagellin gene using the following primers and probe: forward primer ST5 (5′-CAA CCT GGG CAA TAC CGT AAA TAA-3′), reverse primer ST6A (5′-TTC GGT TGC GTA GTC GGA AT-3′), and dually labeled probe ST7 (5′-HEX-TG TCT TCT GCC CGT AGC CGT ATC G-BHQ1-3′).

The real-time PCR was conducted in a 20-μl reaction mix containing 2 μl of the DNA extract, 400 nM (each) ST5 and ST6A primers, 400 nM TaqMan probe ST7, 1× QuantiTect Mastermix (Qiagen), and nuclease-free water. All reactions were conducted on a Bio-Rad CFX-96 real-time system with the following cycling parameters: 50°C for 2 min, followed by 40 cycles of 94°C for 1 min and 60°C for 1 min. Negative and positive controls were included in each run. A positive result was defined as a sample having a cycle threshold (C_T) between 16 and 40 using an autocalculated single-threshold baseline using the Bio-Rad CFX Manager software version 2.0.

The specificity of this real-time PCR assay has been previously described, and it was found to be specific for S. Typhi. It does not amplify closely related bacteria, including Salmonella enterica serovars Paratyphi, Typhimurium, Choleraesuis, and Enteritidis, or non-Salmonella species Escherichia coli, Klebsiella pneumoniae, Klebsiella aeruginosa, Proteus mirabilis, Proteus vulgaris, Pseudomonas aeruginosa, Enterobacter spp., Citrobacter freundii, or Alcaligenes faecalis (18). The limit of detection was determined by amplifying triplicate 10-fold serial dilutions of a whole-genome extraction of pure S. Typhi culture at known CFU. The detection limit was defined as the highest dilution with a positive result in each of the triplicate samples tested, resulting in a lower limit of 4.4 CFU. Amplification efficiency was 93%, and R² was 0.991.

Data analysis.

Data were entered into an Excel spreadsheet (Microsoft Corporation, Redmond, WA). Analysis was conducted in Excel and SPSS Statistics 20 (IBM Corporation, Armonk, NY). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated as described by the TDR Diagnostics Evaluation Expert Panel (3). Samples that were positive by either blood culture or PCR were considered composite reference standard positive: samples negative by both blood culture and PCR were composite reference standard negative.

Correlations were sought between the presence of typhoid fever (as determined by the composite reference standard) and the severity and duration of fever using chi-square analysis. Patients were stratified into three groups according to severity of fever: no fever, low-moderate fever, and high fever (axillary temperatures of <37.5°C, 37.5 to 38.4°C, and ≥38.5°C, respectively). Patients were also stratified according to self-reported duration of fever (<2 days, 2 to 6 days, and ≥7 days). Where correlations occurred, diagnostic tests were reevaluated using that parameter as part of the inclusion criteria.

Ethical approval.

This study was reviewed and approved by the appropriate review boards, the Papua New Guinea Institute of Medical Research Institutional Review Board (0818) and the Papua New Guinea Medical Research Advisory Committee (08/17), and informed consent was obtained from all the participants or their parents or guardians.

RESULTS

Participants.

A total of 530 patients with axillary temperatures of ≥37.5°C for at least 2 days and/or clinical suspicion of typhoid fever were recruited over a period of 18 months from March 2009 to September 2010. Of the 530 participants, 321 (60%) were enrolled at the Goroka General Hospital and 209 (40%) were enrolled at the Lopi Urban Clinic, and 269 (51%) were male and 261 (49%) were female. Participants were between 1 year of age and 60 years of age, with a median age of 38.5 years. Thirty patients (6%) were malaria positive, and thus, no further analysis was conducted, on the assumption that malaria was the sole cause of fever. Of the remaining 500 participants, 54 had no fever at the time of recruitment (and thus were enrolled in the study on clinical suspicion of typhoid fever), while 262 and 184 had axillary temperatures of 37.5 to 38.4°C and ≥38.5°C, respectively. Of the 446 patients with fever, 6 reported having had fever for <2 days, 241 reported having had fever for 2 to 6 days, and 199 reported having had fever for ≥7 days.

Blood culture.

Culture for S. Typhi was conducted on the 500 malaria-negative samples, with an isolation rate of 4% (n = 22). Only one sample was culture positive for another bacterial pathogen (Escherichia coli).

PCR.

Forty patients (8%) were diagnosed with typhoid fever by real-time PCR: 15 of the PCR-positive patients were blood culture positive, while 25 were blood culture negative. Seven blood culture-positive samples were PCR negative. Thus, by combination of PCR and blood culture 47 participants (9.4%) were diagnosed with typhoid fever, over twice the number of positive patients diagnosed by blood culture alone.

Serology.

Forty-three (8%) patients tested positive using the Widal test at a titer of 160, 77 (15%) tested positive using TubexTF, 121 (23%) tested positive using TyphiDot, and 110 (22%) tested positive using the TR-02 prototype test. The Widal test was positive for 19/22 blood culture-positive samples and 17/40 PCR-positive samples. Corresponding data for the other rapid detection tests were as follows: TubexTF, positive for 17/22 and 19/40 samples, respectively; TyphiDot, 21/22 and 24/40 samples, respectively; and TR-02, 22/22 and 35/40 samples, respectively.

Evaluation of diagnostic tests.

All of the serological tests were evaluated using the composite reference standard and also using blood culture alone as the reference standard (Table 1). This was to better enable comparison of our findings with previous evaluations of Tubex and TyphiDot.

Table 1.

Sensitivity, specificity, PPV, and NPV of typhoid fever diagnostic tests, using blood culture and a composite reference standard (blood culture and real-time PCR) as comparators

Test	% (95% confidence interval)
	Sensitivity	Specificity	PPV	NPV
Blood culture
Tubex	77.3 (59.8–94.8)	87.4 (84.5–90.4)	0.221 (0.128–0.313)	0.988 (0.978–0.998)
TyphiDot	95.5 (86.8–104)	79.1 (75.4–82.7)	0.174 (0.106–0.241)	0.997 (0.992–1.00)
TR-02	100	81.6 (85.1–78.1)	0.200 (0.125–0.275)	1
Widal (titer, 160)	86.4 (72.0–100.7)	95.0 (93.0–96.9)	0.442 (0.293–0.590)	0.993 (0.986–1.001)
Blood culture plus PCR (composite reference standard)
Tubex	51.1 (36.8–65.4)	88.3 (85.3–91.2)	0.312 (0.208–0.415)	0.946 (0.924–0.967)
TyphiDot	70.0 (52.4–79.5)	80.1 (76.5–83.8)	0.256 (0.178–0.334)	0.958 (0.938–0.978)
TR-02	89.4 (80.5–98.2)	85.0 (81.7–88.3)	0.382 (0.291–0.473)	0.987 (0.976–0.998)
Widal (titer, 160)	51.1 (36.8–65.4)	95.8 (94.0–97.7)	0.558 (0.410–0.707)	0.950 (0.930–0.970)

A correlation was observed between S. Typhi infection and severity of fever when stratified (χ² = 7.309; degrees of freedom [df] = 2; P = 0.026). Further analysis revealed a correlation between the absence of S. Typhi infection and low-moderate fever (χ² = 7.009; df = 1; P = 0.008) and the presence of S. Typhi infection and high fever (χ² = 5.993; df = 1; P = 0.014). No correlations were observed between the duration of fever and S. Typhi infection. On this basis, serological tests were reevaluated, excluding patients with no and low-moderate fever (<38.5°C). Thus, data from 184 patients were included, of which 25 (13.6%) were composite reference standard positive. Sensitivity, specificity, PPV, and NPV were similar to those derived from the evaluations using the entire data set (Table 2).

Table 2.

Sensitivity, specificity, PPV, and NPV of typhoid fever diagnostic tests when used on patients with axillary temperatures of ≥38.5°C (using a composite reference standard of blood culture and real-time PCR as the comparator)

Test	% (95% confidence interval)
	Sensitivity	Specificity	PPV	NPV
Tubex	52.0 (32.4–71.6)	90.6 (86.0–95.1)	0.464 (0.280–0.649)	0.923 (0.881–0.965)
TyphiDot	64.0 (45.2–82.8)	81.8 (75.8–87.8)	0.356 (0.226–0.495)	0.935 (0.894–0.976)
TR-02	92.0 (81.4–100.3)	85.5 (80.1–91.0)	0.500 (0.356–0.644)	0.986 (0.966–1.01)
Widal (titer, 160)	40.0 (20.8–59.2)	97.5 (95.1–99.9)	0.714 (0.478–0.951)	0.912 (0.869–0.954)

DISCUSSION

Our study demonstrates the value of using the molecular detection of S. Typhi in blood when evaluating rapid diagnostic tests. Despite bone marrow culture being the gold standard of typhoid diagnosis, blood culture is universally used as the reference standard when evaluating typhoid diagnostic tests. We have demonstrated a 2.1-fold increase in typhoid fever diagnosis when blood culture and real-time PCR detection were used, relative to blood culture alone.

Previous studies have demonstrated the potential of molecular detection of S. Typhi from blood; however, debate remains regarding the true utility of PCR as a diagnostic tool for typhoid fever. The validity of the debate is evident in the current study, where seven of 22 (32%) blood culture-positive samples were PCR negative. Nga and colleagues also demonstrated that blood samples that were culture positive could be PCR negative, where 58% of culture-positive samples were negative using detection by real-time PCR. In contrast to our findings, Nga et al. did not detect S. Typhi (or S. Paratyphi A) in any blood samples with symptoms of enteric (typhoid) fever that were culture negative (23). However, other researchers have detected S. Typhi in culture-negative blood by using regular, nested, and real-time PCR platforms (4, 11, 18, 22). Our data suggest that PCR is useful as a complementary form of diagnosis in the context of diagnostic evaluation and provides the advantage of being able to detect nonviable cells (e.g., in patients who have recently commenced antibiotics). However, for reasons previously outlined PCR is not currently applicable for routine diagnostic use (2).

Typhoid fever diagnostic evaluations have consistently demonstrated less than optimal sensitivity and specificity of rapid tests (26). When using blood culture as the reference standard, we found the sensitivity and specificity of the Tubex and TyphiDot tests to be comparable to those in previous reports (13, 14, 25). However, these assays performed less satisfactorily when evaluated against the composite reference standard. In contrast, the performance of the prototype diagnostic test (TR-02) was promising when evaluated against the composite reference standard, having both a sensitivity and a specificity of ≥85%.

The correlation between typhoid fever and high-grade fever (axillary temperature of ≥38.5°C) may have important implications for typhoid fever diagnosis. When study participants with low-grade fever were excluded from analysis, the sensitivity and specificity of the assay improved, albeit marginally. In our study, 22 patients with confirmed typhoid fever would be excluded from initial testing if such a testing algorithm were adopted. However, this is from a total of 316 excluded patients (i.e., 6.96% positive): of the 184 patients who would be tested for typhoid fever, 25 (13.59%) were positive. In a clinical setting, considerable money could be saved on diagnostic tests if only patients with a current fever of ≥38.5°C (axillary temperature) were tested for typhoid fever. Patients not tested would need to be advised to present in 2 to 3 days if symptoms persisted to minimize the number of true cases not correctly diagnosed. On the basis of our findings, there could be a reduction of 63% in the number of tests required, with no compromise in the ability to accurately diagnose typhoid fever.

It has been speculated that the lack of specificity of some rapid diagnostic tests might be in part due to the inability of blood culture to detect all active infections (13), i.e., the rapid diagnostic kits may be correctly diagnosing an active typhoid case when blood culture is negative. The use of both blood culture and PCR to detect S. Typhi infection is no guarantee that all true positive cases will be detected, but it did increase the rate of detection. However, in our study the specificity of the rapid diagnostic tests evaluated did not improve when the composite reference standard was used, as opposed to blood culture alone. This suggests some inherent specificity issues with the currently available typhoid fever diagnostic assays.

In the primary health care setting, none of the established typhoid rapid diagnostic tests is sufficiently sensitive to warrant their immediate routine use in PNG. Similar findings have been observed in Bangladesh (21) and sub-Saharan Africa (14), although under certain circumstances, it has been suggested that the use of rapid diagnostic tests with known limitations might be of some value (7). On the basis of our findings, the TR-02 prototype warrants further evaluation for potential routine use in PNG and other settings. Moreover, the TR-02 is user-friendly, being a solid-phase immunochromatic assay (similar to a rapid pregnancy test or a malaria RDT); can be stored at 20 to 25°C; is rapid (∼10 min); and is easy to interpret. It detects the same anti-Salmonella antibodies as does the TyphiDot assay but has better sensitivity and specificity. This is presumably a function of the different immune platforms on which the tests are based. Another difference is that the TR-02 detects only IgM, whereas the TyphiDot and Tubex detect both IgM and IgG. The TR-02 prototype and the TyphiDot assay have recently become available through Reszon Diagnostics International Sdn. Bhd (Malaysia); the prototype TR-02 is marketed under the name TyphiDot Rapid (or Typhi Rapid).

Typhoid fever diagnostic evaluations conducted on hospitalized patients (13, 25) provide little insight into the application of the diagnostic tests in the community health care setting. However, it is the primary health care level where sensitive, specific, rapid, cheap, and user-friendly typhoid diagnostic kits are most required. The clinical differentiation of typhoid fever from other causes of febrile illness present in regions where typhoid is endemic (e.g., malaria, dengue and other arbovirus infections, rickettsial infections, and leptospirosis) is difficult, but the treatments differ markedly (20, 26). Moreover, with the rollout of malaria rapid diagnostic tests through the Global Fund Initiative, there may be an increased need for diagnosis of other causes of febrile illness if the guidelines are adhered to and no malaria treatment is given to malaria RDT-negative patients. It is in this context that high-grade fever is important: in areas of malaria and typhoid endemicity where a malaria RDT yields a negative result, there may be clinical signs and symptoms such as severity of fever that can help determine the value of conducting a typhoid diagnostic test without negatively impacting patient outcome.

It is likely that the Widal test will remain in use in PNG and many other low-income settings for the foreseeable future, despite its known limitations in such settings (17). We have demonstrated that using the locally recommended cutoff titer of 160 gives rise to a test with poor sensitivity, albeit high specificity. Lowering the cutoff titer may improve sensitivity but decrease specificity. If use of the Widal test is continued in PNG, a multicenter study should be considered to determine background titers in asymptomatic patients and to further investigate the cutoff titer which is optimized for both sensitivity and specificity.