Sensitivity and specificity of typhoid fever rapid antibody tests for laboratory diagnosis at two sub-Saharan African sites

Abstract

Objective

To evaluate three commercial typhoid rapid antibody tests for Salmonella Typhi antibodies in patients suspected of having typhoid fever in Mpumalanga, South Africa, and Moshi, United Republic of Tanzania.

Methods

The diagnostic accuracy of Cromotest^® (semiquantitative slide agglutination and single tube Widal test), TUBEX^® and Typhidot^® was assessed against that of blood culture. Performance was modelled for scenarios with pretest probabilities of 5% and 50%.

Findings

In total 92 patients enrolled: 53 (57.6%) from South Africa and 39 (42.4%) from the United Republic of Tanzania. Salmonella Typhi was isolated from the blood of 28 (30.4%) patients. The semiquantitative slide agglutination and single-tube Widal tests had positive predictive values (PPVs) of 25.0% (95% confidence interval, CI: 0.6–80.6) and 20.0% (95% CI: 2.5–55.6), respectively. The newer typhoid rapid antibody tests had comparable PPVs: TUBEX^®, 54.1% (95% CI: 36.9–70.5); Typhidot^® IgM, 56.7% (95% CI: 37.4–74.5); and Typhidot^® IgG, 54.3% (95% CI: 36.6–71.2). For a pretest probability of 5%, PPVs were: TUBEX^®, 11.0% (95% CI: 6.6–17.9); Typhidot^® IgM, 9.1% (95% CI: 5.8–14.0); and Typhidot^® IgG, 11.0% (6.3–18.4). For a pretest probability of 50%, PPVs were: TUBEX^®, 70.2% (95% CI: 57.3–80.5); Typhidot^® IgM, 65.6% (95% CI: 54.0–75.6); and Typhidot^® IgG, 70.0% (95% CI: 56.0–81.1).

Conclusion

Semiquantitative slide agglutination and single-tube Widal tests performed poorly. TUBEX^® and Typhidot^® may be suitable when pretest probability is high and blood cultures are unavailable, but their performance does not justify deployment in routine care settings in sub-Saharan Africa.

Introduction

Typhoid fever remains an important cause of disease in developing countries. In 2002, it caused an estimated 408 837 episodes of illness in Africa.¹ Salmonella Typhi, the causative agent, is most frequently isolated from blood during the first week of illness but can also be isolated during the second or third week of illness, during the first week of antimicrobial therapy and during clinical relapse.² Isolation of Salmonella Typhi from bone marrow is the current gold standard method for confirming a case of typhoid fever. However, this requires equipment, supplies and trained laboratory personnel seldom found in primary health-care facilities in the developing world.^3,4 Blood culture is a more practical albeit less sensitive alternative to bone marrow culture. However, it is not always available and, when it is, it takes 2 to 3 days. As a result, diagnosis may be delayed or overlooked and patients without typhoid fever may receive unnecessary and inappropriate antimicrobial treatment. For this reason, in developing countries typhoid rapid antibody tests can facilitate diagnosis and disease management.

New commercially available typhoid rapid antibody tests have been evaluated in Asia, where typhoid fever is known to be highly endemic.^1,5 In Asian studies the tests have shown variable performance. While the TUBEX^® test was the most sensitive and specific in the Philippines,⁶ neither TUBEX^® nor Typhidot^® was both sensitive and specific in two evaluations undertaken in Viet Nam^7,8 and performance was poor in a trial conducted in a community clinic in Bangladesh⁹ and in a study in Egypt in which it was compared with a new ELISA not yet commercially available.¹⁰ Rapid typhoid tests have not been evaluated in sub-Saharan Africa, where the typhoid fever burden may be smaller than in Asia.^11–13 The World Health Organization (WHO) has issued no recommendations on the use of typhoid rapid antibody tests.¹⁴ Accurate diagnostics for typhoid fever could provide valuable diagnostic information for patient management and make it possible to estimate the incidence of typhoid fever in low-resource settings.

We evaluated three diagnostic kits that are commercially available internationally using four rapid methods for detecting antibodies to Salmonella Typhi (typhoid rapid antibody tests) and used blood culture as the standard for comparison.

Methods

Participants

Patients were recruited from two sub-Saharan African sites: Mpumalanga province, South Africa, and Moshi, United Republic of Tanzania. They were selected to represent patients from southern and eastern Africa, respectively.

The recruitment method differed between sites. In South Africa, we enrolled subjects suspected of having typhoid fever; in the United Republic of Tanzania, we enrolled patients who were participants in a study on the etiology of febrile illness.

Patients were recruited at both sites between 2007 and 2009. In the South African site we obtained blood from suspected typhoid fever cases reporting no current use of antimicrobials who presented to Rob Ferreira Hospital (RFH), in Nelspruit, Mpumalanga province, or to hospitals referring patients to RFH. In the United Republic of Tanzania site, we obtained blood from consecutive febrile inpatients admitted to Kilimanjaro Christian Medical Centre (KCMC) and Mawenzi Regional Hospital.^15,16 At both sites we incubated the blood in a continuously monitored blood culture system (Bac-T Alert, bioMérieux, Marcy L’Étoile, France). Bottles flagged as positive by the instrument were removed for subculture and identification by standard techniques.¹⁷

In both study sites, we enrolled patients who presented with a febrile illness suspected of being typhoid fever. We collected data on those patients who fulfilled the clinical criteria for suspected typhoid fever (a history of fever or demonstrated pyrexia [body temperature > 38 °C.]) before performing the index test and blood culture.

Test methods

In both study sites a typhoid fever case was defined as being a patient whose blood culture was positive for Salmonella Typhi. Patients whose blood cultures were negative or yielded pathogens other than Salmonella Typhi were used as controls. We drew additional blood and separated the serum, which was stored at −20 or −80 °C in cryotubes and shipped on dry ice to the Enteric Diseases Reference Unit, National Institute of Communicable Diseases (Sandringham, South Africa), for evaluation with typhoid rapid antibody tests. We screened the serum using the semiquantitative slide agglutination and tube Widal tests, TUBEX^® and the typhidot test. Laboratory staff were blinded to the blood culture results, which were reviewed only after testing was completed.

Typhoid rapid antibody tests were carried out according to manufacturers’ instructions. Test characteristics are summarized in Table 1. Laboratory personnel, trained in the use of all tests, recorded information about the ease of use and non-kit consumables and equipment required for each test. Because the cost of consumables, equipment and personnel differed between the two study sites, we did not calculate the cost of the tests.

Table 1. Comparative characteristics of three rapid tests for the detection of Salmonella Typhi antibodies.

Characteristic	Cromotest® – semiquantitative slide agglutination	Cromotest® – single tube Widal	TUBEX®	Typhidot®
Antibody	IgM and IgG	IgM and IgG	IgM	IgM or IgG
Antigen	O and H	O and H	O9	OMP
Turnaround time per test	2 minutes at room temperature	O: 4 hours at 50 °C	3 minutes at room temperature	60 minutes at room temperature
		H: 2 hours at 50 °C
Storage temperature (°C)	2–8	2–8	2–8	2–8
Equipment supplied by manufacturer	Febrile antigen	Febrile antigen	Colour scale	Predotted antigen strips
	Positive control	Positive control	Blue and brown reagent	Sample diluent
	Negative control	Negative control	Negative control	Washing buffer
			Positive control	Prediluted anti-human IgM and IgG
			Reaction well strip	Substrate A and B
			Sealing tape	Positive control
			Coloured sticker	Negative control
			Timer	Worksheet
Equipment supplied by laboratory	Disposable slides	Thermostatic waterbath (30–50 °C)	Precision pipette	Measuring cylinder
	Saline solution	Disposable sterile glass tubes (12 × 100 mm)	Vortex	Micropipettes and tips
		Disposable stirrers		Conical flask
		Saline solution		Forceps, wash bottle
		Mechanical stirrer		Filter paper, distilled water
				Rocker platform
				Aspirator
				Aluminium foil
				Dark reagent bottle/flask covered with aluminium foil
User comments	Particles present before adding the antisera, rendering false-positive results. Simple to use and inexpensive.	Particles present before adding the antisera, rendering false-positive results. Requires costly additional laboratory equipment.	Subjective interpretation of colour reactions. Haemolysis may result in difficulty in interpretation. Simple to use and limited need for additional laboratory equipment.	More complex assay requiring additional steps and preparation of consumables. Interpretation may be affected, as IgG can persist for more than 2 years after typhoid infection. Detection of specific IgG cannot differentiate between acute and convalescent cases. Requires costly additional laboratory equipment.

OMP, outer membrane protein.

Linear Cromotest^® (Linear Chemicals, Barcelona, Spain)

This test, derived from Salmonella Typhi O and H antigens, was performed in two ways: (i) as a semiquantitative slide agglutination test with visual examination as per the package insert; (ii) as a Widal test¹⁸ performed with a single tube, as described by Parry et al.¹⁹ The presence or absence of visible agglutination indicates the presence or absence of the corresponding antibody to the O and H antigens of Salmonella Typhi. We defined the positivity cut-off point for the slide and tube agglutination reactions for both O and H antigens as antibody titres ≥ 1:80.

IDL TUBEX^® TF (IDL Biotech AB, Bromma, Sweden)

This semiquantitative colorimetric test detects anti-O:9 antibody titres in patient specimens on visual examination.²⁰ A positive TUBEX^® result was defined as a reading of ≥ 4, as per manufacturer’s instructions. The manufacturers warn that the test may have to be repeated after 48 hours if indeterminate results are obtained.

Typhidot^® (Malaysian Biodiagnostic Research, Bangi, Malaysia)

This qualitative antibody detects the presence of IgM and IgG antibodies to a 50kDa outer membrane protein.²¹ A positive Typhidot^® result (IgG and IgM) was defined as a visible reaction of an intensity equal to or greater than that of the control reaction on the commercially prepared filter paper. The manufacturers warn that if indeterminate results are obtained, the test may have to be repeated after 48 hours.

Statistical methods

Data were captured into Excel 2003 (Microsoft Corporation, Redmond, United States of America) and converted to STATA version 11 (StataCorp. LP, College Station, USA), in which analysis was performed by Stat/Transfer version 10 (Circle Systems, Seattle, USA). Stata’s diagt command was used to determine each test’s sensitivity, specificity and positive and negative predictive values (PPV and NPV, respectively),²² which are presented with the understanding that exposure to antimicrobials could have affected the final results. Analysis was performed at a two-sided significance level of 5%. Pretest probabilities of background typhoid fever rates were also calculated at 5% and 50% to ensure that the results were applicable even in conditions of few typhoid fever outbreaks – since incidence would be higher during outbreaks – and of lower endemicity, given that study patients were selected on the probability of having typhoid fever.

Research ethics

The NICD has blanket ethics clearance in relationship to its surveillance duties (M06–04–49), but further approval was obtained from the Committee for Research on Human Subjects (CRHS) at the University of the Witwatersrand to update CRHS on this aspect of typhoid fever surveillance.

The part of the study conducted in the United Republic of Tanzania was approved by the Kilimanjaro Christian Medical Centre (KCMC) Research Ethics Committee, the United Republic of Tanzania National Institutes for Medical Research National Research Ethics Coordinating Committee, and an institutional review board of Duke University Medical Center.

Results

Participants

Ninety-two patients were enrolled: 53 (58%) in South Africa (between 25 May 2007 and 10 November 2009) and 39 (42%) in the United Republic of Tanzania (between 17 September 2007 and 25 August 2008). Participants had a median age of 24 years (range: < 1 to 96). Twenty-five (27%) patients (23 South African and 2 Tanzanian) were under the age of 15 years; the ages of two participants (2.2%) were unknown. Forty-two (46%) patients were female; the sex of two (2%) was not available. Thirty-six (39%) blood cultures grew a pathogen; 28 (78%) of these cultures grew Salmonella Typhi. Other pathogens isolated included Salmonella Typhimurium, Streptococcus pneumoniae, Staphylococcus aureus and Mycobacterium tuberculosis (one culture each) and Cryptococcus neoformans (four cultures). Of the 92 blood cultures, 52 (57%) were negative and 4 (4%) grew organisms considered to be contaminants.

In compliance with eligibility criteria, no South African patient was taking antimicrobials at the time of the blood culture. Of the 20 Tanzanian patients whose blood cultures were negative for Salmonella Typhi, 2 (10%) had received trimethroprim–sulfamethoxazole and 8 (40%) had received quinine or sulfadoxine-pyrimethamine or had an unknown history of antimicrobial exposure. Of 19 Tanzanian patients with blood cultures positive for Salmonella Typhi, 17 (89%) had received antibacterials or antimalarials or had an unknown history of antimicrobial exposure.

Test results

Blood cultures were done as soon as patients were admitted to hospital. Serological tests were performed within the subsequent 6 months at the South African site and within 2 years at the Tanzanian site. Table 1 shows the characteristics of the three assays, the equipment required to perform each test and the results of the technologists’ assessments regarding ease of use and perceived laboratory costs. None of the sera gave indeterminate results.

Estimates

Sensitivity, specificity and predictive values are shown in Table 2. Of 28 patients with a blood culture positive for Salmonella Typhi, 1 (4%) was positive on the Cromotest^® semiquantitative slide O test; 14 (50%) were positive on the Cromotest^® semiquantitative slide H test; 2 (7%) were positive on the Cromotest^® Widal O agglutination test; 4 (14%) were positive on the Cromotest^® Widal H agglutination test; and 19 (68%) were positive on the TUBEX^® test. Of 27 patients with a blood culture positive for Salmonella Typhi with sufficient serum available for testing, 19 (70%) were positive on the Typhidot^® IgG test and 17 (63%) on the Typhidot^® IgM test. The positive and negative predictive values for each of the pretest probability calculations are presented in Table 3.

Table 2. Sensitivity, specificity and predictive values of four rapid diagnostic tests for typhoid fever as determined by comparison with blood culture results.

Kit	Sensitivity % (95% CI)	Specificity % (95% CI)	PPV % (95% CI)	NPV % (95% CI)
Cromotest® O : semiquantitative slide agglutination	95.2 (86.5–99.0)	3.6 (0.1–18.3)	25.0 (0.6–80.6)	68.6 (57.7–78.2)
Cromotest® H : semiquantitative slide agglutination	80.3 (68.2–89.4)	50.0 (30.6–69.4)	53.8 (33.4–73.4)	77.8 (65.5–87.3)
Cromotest® O: single tube Widal	87.3 (76.5–94.4)	6.9 (0.8–22.8)	20.0 (2.5–55.6)	67.1 (55.8–77.1)
Cromotest® H: single tube Widal	95.2 (86.5–99.0)	13.8 (3.9–31.7)	57.1 (18.4–90.1)	70.2 (59.3–79.7)
TUBEX®	73.0 (60.3–83.4)	69.0 (49.2–84.7)	54.1 (36.9–70.5)	83.6 (71.2–92.2)
Typhidot® IgM	75.0 (61.1–86.0)	60.7 (40.6–78.5)	56.7 (37.4–74.5)	78.0 (64.0–88.5)
Typhidot® IgG	69.2 (54.9–81.3)	70.4 (49.8–86.2)	54.3 (36.6–71.2)	81.8 (67.3–91.8)

CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

Table 3. Predictive values of four rapid diagnostic tests for typhoid fever as determined by comparison with blood culture results under assumed pretest probabilities of 5% and 50%.

Kit	Pretest probability
	5%			50%
	PPV % (95% CI)	NPV % (95% CI)		PPV % (95% CI)	NPV % (95% CI)
Cromotest® O : semiquantitative slide agglutination	4.9 (4.5–5.4)	93.3 (60.4–99.2)		49.7 (47.4–51.9)	42.5 (7.4–87.2)
Cromotest® H : semiquantitative slide agglutination	7.8 (5.4–11.1)	98.0 (96.3–98.9)		61.6 (52.1–70.4)	71.8 (57.6–82.6)
Cromotest® O: single tube Widal	4.7 (4.1–5.4)	91.2 (70.0–97.9)		48.4 (45.0–51.8)	35.2 (10.9–70.6)
Cromotest® H: single tube Widal	5.5 (4.7–6.4)	98.2 (92.8–99.6)		52.5 (48.6–56.3)	74.0 (40.5– 2.3)
TUBEX®	11.0 (6.6–17.9)	98.0 (96.8–98.7)		70.2 (57.3–80.5)	71.9 (61.4–80.4)
Typhidot® IgM	9.1 (5.8–14.0)	97.9 (96.4–98.8)		65.6 (54.0–75.6)	70.8 (58.2–80.9)
Typhidot® IgG	11.0 (6.3–18.4)	97.8 (96.4–98.6)		70.0 (56.0–81.1)	69.6 (58.7–78.6)

CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.

Discussion

All four typhoid rapid antibody tests performed poorly compared with blood culture. Some tests performed better than others, but none stood out in all respects. In sub-Saharan Africa, cost and ease of use are important considerations in addition to diagnostic accuracy.

The single-tube Widal and Typhidot^® tests were found to require the most non-kit laboratory supplies, consumables and equipment, and this increased the overall cost of the test. The semiquantitative slide agglutination and TUBEX^® tests had shorter turnaround times than the Widal tube and Typhidot^® tests. However, the results of all the tests were available the same day the specimen was received in the laboratory.

Of the four tests evaluated, the semiquantitative slide agglutination test performed the worst. It had very poor specificity and low PPV and NPV, even though it was performed under optimal conditions in a national reference laboratory. This poor performance was further compounded by substantial inter-test variability, which suggests that in a field situation results would not be comparable between sites.²³ Hence, the slide agglutination test should not be used as a diagnostic tool. Although the sensitivity and specificity of the H slide agglutination test appeared to be greater, this was offset by the inconsistent results obtained with the O slide agglutination. Others have noted this disparity between the sensitivity and specificity of the Widal test containing O and H antigens.¹⁹

The single-tube Widal agglutination test also performed poorly. The original Widal agglutination test was described using paired sera obtained 10 days to 2 weeks apart and examined for a twofold or greater change in titre.¹⁸ It is possible that the Widal test would have performed better in our study had we used paired sera, but we chose to apply the test under the conditions normally found in clinical practice. In our experience, patients rarely return for outpatient follow-up once treated, so that obtaining paired sera in a routine clinical setting is unlikely. Recently, the use of paired sera has been re-examined and has been shown to improve both the sensitivity and specificity of serological tests for typhoid fever.²⁴

Both the TUBEX^® and Typhidot^® tests had lower sensitivity than the semiquantitative slide agglutination and the Widal tests, but they had considerably greater specificity. In our setting, TUBEX^® had marginally less sensitivity but more specificity than the Typhidot^® IgM test and it had a slightly better PPV. Typhidot^® IgG was comparable to TUBEX^® with respect to sensitivity, specificity and PPV, but none of these tests performed as well as the blood culture comparator assay.

This study had several limitations. Typhoid fever was confirmed by blood culture in almost one third of the study participants, a much larger proportion than expected under field conditions in sub-Saharan Africa, where typhoid fever is relatively uncommon.^12,13 Lowering the pretest probability for typhoid fever to 5% further degraded the performance characteristics of the typhoid rapid antibody tests (Table 3), which suggests that these tests would not be useful in routine diagnostic situations. At a pretest probability of 50%, higher than the actual fraction of blood-culture-positive cases used in this evaluation, the performance of the new rapid antibody tests improved. Hence, these tests can perhaps be judiciously used during outbreaks.

The time elapsed between the onset of symptoms and serum collection can affect the performance of antibody-based tests.²⁵ We did not analyse this aspect to reflect how the tests would be used under routine health-care conditions in sub-Saharan Africa. Similarly, human immunodeficiency virus (HIV) infection is highly prevalent in sub-Saharan Africa²⁶ and we enrolled participants without reference to their HIV serostatus to reflect field conditions, although many of our patients could have been HIV-infected. HIV infection rates among 1504 adult outpatients tested in Nelspruit (RFH) were reportedly as high as 45% in 2010 (G Hoyland, personal communication). The prevalence of HIV infection among participants in the study on febrile illness at KCMC and Mawenzi Regional Hospital was 39% for adolescents and adults and 12% for infants and children.^15,16 Although recent studies at KCMC have shown that HIV appears to protect against typhoid fever, disease may still occur in HIV-infected individuals.¹⁵ It is possible that HIV-associated immune dysregulation affects the production of antibodies specific to Salmonella Typhi outer membrane proteins, present in both the Typhidot^® and the older Widal tests. This has been observed in patients infected with invasive non-typhoidal Salmonella (NTS).²⁷ This theoretical effect can also impair antibody binding in the TUBEX^® test, which is based on the O9 antigen. The production of antibodies against Salmonella lipopolysaccharide (LPS) is increased in patients with invasive NTS infection who are also HIV-infected. If antibody production were also higher in HIV+ typhoid fever patients, TUBEX^® would have performed better than the other typhoid rapid antibody tests, but it did not.

The sensitivity of blood culture is known to be less than 100%, even in the absence of antimicrobial exposure, and is further reduced by patient antimicrobial use. Although two Tanzanian patients had been exposed to antibacterials, they represented only 3.8% of the 52 patients whose blood cultures were negative for Salmonella Typhi. These patients probably affected our results very little. Furthermore, blood culture sensitivity was optimized in our study because we used modern blood culture techniques.²⁸

Our findings on the Widal test and the newer typhoid rapid antibody tests are similar to those from studies conducted in Asia and Egypt;^{8–10,25,29,30} none of the rapid tests performed nearly as well as blood culture for the diagnosis of typhoid fever. Some reports suggest that the Typhidot^® test may be more useful in Asia.^31,32 However, the true incidence of typhoid fever in the catchment population differed in these studies and in ours. The pretest probability of typhoid fever was artificially elevated in our evaluation because we included South African patients suspected of having typhoid fever and specifically analysed a subset of antisera from the United Republic of Tanzania in which half of the cases were known to have typhoid fever. The earlier studies also focused on paediatric populations and allowed for inclusion of microbiologically unconfirmed typhoid fever.^31,32

NTS bacteraemia, which is predominantly caused by Salmonella serotypes Typhimurium and Enteritidis, is much more common in sub-Saharan Africa than typhoid fever.^12,13 An important limitation of our study is the absence of cases of NTS in the control group; one patient in our study had Salmonella Typhimurium bacteraemia and none had Salmonella Enteritidis bacteraemia. It has been observed in previous studies that bacteraemia due to Salmonella Enteritidis may result in false-positive results with TUBEX^® because they have an O9 antigen in common.³³ Although the patient with Salmonella Typhimurium had negative typhoid rapid antibody tests, we could not examine the rate of false positives for the TUBEX^® test in patients with Salmonella Enteritidis bacteraemia.

In conclusion, typhoid rapid antibody tests appear to correlate poorly with blood culture results in sub-Saharan Africa, even in a study with inflated pretest probability. While such tests may be useful for rapidly diagnosing typhoid fever in emergencies – e.g. during outbreaks, when pretest probability would be high, and following blood culture confirmation of initial cases – their performance is unlikely to justify deployment in routine care settings in sub-Saharan Africa. TUBEX^® and Typhidot^® appeared to have comparable performance and were more specific although less sensitive than the semiquantitative slide agglutination test and the unpaired Widal test. Unpaired Widal and semiquantitative slide agglutination are unreliable, with poor specificity and PPV. It is important to remember that antimicrobial susceptibility testing and molecular epidemiological linkage cannot be elicited on serological diagnosis. Blood culture before initiating antimicrobial therapy remains the diagnostic method of choice.