Impact of Reference Standard Sensitivity on Accuracy of Rapid Antigen Detection Assays and a Leukocyte Esterase Dipstick for Diagnosis of Chlamydia trachomatis Infection in First-Void Urine Specimens from Men

Abstract

A total of 128 previously frozen first-void urine (FVU) specimens from selected asymptomatic men were centrifuged and tested by three Chlamydia trachomatis rapid antigen detection tests and with a leukocyte esterase (LE) dipstick. When the results were compared to those of a reference standard of positivity determined by the Chlamydiazyme enzyme immunoassay as confirmed by a blocking assay, the sensitivities of the Testpack Chlamydia (Abbott), Clearview Chlamydia (Unipath), and Surecell Chlamydia (Kodak) tests and the LE dipstick test were 76.4, 76.4, 67.3, and 88.6%, respectively. Use of the ligase chain reaction (LCR), whose results were confirmed by direct fluorescent-antibody staining of elementary bodies, as the reference standard reduced the sensitivities to 70.9, 67.7, 62.9, and 87.5%, respectively. The specificities by use of LCR as the reference standard were 95.5, 95.5, 100, and 92.4%, respectively. These rapid chlamydial antigen tests performed reasonably well with FVU specimens, but the simple LE dipstick test, which had the highest sensitivity, would have enabled treatment of the greatest number of infected male patients.

Because Chlamydia trachomatis is such a prevalent sexually transmitted organism, causing serious sequelae such as pelvic pain, ectopic pregnancy, and infertility, there is a need to screen individuals and perform diagnostic tests which would enable immediate treatment of infected individuals. The Sexually Transmitted Diseases Diagnostics Initiative has promoted the development and use of rapid diagnostic tests for use in developing countries (3). Several commercial point-of-care (POC) tests have been developed and have shown variable performance traits. Rapid POC assays such as Testpack Chlamydia (Abbott, North Chicago, Ill.), Clearview Chlamydia (Unipath, Bedford, United Kingdom), and Surecell Chlamydia (Kodak, Rochester, N.Y.) have been used in doctor’s offices in developed countries. They have been compared individually or to one another by use of cervical swab, urethral swab, and urine specimens, and the reference standard of comparison has been culture, another antigen detection assay, a nucleic acid amplification test, or a combination of assays that create an expanded reference standard. Tests with rapid leukocyte esterase (LE) dipsticks have been used as surrogates for the identification of patients with chlamydia urethritis but have rarely been evaluated with urine specimens from men by using expanded reference standards for the detection of C. trachomatis and/or Neisseria gonorrhoeae.

Because urine has proved to be a convenient specimen for use in the testing or screening for C. trachomatis in men and can be obtained by noninvasive means, (4) we tested selected specimens with LE dipsticks and examined the abilities of three rapid POC tests to detect C. trachomatis antigens in first-void urine (FVU) from men. We compared their performances to the results achieved by a ligase chain reaction (LCR) assay (LCx) and a confirmed Chlamydiazyme assay (Abbott).

MATERIALS AND METHODS

Patient specimens.

A total of 55 positive specimens and 73 negative specimens were selected from 762 males who submitted an FVU specimen (first 20 to 30 ml of any void) for testing for C. trachomatis in a private laboratory in Toronto, Ontario, Canada, by the Chlamydiazyme enzyme immunoassay (EIA). The specimens were coded in a blinded fashion and were then shipped frozen to the Regional Chlamydiology Laboratory in Hamilton, Ontario, Canada. The urine specimens were thawed, vortexed for 20 s, and then divided into aliquots for the three rapid tests and the LCx assay. The study was conducted during 1997.

Assay procedures.

The Chlamydiazyme assay was performed according to the manufacturer’s package insert, with confirmatory testing done for all positive specimens and those for which the results fell into a grey zone below the cutoff (11). The 1-ml sample for testing by the LCx assay was processed according to the package insert and as described previously (4). All specimens positive by the LCx assay were stained with the Microtrak direct fluorescent-antibody (DFA) reagent, and staining of a minimum of three elementary bodies (EBs) was required to define a positive (11).

For the Clearview Chlamydia test the package insert instructions were followed, with a few modifications. A minimum of 5 ml was topped with 15 ml of distilled water in a conical tube, and then the tube was centrifuged for 20 min at 2,000 × g. The pellet was resuspended in 0.6 ml of extraction reagent I and was vortexed for 30 s, the suspension was then transferred to an extraction tube, and the tube was heated for 10 min at 80°C. Five drops of this extracted specimen was transferred to the sample well, and the results were read after 15 min.

For the Testpack Chlamydia assay we adhered to the package insert instructions but made the following modifications: The urine was centrifuged for 2 min at 2,000 × g, and the pellet was resuspended in 1 drop of reagent A and 2 drops of reagent B and was then vortexed for 15 s. The sample was transferred to an extraction device for filtration. The filtrate was poured into a reaction disc and was then washed with reagent C. We added 2 drops of reagent D for 5 min, followed by the addition of 2 drops of reagent E. A final wash with reagent F was followed by the addition of 2 drops of reagent G, and the results were read after 5 min.

For the Surecell Chlamydia test we followed the kit instructions, with slight modifications. We centrifuged the 5 ml of FVU at 2,000 × g for 10 min and then resuspended the pellet in 8 drops of reagent 1. The specimen was transferred to a Surecell extraction tube, in which it was mixed for 15 s. We mixed in 10 drops of reagent 2 and 2 drops of reagent 3 at 2-min intervals. Five drops of the fluid was filtered into each of the three wells of the disposable test cell. The test was read within 5 min.

LE dipsticks (Chemstrip 2LN; Boehringer, Indianapolis, Ind.) were used as described previously (23). The enzymatic activity was scored as trace, 1+, or 2+, according to the manufacturer’s chart, at 1 min.

Performance calculations.

The reference standards for performance of calculations were specimens that were positive by the Chlamydiazyme assay and whose results were confirmed by the blocking test, as indicated above, or the LCx assay results as confirmed by DFA staining of the centrifuged urine. Sensitivity and specificity were calculated by standard techniques.

RESULTS

Of the 128 urine specimens tested, the 55 specimens confirmed to be positive by the Chlamydiazyme assay were also positive by the LCx assay and contained EBs. Eleven of the 73 samples which were negative as confirmed by the Chlamydiazyme assay were positive by the LCx assay, but only 7 of the 11 samples had three or more EBs. The four LCx assay-positive specimens without EBs were negative by the three rapid tests for detection of C. trachomatis, but two were positive with a 2+ reading by the LE dipstick test. Therefore, in the comparison of the rapid tests for the detection of C. trachomatis we had 55 specimens that were confirmed to be positive by the Chlamydiazyme assay and 73 that were confirmed to be negative and 62 specimens that were confirmed to be positive by the LCx assay and 66 that were confirmed to be negative. A subset of 48 LCx assay-positive specimens and 66 negative specimens was available for testing by the LE dipstick test.

The sensitivities and specificities of each rapid test for the detection of C. trachomatis antigen and the LE dipstick assay are presented in Table 1, with both LCR and Chlamydiazyme results used as reference standards. All of the rapid tests had higher sensitivities if the less sensitive Chlamydiazyme test was used as the reference standard (elevations of approximately 5%). Compared with the sensitivity of the LCx assay, the Testpack Chlamydia test had a slightly higher sensitivity of 70.9%, followed by the Clearview Chlamydia (67.7%) and Surecell Chlamydia (62.9%) assays. Conversely, the Surecell Chlamydia test was 100% specific, while by each of the other two tests three specimens had false-positive results, for a specificity of 95.5%. The LE dipstick test showed 87.5% sensitivity (7 specimens had scores of a trace, and the remaining 35 specimens had scores of 1+ or 2+) compared to the results of LCR, with 5 specimens having false-positive results by the LE dipstick test (specificity, 92.4%).

TABLE 1.

Diagnostic performances of three rapid tests and LE dipstick assay compared to those of two reference standards for detection of C. trachomatis with FVU specimens from men

Rapid test	Standard	% Sensitivity	% Specificity
Testpack Chlamydia (Abbott)	EIAa	76.4 (42/55)b	93.2 (68/73)
	LCRc	70.9 (44/62)	95.5 (63/66)
Surecell Chlamydia (Kodak)	EIA	67.3 (37/55)	97.3 (71/73)
	LCR	62.9 (39/62)	100 (66/66)
Clearview Chlamydia (Unipath)	EIA	76.4 (42/55)	95.9 (70/73)
	LCR	67.7 (42/62)	95.5 (63/66)
LE dipstick (Boehringer)	EIA	88.6 (39/44)	89.0 (65/73)
	LCR	87.5 (42/48)	92.4 (61/66)

^aSpecimens positive by the Chlamydiazyme EIA confirmed with chlamydia blocking reagent. 

^bValues in parentheses are number of positive specimens/number of specimens tested. 

^cSpecimens positive by LCR by the LCx Chlamydia assay as confirmed by the presence of three or more EBs by fluorescent-antibody staining. 

DISCUSSION

Compared to one of the most sensitive assays available for detection of C. trachomatis in urine samples (LCx chlamydia), the three rapid tests identified between 62.9 and 70.9% of the positive urine samples. Use of a less sensitive test, such as the Chlamydiazyme assay as the reference standard, inflated the sensitivities by approximately 5 to 9%. In the comparison of assays in which LCR was used as the reference standard, the Testpack and Clearview Chlamydia assays recorded false-positive results, while the Surecell test did not.

The only other evaluation of the Testpack Chlamydia assay with FVU specimens from men was published by Sellors et al. (17). They compared the results of the test with FVU specimens to the results of urethral swab cultures and showed that the Testpack Chlamydia assay was 76.2% sensitive and 95.5% specific, whereas the Chlamydiazyme assay was 81.0% sensitive and 96.5% specific. The present study indirectly confirms these data. Other studies with the Testpack Chlamydia assay have focused on determination of the test’s ability to diagnose infections by use of traditional swab specimens. The original publication of Coleman et al. (5) showed that the Testpack Chlamydia assay with endocervical swab specimens was 76.5% sensitive in a study in which culture sensitivity was 86.7%. In a comparison of the Testpack Chlamydia assay to the Chlamydiazyme assay with cervical swab specimens from 1,376 women in whom the prevalence of C. trachomatis infection was 3.6%, Mercer et al. (12) reported a very high sensitivity and a very high specificity (90 and 98.6%, respectively). In contrast, Hook and coworkers (9) studied a population of women in Baltimore, Md., with a 6.6% prevalence of C. trachomatis infection and found that the Testpack Chlamydia assay with cervical swab specimens had a very low sensitivity (48.5%) compared to the results of culture. The specificity in that study was 99.7%. Other studies (5–7, 15, 16) have shown that the Testpack Chlamydia assay with cervical swab samples had sensitivities between 51.7 and 70% compared to the results of culture, the Chlamydiazyme assay, or an expanded reference standard by DFA staining.

This is the first report of a study in which the Clearview Chlamydia assay was used with FVU specimens from men. When LCR with DFA staining was used as the reference standard, the sensitivity of the Clearview Chlamydia assay was 67.7% and there were three specimens with false-positive results. We were unable to test nonfrozen urine specimens, as indicated in the manufacturer’s instructions, which may have affected the values that we achieved. Compared to culture, the Clearview Chlamydia test with cervical swab specimens has been shown to have sensitivities ranging from 79.0 to 95.0% (2, 6, 10, 20, 22, 25), with very few false-positive results. Using an expanded reference standard with PCR and discrepant analysis, Kluytmans et al. (10) demonstrated that the Clearview Chlamydia test had sensitivities and specificities of 62.3 and 99.7%, respectively, with cervical swab specimens from women and 60.4 and 86.3%, respectively, with urethral swab specimens from men. These sensitivities are similar to those achieved in our study when LCR was used as the reference test. Kluytman et al. (10), in the study with urethral swab specimens from men, assumed that the lower specificity was created by substances in the swabs that they used.

In our study the Surecell Chlamydia test was the least sensitive but was the most specific test for the detection of C. trachomatis in FVU specimens from men, although this was not statistically significant (P < 0.05). Two other studies have used this assay with urine specimens from men. Ferris et al. (8) showed that the Surecell Chlamydia test with centrifuged FVU specimens from 207 men had a sensitivity of 70% and a specificity of 90% compared to the results of culture. Schubiner et al. (16) demonstrated that the Surecell Chlamydia test had a sensitivity of 64% with urethral swab specimens from men. Moncada and coworkers (13) showed that the Surecell Chlamydia test was 85% sensitive and 97% specific with FVU specimens from 1,341 symptomatic men, with the sensitivity of the Chlamydiazyme assay being 91%. We do not have accurate figures but estimate that more than 50% of the men tested in our study were asymptomatic, which might explain the lower positivity rates. Other studies of the Surecell Chlamydia test with cervical swab specimens from women (6, 7, 16, 21, 24) have reported that its sensitivity ranges from 75 to 90.0% compared to the results of culture or expanded standards.

The three rapid POC tests showed few false-positive results in the comparison to LCR (3 of 66 specimens for the Testpack and Clearview Chlamydia tests and none for the Surecell Chlamydia test). Examination of false-positive results for all of the published studies in which specificity with swab specimens was evaluated has shown relatively few false-positive results compared to the number of true-positive results even in populations with a low prevalence of C. trachomatis infection. The lowest specificities have been reported when the tests have been performed with FVU specimens. Using the Surecell Chlamydia test in a study with a population with a 14% prevalence of infection, Moncada et al. (13) reported that 9 specimens had false-positive results, whereas 343 specimens were negative (specificity, 97%); and in an evaluation of concentrated FVU specimens from men with a 10.3% prevalence of infection, Ferris et al. (8) showed that 6 specimens had false-positive results, whereas 175 specimens were negative (specificity, 96%). Sellors et al. (17) also experienced a higher rate of false positivity (9 of 198 positive specimens) by the Testpack Chlamydia assay with FVU specimens from men with a 14% prevalence of C. trachomatis infection.

Compared to the three rapid tests for the detection of C. trachomatis, the LE dipstick test detected a higher percentage of positive specimens. The LE dipstick test results in this study confirmed what has been seen in three other studies (18, 19, 23), that this marker for urethritis may be a sensitive test for screening for C. trachomatis infection but that it usually lacks specificity. In this selected group of men, none of whom had N. gonorrhoeae infection, inclusion of LCR testing with the Chlamydiazyme assay increased the specificity of LE dipstick testing from 89.0 to 92.4% and reduced the sensitivity by 1%. Ånestad et al. (1) found that 8 of 10 LCR-positive urine specimens from 358 asymptomatic men were positive by the LE dipstick. An additional 17 were LE dipstick test positive and LCR negative, indicating a poor specificity of the LE dipstick test. Use of an LE dipstick test as a rapid initial test may be cost beneficial, even with an asymptomatic population with other causes of urethral inflammation.

In summary, we have shown that rapid tests for detection of C. trachomatis antigens have reduced sensitivities when more sensitive reference standards are used for comparison. More recently, two new rapid tests have been evaluated. Quick Vue-Chlamydia (Quidel Corp., San Diego, Calif.) was shown to be 92.0% sensitive and 99.1% specific with 742 cervical swabs specimens compared with the results of culture when the discrepancies were resolved by PCR (21). The Biostar optical immunoassay (Biostar Inc., Boulder, Colo.) was evaluated with 306 cervical swab specimens (prevalence of C. trachomatis infection, 13.7%) collected from women attending a sexually transmitted disease clinic (14). Compared to an expanded standard which combined culture, DFA staining, and PCR, the Biostar optical immunoassay was 73.8% sensitive compared to the results of a combination of culture and PCR, which were 92.9% sensitive, and DFA staining, which was 59.5% sensitive; and there were no false-positive results. It has been argued that better rapid tests for the diagnosis of chlamydial infections are needed to enable immediate treatment of infected persons and their contacts in order to prevent sequelae. If POC tests were designed to accommodate specimens that can be collected by noninvasive means, such as urine, the tests would also be more useful. When such tests are evaluated, the most accurate standard of comparison should be used, and this should include at least one nucleic acid amplification test.