Evaluation of COBAS AMPLICOR (Roche): Accuracy in Detection of Chlamydia trachomatis and Neisseria gonorrhoeae by Coamplification of Endocervical Specimens

Abstract

We evaluated further the accuracy of the COBAS AMPLICOR (Roche) (CA) PCR-based system in detection of Chlamydia trachomatis and Neisseria gonorrhoeae in endocervical specimens. Endocervical specimens collected for any indication for testing for C. trachomatis and N. gonorrhoeae among a university hospital health system population were included. Testing for C. trachomatis was done by two PCR methods, CA and manual microwell AMPLICOR (Roche) (MWA), and by culture; testing for N. gonorrhoeae was done by CA and culture. Discrepancy resolution was performed. Reproducibility testing and hands-on labor time measurements for CA were done. Among 654 C. trachomatis samples, the prevalence of true positivity was 9.2%, and among the 618 N. gonorrhoeae samples, the prevalence of true positivity was 4.4%. For detection of C. trachomatis, the sensitivity, specificity, and negative and positive predictive values were, respectively, as follows for each test: CA, 93.3, 99.7, 99.3, and 96.4%; MWA, 91.7, 99.7, 99.2, and 96.5%; and culture, 65.0, 100, 96.6, and 100%. For detection of N. gonorrhoeae those values were as follows: CA, 96.3, 100, 99.8, and 100%; and culture, 92.6, 100, 99.7, and 100%. Hands-on labor time for each clinical result was estimated to be at 7.5 min. The prevalence of inhibitory specimens was 3.5%, including two positive C. trachomatis samples which would have been missed otherwise. The direct cost of each clinical result with CA was estimated to be $9.09. Our methods include a diverse range of indications for testing among women, using endocervical swabbing samples, 2 M sucrose phosphate transport medium, and discrepancy resolution for comparison. Under our test conditions, the CA system is an accurate, rapid, and cost- and labor-efficient method for detection of C. trachomatis and N. gonorrhoeae.

Diagnosis of infection through laboratory testing based on the identification of amplified nucleic acid specific to the causative agent has been rapidly integrated into clinical practice, particularly in female reproductive tract infections (2). These tests offer enhanced accuracy and a marked decrease in time to result compared to traditional methods. In general these tests are also labor saving. Broad-based screening for genital Chlamydia trachomatis and Neisseria gonorrhoeae infections in women has been recommended (3), as they are prevalent and often asymptomatic and are the major known causes of ectopic pregnancy and tubal factor infertility when not detected and treated. They are also the two known causes of the acute clinical syndromes of mucopurulent cervicitis and pelvic inflammatory disease. Thus, the availability of a largely automated amplified nucleic acid-based test to assay simultaneously for chlamydia and gonococcal infection from a single specimen is an important advance and deserves intensive study in different settings to assess its true value.

In this study we evaluated the accuracy, reproducibility, labor requirements, and costs of the COBAS AMPLICOR CT/NG system (Roche Diagnostic Systems, Branchburg, N.J.) (CA) for the detection of C. trachomatis and N. gonorrhoeae infections in endocervical swab samples.

MATERIALS AND METHODS

Female patients for whom testing for endocervical C. trachomatis and N. gonorrhoeae infection was indicated, as determined by their practitioners, provided specimens for this study. These patients were seen in our emergency department, private and staff obstetric and gynecologic clinics, and inpatient units. Verbal consent for one extra endocervical swab sample was obtained from each patient, using a script provided to each practitioner. This script included the key elements of voluntary participation, confidentiality, risks, and identification of the procedures as medical research, and it was approved by our institutional review board.

Following any other sampling, an endocervical swab sample was plated directly onto a Jembec plate at room temperature for N. gonorrhoeae culture. A second Dacron tip, plastic stick swab sample of the endocervix was taken and inoculated into AMPLICOR transport medium (Roche) for testing in the manual microwell AMPLICOR (MWA) PCR-based system for detection of C. trachomatis, and the swab was discarded. Finally, a Dacron tip, plastic stick swab sample of the endocervix was obtained and inoculated into 1.5 ml of 2 M sucrose phosphate (2SP) transport medium prepared in our laboratory; this sample was used as the specimen for C. trachomatis culture and for assay in the CA system to detect C. trachomatis and N. gonorrhoeae simultaneously. Clinicians were instructed in this sequence of swab collection for this study, and this is the sequence routinely used at our institution; however, we did not monitor it during the study. Our 2SP medium includes 10% fetal calf serum pretested only for antagonists to growth of C. trachomatis in our culture system, gentamicin (10 mg/liter), vancomycin (100 mg/liter), and nystatin (25,000 U/liter); it is adjusted to pH 7.0 and filter sterilized at a pore size of 0.22 μm. Specimens in 2SP and AMPLICOR were stored at 4°C and shipped at ambient indoor temperature by a dedicated courier over 10 to 15 min to the laboratory daily.

Samples for N. gonorrhoeae culture were handled as routine specimens in our general microbiology laboratory. They were transported per the protocol of the Jembec system to the laboratory, incubated in 5% CO₂, and examined for typical colonies at 24, 48, and 72 h. Colonies with oxidase activity, gram-negative diplococci, and a positive Gonogen II (sensitivity, 98%; Becton Dickinson, Franklin Lakes, N.J.) reaction were reported as positive for N. gonorrhoeae; otherwise colonies were reported as negative. Chlamydia cultures were performed as previously described (7, 8), using aliquots from the specimen in 2SP medium prior to its use in the CA system. Briefly, our culture procedure uses centrifuge-assisted inoculation of the specimen onto McCoy cell monolayers on coverslips in shell vials, incubation at 37°C in 5% CO₂, inhibition of cellular energy use by cycloheximide, one blind passage, iodine staining, and examination at ×400 magnification for typical chlamydia inclusions by strict criteria (1). PCR testing for C. trachomatis in the MWA system was carried out in duplicate as previously described (8) (according to the protocol of the manufacturer), including methods of resolution for specimens with initially indeterminate results. Such resolution consisted of reassaying a fresh aliquot of the specimen in triplicate when an initial result showed absorbance of ≥0.2 to <0.5. Among the five results then available, the test was reported as positive if three or more values were ≥0.25, and otherwise the test was reported as negative.

Specimens in 2SP medium to be assayed simultaneously for detection of C. trachomatis and N. gonorrhoeae in the CA system were also handled according to the manufacturer's protocol. These specimens were kept at 2 to 8°C for up to 7 days, and thereafter they were kept at −70°C prior to processing. After cell lysis and addition of diluent, an aliquot of each specimen was added to a mixture provided by the manufacturer containing deoxynucleoside triphosphates, polymerase, biotinylated primers, uracil-N-glycosylase, and internal control components and was placed into the CA system, where amplification and detection of target DNA were carried out automatically (COBAS AMPLICOR CT/NG product manual, Roche Diagnostic Systems, Inc.).

All amplified DNA in the CA system contains uracil in place of guanosine. Uracil-N-glycosylase eliminates any uracil-containing DNA contaminating the specimen prior to the amplification steps, and it is then itself inactivated by the higher temperatures in those steps. The primers for C. trachomatis target DNA are specific to a cryptic plasmid common to all serotypes of the organism. Those for N. gonorrhoeae target the highly conserved M.Ngo PII gene, which is not found in most other species of the genus. The internal control consists of a random target DNA sequence and a recombinant DNA strand with primer binding sites identical to that of the chlamydia plasmid. Exposure of the strand to a primer identical to that for C. trachomatis but with a unique probe-binding site provides for a positive internal control result unless either the amplification process is inhibited or amplification of the internal control is competitively inhibited by an overwhelming mass of target chlamydial or gonococcal DNA. This internal control is calculated by the manufacturer to detect inhibition of amplification of as few as 20 copies of target DNA. Using the test as it is approved by the Food and Drug Administration, C. trachomatis specimens were determined to be positive if the absorbance value was ≥2.0 and negative if the absorbance value was <0.2 with positive internal control results (>0.2). Results in the range from 0.2 to 2.0 were equivocal, and the specimens needed to be retested by another means or another specimen from that sample had to be tested. We encountered two specimens with equivocal results in the initial test and retested them by CA in duplicate. Both yielded all results of <0.2 on the retests and were counted as negative specimens for the purposes of this study. N. gonorrhoeae specimens were determined to be positive if the absorbance was ≥3.5, and negative if the absorbance was <0.2 with positive internal control. Specimens with results in the range from 0.2 to 3.5 were retested in duplicate and deemed positive if two of the three results were >2.0. Nine N. gonorrhoeae specimens were found to have equivocal results in this study and were retested as described, yielding seven negative and two positive results.

The accuracy of each test that we studied was determined by comparison of the result for each specimen in each test as it would have been reported from the laboratory (i.e., after resolution of equivocal results, inappropriate positive or negative control results, specimens with amplification inhibitors, or instrument failure) to a true result for each specimen. If all results were concordant, this was considered the true result. If one of the three tests for C. trachomatis produced a discrepant result, all three were repeated: the culture with two blind passages, the MWA assay in triplicate, and the CA assay in duplicate. This process yielded five results for MWA and three for CA, and the majority result among them was considered to be the consensus result for each of these tests for the purpose of discrepancy resolution. Any positive result in culture assigned a true positive result to that specimen, as did concordant positive consensus results in the two PCR-based tests; otherwise a true negative result was assigned to the specimen. If the two tests for N. gonorrhoeae disagreed, the CA assay was repeated once in another laboratory where a ligase chain reaction assay (LCx; Abbott Laboratories, Abbott Park, Ill.) was also performed in duplicate. If two or more of these four amplified-DNA results or the culture were positive, the specimen was assigned a true positive result; otherwise it was assigned a true negative result. When negative internal controls were encountered, serial dilutions were carried out until a positive internal control result was obtained, and assays for C. trachomatis and N. gonorrhoeae were performed at that dilution as well.

For reproducibility testing we selected a clinical sample from this study series which was positive for both organisms and which yielded a result within the quantitative range of the spectrophotometer (absorbance of 0.2 to 4.0) for both when diluted 1:10. A specimen negative for both organisms was also used. Each of these two samples was tested as six replicates in a single assay daily for three consecutive days.

Workload analysis was carried out by timing the interval to complete the task for each labor-dependent step in the procedure based on 20 patient specimens (with control specimens), taking the mean of three measurements for each step, and rounding to the nearest 0.1-h interval.

Standard formulas for sensitivity, specificity, and negative and positive predictive values were used, as well as that for the coefficient of variation in the reproducibility testing.

RESULTS

Results were available from properly obtained, transported, and identified endocervical specimens for all test methods intended by this study for 654 chlamydia samples and 618 gonococcus samples. During the study period approximately 41 specimens were excluded from this study because they arrived in a transport medium other than 2SP medium, were from an area other than the endocervix, were collected with an improper swab, lacked results from all tests required by our protocol, or were left at room temperature for more than 1 h as reported by the provider.

Among the chlamydia samples, all three assays were negative for 590 samples, positive for 33 samples, and discrepant for 31 samples. After resolution of the discrepancies, there were a total of 594 true negative samples and 60 true positive samples, for a prevalence of 9.2%. Table 1 presents initial test results for each of the three assays compared to true results. Accuracy calculations are shown in Table 2.

TABLE 1.

Comparison of original results to true results for each test

Organism	Original resulta	No. of samples with result
		True positive	True negative
C. trachomalis	CA positive	56	2
	CA negative	4	592
	MWA positive	55	2
	MWA negative	5	592
	Culture positive	39	0
	Culture negative	21	594
Total		60	594
N. gonorrhoeae	CA positive	26	0
	CA negative	1	591
	Culture positive	25	0
	Culture negative	2	591
Total		27	591

^aResults as they would have been reported following routine assay and, if necessary, prescribed procedures for resolution of equivocal results, negative internal control results, and inappropriate results for control samples. 

TABLE 2.

Summary of accuracy of assays tested

Organism	Test	Sensitivity (%)	NPVa (%)	Specificity (%)	PPVb (%)
C. trachomatis	CA	93.3	99.3	99.7	96.4
	MWA	91.7	99.2	99.7	96.5
	Culture	65.0	96.6	100	100
N. gonorrhoeae	CA	96.3	99.8	100	100
	Culture	92.6	99.7	100	100

^aNPV, negative predictive value. 

^bPPV, positive predictive value. 

Among the gonococcus samples, both assays were negative for 591 samples, positive for 24 samples, and discrepant for 3 samples. One of these three was a positive culture and negative CA combination which remained negative when retested in CA and was positive in both LCx replicates. The other two were culture negative and CA positive, and both were positive on retesting in CA but negative in the LCx replicates. After discrepancy resolution, there were a total of 591 true negative samples and 27 true positive samples (prevalence, 4.4%). Table 1 summarizes results in the original assays relative to true results. Accuracy parameters for detection of N. gonorrhoeae by CA are presented in Table 2. Among the nine specimens with initially equivocal N. gonorrhoeae results, eight were culture negative, seven were negative by retesting in the CA system, and eight were negative when tested in LCx.

A negative result for the internal control was encountered with 35 specimens when initially assayed in the CA system. Of these, 12 specimens were positive in that assay for C. trachomatis and/or N. gonorrhoeae. All 35 specimens were retested after serial dilution, and all produced a positive internal control result with this procedure: 16 specimens when retested undiluted, 11 specimens at a 1:1 dilution, 5 specimens at a 1:5 dilution, and 3 specimens at a 1:10 dilution. The prevalence of inhibitory specimens was thus 3.5% (23 of 654) after excluding those with positive C. trachomatis or N. gonorrhoeae results. In all cases an original positive result for C. trachomatis and/or N. gonorrhoeae persisted in the dilution at which the internal control converted; two new positive results for C. trachomatis and none for N. gonorrhoeae evolved at that dilution.

Of 69 starts of the CA system in this study, 10 were invalid assays, all because of negative results on the positive control for N. gonorrhoeae. Invalid assays all occurred in clusters of two or three and were resolved in a single restart using freshly prepared N. gonorrhoeae positive control samples. Thus, the cost of each of these assays was doubled.

Reproducibility testing results are shown in Table 3. Coefficients of variation for the negative C. trachomatis samples were somewhat high, at 27 to 29%, but 100% of these replicates were within the expected range. Results of our labor requirement analysis are shown in Table 4 and include all tasks from specimen receipt to delivery of results. As expected, specimen preparation, the nonautomated portion of the CA procedure, accounts for the majority of the labor requirement. We calculated the direct costs of CA for each C. trachomatis and N. gonorrhoeae result to be $9.09, including labor ($2.00 at $16/h), reagents ($6.69), and equipment ($0.40). The cost of each result from C. trachomatis culture has been previously calculated to be $18.88 (8).

TABLE 3.

Results of reproducibility testing

Organism and result of original test	No. of replicates assayed	Replicates in expected range (%)	Mean absorbance	SD		Coefficient of variation (%)
				Within run	Between runs or days	Within run	Between runs or days
C. trachomatis
Positive	18	100	3.834	0.087	0.123	2.29	3.21
Negative	18	100	0.017	0.005	0.005	27.16	29.56
N. gonorrhoeae
Positive	18	100	2.782	0.450	0.620	16.24	22.28
Negative	18	94.4	0.015	0.002	0.002	13.26	14.77

TABLE 4.

Hands-on labor requirement to produce C. trachomatis and N. gonorrhoeae results from 20 clinical specimens in the CA system

Task	Time (h)
Specimen receipt and log-in	0.5
Preparation of amplification reagents, control samples, and
patient specimens	2.5
CA daily maintenance, loading, and order entry	0.6
Detection reagent preparation	0.4
Result management: verification and report entry	0.8
Area cleanup	0.2
Totala	5.0

^aLabor per clinical result (C. trachomatis [n = 20] and N. gonorrhoeae [n = 20]) (5.0 h/40 results), 0.125 h. Labor per result (C. trachomatis [n = 20], N. gonorrhoeae [n = 20], positive and negative controls [n = 2], and internal controls [n = 20]) (5.0 h/62 results), 0.08 h. 

DISCUSSION

In a prototype of the CA system, Crotchfelt et al. (4) studied 192 endocervical swab specimens transported in 2SP medium, using matched urine samples and a culture technique less sensitive than that in this study (9) as comparators and repeat testing, PCR using other primers, and direct fluorescent antibody staining for discrepancy resolution. Interestingly, an absorbance of ≥0.8 was considered a positive result for both C. trachomatis and N. gonorrhoeae. These investigators found the sensitivity and specificity of the prototypic system to be 100 and 100% for C. trachomatis and 100 and 99.4% for N. gonorrhoeae, respectively. The sensitivities of culture for C. trachomatis and N. gonorrhoeae in that study were 55.9 and 65.2%, respectively.

Jungkind et al. (6) studied the fully automated CA system as we did, though Amplicor transport medium was used. This medium is apparently inferior to 2SP medium (14; B. Van Der Pol, J. A. Williams, and R. B. Jones, Abstr. 95th Gen. Meet. Am. Soc. Microbiol. 1995, abstr. C-489, p. 85, 1995), and it troubled these authors with precipitation and consequent false-positive results. Ultimately they reported on 199 endocervical specimens, comparing automated CA results to those produced by the nonautomated system described above. An absorbance of >0.250 was considered to be a positive result in both systems for both organisms. Culture was also performed for both organisms, but these methods were not described. The two PCR-based systems demonstrated 100% concordance in 20 positive and 179 negative C. trachomatis tests (one originally negative specimen with a negative internal control became positive after dilution to produce a positive internal control) and in 4 positive and 195 negative N. gonorrhoeae tests. The PCR-based tests identified one culture-negative specimen as positive for each organism; resolution of these two samples was not undertaken. These authors also studied the risk of carryover contamination in the automated CA system and found none. They reported hands-on technologist time according to the College of American Pathologists workload recording method of 3.3 min per sample to produce C. trachomatis, N. gonorrhoeae, and internal control results, and they reported that this time was largely independent of the individual's experience when that of a few weeks with that of more than 1 year were compared. Optimal work flow was described, and availability of results in 5.5 h was reported.

Dubuis et al. (5) compared the fully automated CA system to the partially automated LCx system, prospective chlamydia culture without passage, and retrospective assessment of N. gonorrhoeae culture results. Absorbance deemed positive was identical to ours, and 2SP transport medium similar to ours was used. In 410 cervical specimens, these authors found prevalence of C. trachomatis at 2.7%, of N. gonorrhoeae at 0.2%, and of DNA amplification inhibition in the CA system at 5.1%. For CA, they report sensitivities, specificities, and positive-predictive values of 100, 99.8, and 92.9% for C. trachomatis and 100, 98.9, and 33.3% for N. gonorrhoeae, respectively. For the LCx, 100% sensitivity and specificity were reported for both organisms. Only N. gonorrhoeae specimens for which a culture result was available (n = 361) were included in these calculations. The false-positive N. gonorrhoeae results from CA in that study were felt to result from amplification of nonpathogenic Neisseria spp. contaminating some specimens.

Another study (11) compared only chlamydia results for 98 women with a prevalence of infection of 15.3% and found 100% concordance of results from CA, LCx, and the MWA test used in our study. Interestingly, the internal control of the CA system detected 20 specimens with inhibition of amplification in this sample, though none were C. trachomatis positive. Toye et al. (12), using the CA system, found inhibitors in 7% of 906 endocervical swab specimens. Van Der Pol et al. (13) tested 2,236 endocervical swab specimens for C. trachomatis in this system and reported sensitivity after discrepancy resolution of 89.7% and inhibitors in 2.4%. Young et al. (15), who also tested only for C. trachomatis in 286 endocervical samples, describe sensitivity, specificity, and positive and negative predictive values for CA of 96, 100, 100, and 99.1%, respectively.

Comparison of our results to those of previous studies is difficult because of differences in transport medium, prevalence of infection, absorbance values deemed to represent positive results, and comparators. Nonetheless, our results are similar to those found in these studies. We and others did not encounter the difficulty with false-positive N. gonorrhoeae results in CA reported by Dubuis et al. (5), a realistic concern because the primer for this organism may also amplify DNA of certain nonpathogenic Neisseria spp. which may colonize the oropharynx of humans (Neisseria cinerea and Neisseria subflava). We had only two specimens which were repeatedly N. gonorrhoeae positive by CA and negative by culture and duplicate LCx testing. According to the manufacturer (personal communication), specimen contamination by these species in the laboratory is unlikely, and a more likely source is the vaginal flora of the patient. We have no clear understanding of the cause for our negative N. gonorrhoeae positive-control samples and have reported these to the manufacturer. In the 13 months since completion of this study we have not seen a recurrence of N. gonorrhoeae positive-control failure. Appropriate challenges of our methods might include the use of an iodine stain in our C. trachomatis culture technique, which may be less sensitive than fluorescent antibody staining (10). Iodine has been defended previously (8) because of its provision for examination of inclusion morphology, which is still believed to enhance specificity. Also, we used only one method (DNA amplification) besides culture to establish true results because of the disappointing sensitivity of other methods available to us. We did not test for the adequacy of nucleic acid extraction from specimens. The performance of Amplicor transport medium in the MWA system was quite comparable to that of 2SP medium in the CA system.

The CA system is an accurate and convenient method for laboratory diagnosis of endocervical C. trachomatis and N. gonorrhoeae infections from a single sample. It offers elimination of concerns about false-positive results from specimen contamination by amplified DNA in the laboratory and false-negative results from inhibitors of DNA amplification in the specimen, a decreased labor input through full automation of the procedure following specimen preparation, rapid production of results, preservation of residual sample for additional testing, use of a transport medium also amenable to culture for C. trachomatis and assessment of specimen adequacy, and a port for routing results directly into a laboratory information system. Further assessment of the significance of the N. gonorrhoeae primer cross-reactivity and source(s) of false-negative results should enhance the value of the CA system.