Abstract
The next-generation amplification test for Chlamydia trachomatis and Neisseria gonorrhoeae (Roche cobas 4800), a fully automated system, was compared head to head, using female samples, to Gen-Probe Aptima Combo 2 and BD ProbeTec using Viper. Endocervical swabs, female urine, and endocervical samples in liquid-based cytology medium were run on at least two of three platforms. A total of 4,316 samples were evaluated, and 281 chlamydial and 69 gonococcal infections were identified. Estimates of sensitivity and specificity were obtained relative to the patient infection standard (PIS) and using latent class analysis (LCA). Chlamydia sensitivity estimates ranged from 86.9 to 95.6% using PIS and 97.6 to 98% using LCA. Specificity was ≥99.6% for all sample types. Sensitivity ranged from 95.6 to 100% using PIS and 96.9 to 100% using LCA for the detection of gonococcal infections. Specificity for gonococcal infections was ≥99.8%. cobas 4800 performance was equivalent to the comparator assays (all P values, >0.05), and the fully automated system provides high laboratory efficiency.
INTRODUCTION
Chlamydia trachomatis and Neisseria gonorrhoeae remain highly prevalent sexually transmitted infections (STI), the control of which is a high priority for the Centers for Disease Control and Prevention (CDC) (5). The CDC STD Laboratory Diagnosis Guidelines recommend screening using highly sensitive and specific nucleic acid amplification assays (NAATs) (4), confirming that this class of testing is now considered the standard for the diagnosis of these infections. NAATs have been commercially available for the last 2 decades, and evidence of their performance characteristics is widely available (6, 7, 10–17). As chlamydia and gonorrhea screening programs continue to expand to meet the health care coverage benchmarks described by the U.S. Preventive Task Force (USPTF), the CDC (5), and the Healthcare Effectiveness Data and Information Set (HEDIS), clinical diagnostic laboratories will experience increased volumes. In anticipation of this trend, diagnostic manufacturers have begun to develop next-generation, fully automated platforms for C. trachomatis/N. gonorrhoeae assays to increase specimen throughput and meet testing demands. The evaluation of these new assays provides critical information to laboratories considering moving into this field of diagnostics or considering an evidence-based platform change.
The cobas CT/NG test (c4800; Roche Molecular Systems, Pleasanton, CA) is a new C. trachomatis/N. gonorrhoeae assay that is designed for use in clinical laboratories. We compared the performance of this next-generation assay to two commercially available assays, Aptima Combo AC2 (AC2; Gen-Probe, San Diego, CA) and BD Viper ProbeTec CT/GC Qx amplified DNA assay (CTQ/GCQ; BD Diagnostics, Sparks, MD), using the sample types routinely collected from women in a variety of screening settings. Analyses were performed using multiple statistical methods to provide robust estimates of performance for each of the three assays.
(These data were presented in part at the following professional meetings: 2011 European Congress for Clinical Microbiology and Infectious Diseases Annual Meeting, Milan, Italy, 2011 American Society for Microbiology Annual Meeting, New Orleans, LA, and 2011 International Society for STD Research, Biennial Meeting, Quebec City, Canada.)
MATERIALS AND METHODS
The female screening trial for C. trachomatis/N. gonorrhoeae was a multicenter evaluation of the cobas specimen collection kit and the c4800 performed on the cobas 4800 system. Two FDA-cleared NAATs, AC2 and CTQ/GCQ, were used as comparator assays. The specimen collection sites were geographically diverse and included obstetrics-gynecology practices, family planning clinics, and STD clinics. Four sites performed all testing on the cobas 4800 system. Results obtained for the female study population are described here; results obtained for the male population are described elsewhere (14a).
Patient population.
Inclusion criteria included being at least14 years of age and being eligible for routine C. trachomatis/N. gonorrhoeae screening according to the routine practices at each enrollment site. Individuals were excluded from enrollment if any of the following criteria were met: (i) previously enrolled in the study, (ii) use of antimicrobial agents active against C. trachomatis or N. gonorrhoeae during the preceding 21 days, (iii) use of Replens, a vaginal lubricant (Lil'Drug Store Products, Inc., Cedar Rapids, IA), within the previous 3 days, (iv) history of hysterectomy, or (v) contraindication to Pap test/cervical sampling. Participants were classified as symptomatic if they reported dysuria/pain during urination, coital pain/difficulty/bleeding/discharge, pelvic pain, abnormal vaginal discharge, or pelvic/uterine/ovarian pain. All other participants were classified as asymptomatic.
Specimen collection.
From each female participant, specimens were collected in the following order: first-catch urine; a single vaginal swab (unpublished data); 3 endocervical swabs using each manufacturer's sample collection device (in randomized order); and a sample suitable for liquid-based cytology (LBC) placed into PreservCyt medium (Cytyc Corp., Marlborough, MS). In cases where the LBC specimen was requested for Pap testing for routine patient care, the LBC specimen collection was performed prior to any other swab sampling. All urine specimens were divided into three aliquots and placed into each assay's transport tube. LBC samples were aliquoted into c4800 sample tubes and AC2 transport tubes prior to processing for Pap testing (prequot LBC). Following Pap processing in the laboratory, residual specimens were aliquoted into c4800 sample tubes (postquot LBC). These specimens were handled by the cytology staff as routine specimens, and no extra procedures were implemented to attempt to reduce cross-contamination. All specimens for comparator assays were stored and tested according to each manufacturer's package insert instructions.
cobas CT/NG test.
The c4800 uses a dual-target approach. C. trachomatis primers CP102 and CP103 are used to target a sequence of approximately 206 nucleotides within the cryptic plasmid DNA of C. trachomatis. In addition, C. trachomatis primers CTMP101 and CTMP102 target a sequence of approximately 182 nucleotides within the chromosomal DNA of C. trachomatis. N. gonorrhoeae primers NG514 and NG519 target a sequence of approximately 190 nucleotides from a highly conserved direct repeat region of N. gonorrhoeae called DR-9. In addition, another set of N. gonorrhoeae primers, NG552 and NG579, targets a second sequence of approximately 215 nucleotides identified as a conserved sequence variant from this region. DNA extracted from samples using magnetic bead purification is added to the amplification mixture in a multiwell plate, in which PCR amplification occurs fully automated. The C. trachomatis/N. gonorrhoeae internal control (IC) is a combination of two noninfectious recombinant plasmid DNAs, each with primer binding regions that are identical to those of either the C. trachomatis or the N. gonorrhoeae genomic target sequences. The recombinant plasmid DNAs have identical randomized internal target sequences and unique probe binding regions that differentiates the IC from target amplicons to ensure the independent amplification of the IC and the C. trachomatis and N. gonorrhoeae target DNAs. The IC is included in the c4800 and is introduced into each sample on the cobas 4800 system during sample processing prior to lysis incubation. The selective amplification of target nucleic acids from the specimen is achieved in the c4800 by the use of AmpErase (uracil-N-glycosylase) enzyme and dUTP. The c4800 utilizes real-time PCR technology. The use of dual TaqMan probes labeled with 6-carboxyfluorescein (FAM) and black hole quencher (BHQ) for the detection of C. trachomatis and hexachlorofluorescein (HEX) and BHQ for the detection of N. gonorrhoeae provides for real-time detection of PCR product accumulation by monitoring the emission intensity of fluorescent dyes released during the amplification process. The amplification of C. trachomatis targets, N. gonorrhoeae targets, and the IC are measured independently and at different wavelengths. This process is repeated for a designated number of cycles, with each cycle increasing the emission intensity of the individual reporter dyes.
Patient infection status for C. trachomatis and N. gonorrhoeae.
Patient infection status was defined as being infected with C. trachomatis or N. gonorrhoeae if at least 2 NAATs with different target regions gave positive results in the endocervical swab and/or the urine specimen. LBC results were not included in the calculation of the patient infection standard (PIS), since only one comparator test result was available. LBC results were compared to the PIS that was calculated based on urine and endocervical swab results from all three assays. For example, when LBC performance on the c4800 was assessed, results were compared to the other two NAAT results from urine and endocervical swabs. Additionally, localized infection could occur (e.g., a woman may have had multiple positive urine results and all negative endocervical swab results). In these cases, for analyses of urine performance the participant would have been classified as infected, while in analyses of endocervical sample performance, including LBC, the participant would have been classified as uninfected. The reverse (all endocervical samples positive and all urine samples negative) was possible as well. Thus, each NAAT was evaluated using a PIS constructed based on results obtained with the two other NAATs used in the study (rotating PIS). It is worth noting that this differs from a strictly head-to-head comparison, since a single positive sample by any comparator assay would be sufficient to categorize a participant as infected at that body site, assuming there were other body sites positive by the other comparator. As an example, if a woman had a positive endocervical sample only by AC2 and a positive urine sample only by CTQ, according to the PIS this participant would have been categorized as infected in both the endocervical and urine analyses. Thus, the PIS maximized the number of infections identified while taking into account the possibility of site-specific infections.
Statistical analysis.
The statistical analyses were chosen based on recommendations in the statistical guidance from the FDA (9) and in accordance with the guidelines published by the Clinical and Laboratory Standards Institute (8) for evaluating qualitative test performance. The sensitivity and specificity of c4800 were calculated separately for the detection of C. trachomatis and N. gonorrhoeae by using the rotating PIS as the reference standard. The corresponding 2-sided 95% score (Wilson) confidence intervals (CIs) were also estimated. Venn diagrams were plotted separately by sample type to analyze the list of matched clinical specimens (i.e., same sample type for a given subject) that tested positive from at least 1 NAAT. Fisher exact test was used to assess the statistically significant difference in performance estimates between symptomatic and asymptomatic groups. Latent class analysis (LCA) was also used to estimate the sensitivity and specificity of each NAAT (1). LCA identifies underlying characteristics, in this case infected and uninfected status, as classes, and may be less biased than the PIS, which is strongly influenced by the test with the poorest performance. To characterize the predictive values of positive results obtained with the c4800 by varying the prevalence of C. trachomatis and N. gonorrhoeae, the hypothetical estimates of positive predictive value (PPV) were calculated. All analyses were performed using SAS/STAT software (SAS, Cary, NC) with α set to 0.05.
RESULTS
A total of 4,479 women enrolled in the study. Of these, 17 were excluded because they did not meet the inclusion criteria or did not provide appropriate consent; 146 were considered nonevaluable because of errors in specimen collection, transport, and storage (unpublished data). A total of 4,316 (96.4%) female subjects were evaluable for C. trachomatis and/or N. gonorrhoeae analyses. Patient characteristics are shown in Table 1. Among evaluable subjects, 355 and 351 specimens were classified as nonevaluable for the C. trachomatis and N. gonorrhoeae primary analyses, respectively, for a particular specimen type either because a specimen was not available for testing or because a specimen was repeatedly inhibitory (IC failure). Invalid results due to IC failure were observed in 1.12 and 0.02% of female urine and postquot LBC samples, respectively. There were no IC failures for the remaining sample types. These rates of inhibition are quite low and in the range of or below the rates reported for some assays. The AC2 system does not utilize an assay control, therefore there are no data available on the frequency of the occurrence of inhibition. AC2 failure rates ranged between 0.28 and 0.87% across all specimen types. Failure rates for the CTQ/GCQ assay ranged between 0.12 and 0.65% across all specimen types.
Table 1.
Patient characteristics and disease prevalence
| Characteristicb | No. (%) of participants | % Prevalencea (% CI) |
|
|---|---|---|---|
| C. trachomatis | N. gonorrhoeae | ||
| Race | |||
| Black | 1,860 (43.1) | 9.4 (8.1, 10.8) | 2.9 (2.2, 3.7) |
| White | 2,089 (48.4) | 3.4 (2.7, 4.2) | 0.7 (0.4, 1.1) |
| Asian/Pacific islander | 122 (2.8) | 9.8 (5.7, 16.4) | 0.8 (0.1, 4.5) |
| Other | 245 (5.7) | 6.5 (4.1, 10.3) | 0.4 (0.1, 2.3) |
| Hispanic | 955 (22.1) | 4.7 (3.5, 6.3) | 0.4 (0.2, 1.1) |
| Symptomaticb | |||
| Yes | 2,024 (46.9) | 8.0 (6.9, 9.3) | 2.3 (1.7, 3.0) |
| No | 2,292 (53.1) | 4.8 (4.0, 5.8) | 1.0 (0.7, 1.5) |
| Clinic typeb | |||
| Family planning | 1,762 (40.8) | 7.5 (6.4, 8.8) | 1.6 (1.1, 2.3) |
| Obstetrics-gynecology | 1,079 (25.0) | 3.0 (2.1, 4.2) | 0.1 (0.0, 0.5) |
| STD | 1,475 (34.2) | 7.3 (6.1, 8.8) | 2.7 (2.0, 3.7) |
CI, confidence interval.
P < 0.01 by Fisher exact significance test. Median age, 25 years.
Start-up and daily maintenance time for the cobas 4800 was less than 30 min, and total hands-on time for testing from start to finish was less than 40 min for 96 samples. The time to results was slightly less than 4 h. Three runs of 96 samples could be performed in a single 9-h shift, with the results for the final set of 96 specimens available the next day.
C. trachomatis.
A total of 281 (6.5%) females were infected with C. trachomatis based on the PIS. Symptoms were reported by 59.4% (167/281) of infected women. The sensitivity and specificity of the c4800, AC2, and CTQ/GCQ assays for C. trachomatis in females using the rotating PIS are presented by sample type and symptom status in Table 2. The sensitivity of c4800 ranged from 86.9 to 95.6% and was similar by symptom status (all P values, >0.05). Regardless of symptom status, specificity for C. trachomatis was high, ranging from 99.6 to 100.0% across all sample types. Specificity did not differ significantly between the three molecular assays (all P values, >0.05), with the exception of a significantly higher specificity of the c4800 and QCT compared to that of AC2 for endocervical swabs from asymptomatic and symptomatic individuals (overall P, 0.0005).
Table 2.
Clinical performance for C. trachomatis detection by sample type and symptom status compared to patient infection statusa
| Sample type and symptom status | % Sensitivity (no. positive/total no.) (95% CI) of: |
% Specificity (no. positive/total no.) (95% CI) of: |
||||
|---|---|---|---|---|---|---|
| c4800 | AC2 | CTQ | c4800 | AC2 | CTQ | |
| Endocervical swab | ||||||
| Symptomatic | 93.0 (146/157) (87.9, 96.0) | 96.2 (153/159) (92.0, 98.3) | 94.4 (153/162) (89.8, 97.0) | 99.7 (1,821/1,827)* (99.3, 99.8) | 98.9 (1,843/1,863) (98.3, 99.3) | 99.6 (1,849/1,856)* (99.2, 99.8) |
| Asymptomatic | 89.5 (94/105) (82.2, 94.0) | 97.1 (101/104) (91.9, 99.0) | 96.2 (102/106) (90.7, 98.5) | 100.0 (2,163/2,164)* (99.7, 100.0) | 99.5 (2,173/2,185) (99.0, 99.7) | 99.7 (2,155/2,162) (99.3, 99.8) |
| Overall | 91.6 (240/262) (87.6, 94.4) | 96.6 (254/263) (93.6, 98.2) | 95.1 (255/268) (91.9, 97.1) | 99.8 (3,984/3,991)* (99.6, 99.9) | 99.2 (4,016/4,048) (98.9, 99.4) | 99.7 (4,004/4,018)* (99.4, 99.8) |
| Urine | ||||||
| Symptomatic | 94.4 (153/162) (89.8, 97.0) | 98.1 (152/155) (94.5, 99.3) | 93.8 (152/162) (89.0, 96.6) | 99.7 (1,832/1,838) (99.3, 99.9) | 99.2 (1,848/1,862) (98.7, 99.6) | 99.8 (1,854/1,857) (99.5, 99.9) |
| Asymptomatic | 89.1 (98/110) (81.9, 93.6) | 92.5 (98/106) (85.8, 96.1) | 96.2 (101/105) (90.6, 98.5) | 99.8 (2,165/2,169) (99.5, 99.9) | 99.8 (2,181/2,186) (99.5, 99.9) | 99.7 (2,161/2,167) (99.4, 99.9) |
| Overall | 92.3 (251/272) (88.5, 94.9) | 95.8 (250/261) (92.6, 97.6) | 94.8 (253/267) (91.4, 96.9) | 99.8 (3,997/4,007) (99.5, 99.9) | 99.5 (4,029/4,048) (99.3, 99.7) | 99.8 (4,015/4,024) (99.6, 99.9) |
| PreservCyt (prequot) | ||||||
| Symptomatic | 95.6 (151/158) (91.1, 97.8) | 96.8 (152/157) (92.8, 98.6) | 99.7 (1,826/1,832) (99.3, 99.8) | 99.6 (1,838/1,846) (99.1, 99.8) | ||
| Asymptomatic | 88.8 (95/107) (81.4, 93.5) | 96.0 (97/101) (90.3, 98.4) | 99.6 (2,132/2,141) (99.2, 99.8) | 99.5 (2,148/2,159) (99.1, 99.7) | ||
| Overall | 92.8 (246/265) (89.1, 95.4) | 96.5 (249/258) (93.5, 98.2) | 99.6 (3,958/3,973) (99.4, 99.8) | 99.5 (3,986/4,005) (99.3, 99.7) | ||
| PreservCyt (postquot) | ||||||
| Symptomatic | 91.6 (142/155) (86.2, 95.0) | 99.7 (1,806/1,811) (99.4, 99.9) | ||||
| Asymptomatic | 86.9 (93/107) (79.2, 92.0) | 99.8 (2,124/2,129) (99.5, 99.9) | ||||
| Overall | 89.7 (235/262) (85.4, 92.8) | 99.7 (3,930/3,940) (99.5, 99.9) | ||||
Infection status was constructed based on results obtained with the two other NAATs (rotating PIS using all endocervical swab and urine results; LBC results were not used to calculate the PIS; see Materials and Methods). Asterisks indicate pairwise comparisons that were statistically significant (P < .05); in all cases, specificity was higher than estimates for AC2.
Venn diagrams comparing all positive results, regardless of the final definition of infection status, for C. trachomatis across the three assays in endocervical swabs, liquid-based cytology, and urine specimens obtained from female subjects are shown in Fig. 1.
Fig 1.
Venn diagrams comparing C. trachomatis-positive results across three assays of endocervical swabs (A), liquid-based cytology (B), and urine specimens (C) obtained from female subjects. A total of 4,223 (endocervical swabs), 4,186 (liquid based cytology), and 4,254 (urine specimens) subjects had valid results from c4800, AC2, and CT/GCQ assays.
LCA estimated the sensitivity of c4800, AC2, and CTQ/GCQ to be 95.9, 98.3, and 98.8%, respectively, for the detection of C. trachomatis in endocervical swabs. For urine samples, the LCA estimates of sensitivity were 97.6, 98.4, and 98.0%. LCA estimated the specificity of all three assays to be ≥99.3% regardless of sample type.
N. gonorrhoeae.
A total of 69 (1.6%) women were infected with N. gonorrhoeae, and symptoms were reported in 66.7% (46/69) of these women. The sensitivity and specificity of c4800, AC2, and CTQ/GCQ assays for N. gonorrhoeae in females using the rotating PIS are presented by sample type and symptom status in Table 3. Sensitivity in this analysis ranged from 95.6 to 100.0%. Overall specificity was high, ranging from 99.8 to 100.0% across all sample types. Performance characteristics did not differ significantly between the three molecular assays based on symptom status, order of sampling, clinic type, and PaP smear collection device (all P values, >0.05) with the exception of a significantly higher specificity of the c4800 and AC2 compared to CTQ/GCQ in endocervical swabs (overall P, 0.021).
Table 3.
Clinical performance for N. gonorrhoeae detection by sample type and symptom status compared to patient infection statusa
| Sample type and symptom status | % Sensitivity (no. positive/total no.) (95% CI) of: |
% Specificity (no. positive/total no.) (95% CI) of: |
||||
|---|---|---|---|---|---|---|
| c4800 | AC2 | GCQ | c4800 | AC2 | GCQ | |
| Endocervical Swab | ||||||
| Symptomatic | 95.6 (43/45) (85.2, 98.8) | 100.0 (46/46) (92.3, 100.0) | 95.7 (45/47) (85.8, 98.8) | 99.9 (1,936/1,938) (99.6, 100.0) | 99.9 (1,973/1,974) (99.7, 100.0) | 99.7 (1,966/1,971) (99.4, 99.9) |
| Asymptomatic | 95.7 (22/23) (79.0, 99.2) | 100.0 (23/23) (85.7, 100.0) | 91.3 (21/23) (73.2, 97.6) | 100.0 (2,246/2,246) (99.8, 100.0) | 100.0 (2,266/2,266) (99.8, 100.0) | 99.8 (2,241/2,245) (99.5, 99.9) |
| Overall | 95.6 (65/68) (87.8, 98.5) | 100.0 (69/69) (94.7, 100.0) | 94.3 (66/70) (86.2, 97.8) | 100.0 (4,182/4,184)* (99.8, 100.0) | 100.0 (4,239/4,240)* (99.9, 100.0) | 99.8 (4,207/4,216) (99.6, 99.9) |
| Urine | ||||||
| Symptomatic | 97.6 (41/42) (87.7, 99.6) | 97.6 (40/41) (87.4, 99.6) | 95.3 (41/43) (84.5, 98.7) | 99.9 (1,955/1,957) (99.6, 100.0) | 99.9 (1,977/1,979) (99.6, 100.0) | 100.0 (1,977/1,977) (99.8, 100.0) |
| Asymptomatic | 100.0 (23/23) (85.7, 100.0) | 95.7 (22/23) (79.0, 99.2) | 100.0 (23/23) (85.7, 100.0) | 100.0 (2,255/2,256) (99.7, 100.0) | 100.0 (2,268/2,269) (99.8, 100.0) | 99.9 (2,246/2,249) (99.6, 100.0) |
| Overall | 98.5 (64/65) (91.8, 99.7) | 96.9 (62/64) (89.3, 99.1) | 97.0 (64/66) (89.6, 99.2) | 99.9 (4,210/4,213) (99.8, 100.0) | 99.9 (4,245/4,248) (99.8, 100.0) | 99.9 (4,223/4,226) (99.8, 100.0) |
| PreservCyt (prequot) | ||||||
| Symptomatic | 97.8 (45/46) (88.7, 99.6) | 95.7 (44/46) (85.5, 98.8) | 99.8 (1,942/1,945) (99.5, 99.9) | 99.9 (1,955/1,957) (99.6, 100.0) | ||
| Asymptomatic | 95.7 (22/23) (79.0, 99.2) | 100.0 (23/23) (85.7, 100.0) | 100.0 (2,225/2,225) (99.8, 100.0) | 99.9 (2,236/2,238) (99.7, 100.0) | ||
| Overall | 97.1 (67/69) (90.0, 99.2) | 97.1 (67/69) (90.0, 99.2) | 99.9 (4,167/4,170) (99.8, 100.0) | 99.9 (4,191/4,195) (99.8, 100.0) | ||
| PreservCyt (Postquot) | ||||||
| Symptomatic | 95.7 (44/46) (85.5, 98.8) | 99.9 (1,920/1,921) (99.7, 100.0) | ||||
| Asymptomatic | 95.7 (22/23) (79.0, 99.2) | 100.0 (2,212/2,213) (99.7, 100.0) | ||||
| Overall | 95.7 (66/69) (88.0, 98.5) | 100.0 (4,132/4,134) (99.8, 100.0) | ||||
The patient infection status was constructed based on results obtained in the two other NAATs (rotating PIS using all endocervical swab and urine results; LBC results were not used to calculate the PIS; see Materials and Methods). Asterisks indicate pairwise comparisons that were statistically significant (P < .05); in both cases, estimates were higher than those for GCQ.
Venn diagrams comparing positive results for N. gonorrhoeae across the three assays in endocervical swabs, liquid-based cytology, and urine specimens are shown in Fig. 2. LCA estimates of sensitivity were 98.5, 100, and 97.0% for the c4800, AC2, and CTQ/GCQ assays, respectively, for endocervical samples. Specificity was estimated to be ≥99.7% for all assays. For urine samples, the LCA estimate of sensitivity for N. gonorrhoeae was 100, 96.9, and 100% for c4800, AC2, and CTQ/GCQ, respectively, with all specificity estimates being >99.9%.
Fig 2.
Venn diagrams comparing N. gonorrhoeae-positive results across three assays of endocervical swabs (A), liquid-based cytology (B), and urine specimens (C) obtained from female subjects. A total of 4,221 (endocervical swabs), 4,187 (liquid-based cytology), and 4,255 (urine specimens) subjects had valid results from c4800, AC2, and CT/GCQ assays.
Positive predictive values.
The hypothetical PPV of the cobas 4800 using genital samples for the detection of C. trachomatis and N. gonorrhoeae were calculated at prevalence rates ranging from 1 to 50% and are presented in Fig. 3. For C. trachomatis, the hypothetical PPV ranged from 77.3 to 99.7% at prevalence rates ranging from 1 to 50%. The corresponding ranges for N. gonorrhoeae were from 93.8 to 99.9%.
Fig 3.
Hypothetical PPV based on changing population prevalence for C. trachomatis (A) and N. gonorrhoeae (B) results.
DISCUSSION
c4800 performance was equivalent to other currently available FDA-cleared assays for the detection of chlamydia and gonorrhea infections in women. While the estimates of sensitivity for C. trachomatis were consistently, but not significantly, lower than the estimates obtained for AC2 for the PIS, the LCA showed high levels of similarity in performance. This is consistent with the broad confidence intervals for all estimates. Additionally, the specificity of the assay was consistently high for the c4800 assay. For C. trachomatis, the specificity of the c4800 and QCT assays with endocervical swabs was higher than that of AC2, while for GC the QGC assay showed lower specificity. These statistically significant differences are likely the result of very large sample sizes of negative test results and are not clinically meaningful.
Performance was not affected by the presence or absence of symptoms, making this a useful assay for both screening and diagnosis consistently with the CDC recommendations. Further, the system is suitable for use in a routine clinical laboratory because of the limited hands-on requirements, relatively rapid time to results, and throughput of approximately 388 samples per 9-h shift. Additionally, the cobas 4800 system can perform high-risk human papillomavirus testing (3), thus expanding the menu of the platform and increasing utility in the laboratory.
The use of dual targets for C. trachomatis ensures that the assay can detect variant chlamydial strains, such as those described in Sweden, that contain a mutation in the cryptic plasmid (2). The c4800 assay has been evaluated against a large number of clinical isolates, including new variant strains, and it performs well. Including dual targets for N. gonorrhoeae has also become the standard in next-generation NAATs. The N. gonorrhoeae specificity for this assay was ≥99.8%.
Since few laboratories have access to multiple platforms, head-to-head comparison data may be difficult to obtain. Thus, reports of these findings are particularly relevant to today's laboratory management needs. Head-to-head analysis of the positive results, shown in Fig. 1 and 2, provides an unbiased observation of the performance of all three assays without assigning true- and false-positive status. Furthermore, we applied LCA as a statistical technique for determining two features of a data set: first, the number of classes which account for the values in the data, and second, the probability that a result maps to one of those classes. Although a PIS attempts to classify individuals as infected or uninfected, if a single assay has substantially higher sensitivity, this could be misrepresented as lower specificity, since no other assay will have matching positive results for a small number of cases. LCA does not arbitrarily assign a true or false infection status to results based on arbitrary predefined infection standards in the manner of a PIS, therefore it is considered to provide a less biased estimate of the performance of new diagnostic assays. The LCA is not a rolling patient standard, in that it is performed only for those samples for which head-to-head results are available. It moves beyond a head-to-head analysis, such as a chi-square test, since LCA determines the number of classes described by the data. For the data generated by this study, LCA determined that the optimal number of classes was two (infected and uninfected) and estimated high sensitivity and specificity for all three assays for both pathogens. Had there been three classes identified by analysis, this would have been an indication of poor discriminatory ability, but such was not the case with these data. LCA is not designed for hypothesis testing and therefore cannot estimate differences in performance among the three methods, if any exist. Thus, the use of the PIS for comparison purposes is warranted. The use of multiple analytic approaches is a strength of the data presented here.
A limitation of the study was the relatively high prevalence of chlamydial infection in all study populations compared to the general population. We provided the estimated PPV of the c4800 tests in hypothetical populations with various rates of prevalence to provide a robust approximation of the performance in lower prevalence settings. The prevalence of gonococcal infection was lower and therefore was not affected by this limitation.
The prequot and postquot LBC performance was equivalent, thus allowing this sample type to be used in any setting that handles LBC, whether the samples require preparation for Pap testing before other testing or not. Appropriate caution in handling LBC samples prior to preparation for cytologic exam must be exercised to maintain the integrity of the specimens.
In conclusion, in both symptomatic and asymptomatic patients, the cobas 4800 CT/NG test offers high sensitivity, specificity, and positive predictive values for the direct, qualitative detection of C. trachomatis and N. gonorrhoeae using endocervical swabs, urine specimens, and liquid-based cytology specimens collected in PreservCyt solution.
ACKNOWLEDGMENTS
We are grateful to Rui Li for expert statistical support and to the members of the collection and testing sites.
This study was funded by Roche Molecular Systems, Pleasanton, CA. B. Van Der Pol discloses consulting honoraria or research funding received from Abbott Molecular, BD Diagnostics, and Roche Molecular Systems. E. W. Hook III has received research support from Roche Molecular Systems, BD Diagnostics, Gen-Probe, Siemens, and Cepheid.
Footnotes
Published ahead of print 18 April 2012
REFERENCES
- 1. Baughman AL, et al. 2008. Utility of composite reference standards and latent class analysis in evaluating the clinical accuracy of diagnostic tests for pertussis. Clin. Vaccine Immunol. 15:106–114 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Bjorkman J, Jonsson L, Nilsson P. 2007. Prevalence of the new genetic variant of Chlamydia trachomatis in Sodra Alvsborg County, Vastra Gotaland Region, Sweden. Euro Surveill. 12:E070614.4. [DOI] [PubMed] [Google Scholar]
- 3. Castle P, et al. 2011. Performance of carcinogenic human papillomavirus (HPV) testing and HPV16 or HPV18 genotyping for cervical cancer screening of women aged 25 years and older: a subanalysis of the Athena study. Lancet Oncol. 12:880–890 [DOI] [PubMed] [Google Scholar]
- 4. Centers for Disease Control and Prevention 2010. Laboratory diagnostic testing for Chlamydia trachomatis and Neisseria gonorrhoeae. American Public Health Laboratories, Centers for Disease Control and Prevention, Atlanta, GA: http://www.aphl.org/aphlprograms/infectious/std/documents/ctgclabguidelinesmeetingreport.pdf [Google Scholar]
- 5. Centers for Disease Control and Prevention 2010. Sexually transmitted diseases treatment guidelines, 2010. Morb. Mortal. Wkly. Rep. 59:1–112 [Google Scholar]
- 6. Cheng A, Qian Q, Kirby JE. 2011. Evaluation of the Abbott realtime CtTNG assay in comparison to the Roche Cobas Amplicor CT/NG assay. J. Clin. Microbiol. 49:1294–1300 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Chernesky M, et al. 2009. Validation of the Aptima Combo 2 assay for the detection of Chlamydia trachomatis and Neisseria gonorrhoeae in Surepath liquid-based Pap test samples taken with different collection devices. Sex. Transm. Dis. 36:581–583 [DOI] [PubMed] [Google Scholar]
- 8. Clinical and Laboratory Standards Institute 2007. User protocol for evaluation of qualitative test performance. Approved guideline EP12-A2. Clinical and Laboratory Standards Institute, Wayne, PA [Google Scholar]
- 9. Gun-Munro J, et al. 1996. Detection of Chlamydia trachomatis in genitourinary tract specimens using an automated enzyme-linked fluorescent immunoassay. Sex. Transm. Dis. 23:115–119 [DOI] [PubMed] [Google Scholar]
- 10. Kerndt PR, et al. 2011. First report of performance of the Versant CT/GC DNA 1.0 assay (Kpcr) for detection of Chlamydia trachomatis and Neisseria gonorrhoeae. J. Clin. Microbiol. 49:1347–1353 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Martin DH, et al. 2000. Multicenter evaluation of Amplicor and automated Cobas Amplicor CT/NG tests for Neisseria gonorrhoeae. J. Clin. Microbiol. 38:3544–3549 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Munson E, et al. 2007. Evaluation of Gen-Probe Aptima-based Neisseria gonorrhoeae and Chlamydia trachomatis confirmatory testing in a metropolitan setting of high disease prevalence. J. Clin. Microbiol. 45:2793–2797 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. O'Mahony C, Reeve-Fowkes A, Worthen E, Mallinson H. 2008. Three years of using Aptima Combo 2 (AC2) transcription-mediated amplification for gonorrhoea in a district hospital genitourinary medicine clinic shows it to be superior to culture and has a specificity of almost 100%. Int. J. STD AIDS 19:67–69 [DOI] [PubMed] [Google Scholar]
- 14. Taylor SN, et al. 2011. Clinical evaluation of the BD ProbeTec Chlamydia trachomatis Qx amplified DNA assay on the BD Viper system with XTR technology. Sex. Transm. Dis. 38:603–609 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14a. Taylor SN, et al. 2012. Evaluation of the Roche cobas CT/NG test for detection of Chlamydia trachomatis and Neisseria gonorrhoeae in male urine. Sex. Transm. Dis. [Epub ahead of print.] 10.1097/OLQ.0b013e31824e26ff [DOI] [PubMed] [Google Scholar]
- 15. Van Der Pol B, et al. 2001. Multicenter evaluation of the BD ProbeTec ET system for detection of Chlamydia trachomatis and Neisseria gonorrhoeae in urine specimens, female endocervical swabs, and male urethral swabs. J. Clin. Microbiol. 39:1008–1016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Van Der Pol B, et al. 2000. Multicenter evaluation of the Amplicor and automated Cobas Amplicor CT/NG tests for detection of Chlamydia trachomatis. J. Clin. Microbiol. 38:1105–1112 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Van Der Pol B, et al. 2002. Evaluation of the Digene Hybrid Capture II Assay with the Rapid Capture System for detection of Chlamydia trachomatis and Neisseria gonorrhoeae. J. Clin. Microbiol. 40:3558–3564 [DOI] [PMC free article] [PubMed] [Google Scholar]