Comparison of the cobas 4800 CT/NG Test with Culture for Detecting Neisseria gonorrhoeae in Genital and Nongenital Specimens in a Low-Prevalence Population in New Zealand

Abstract

To assess the clinical utility of replacing microbial culture for Neisseria gonorrhoeae with a nucleic acid amplification test (NAAT), we compared N. gonorrhoeae culture with the cobas 4800 CT/NG test for 18,247 urogenital and 666 nongenital samples. For urogenital specimens, the sensitivity, specificity, and positive and negative predictive values of the cobas N. gonorrhoeae PCR were 98.7%, 100%, 95.6%, and 100%, respectively, and for nongenital specimens, the values were 100%, 99.8%, 92.9%, and 100%, respectively. In our test population, 37% (10,185) of patients tested over the study period were screened for C. trachomatis by PCR but were not screened for gonorrhea by culture. Of these, 43 were N. gonorrhoeae positive by PCR and therefore went undiagnosed. The cobas 4800 CT/NG test diagnosed 33% (n = 30) more urogenital and 25% (n = 3) more rectal gonorrhea infections than culture and, based on the above performance indicators, does not require supplementary testing for urogenital or rectal specimens. The ability to test noninvasive specimens (such as urine and self-taken vulvovaginal swabs) for N. gonorrhoeae will enable more patients to be screened for infection, thus offering significant positive public health benefits.

INTRODUCTION

Nucleic acid amplification tests (NAATs) have been recommended for the diagnosis of Chlamydia trachomatis in New Zealand since 2008 (1). However, problems with the specificity of particular NAATs for Neisseria gonorrhoeae (2, 3) meant that microbial culture remained the method of choice in New Zealand's low-prevalence population (67 per 100,000) (4). Well-established clinical advantages of NAATs for diagnosing sexually transmitted infections (STIs), including higher sensitivity, detection of nonviable organisms, and testing of noninvasive specimens, are driving a change to molecular testing. Replacement of culture with automated NAATs brings significant workflow benefits, a reduction in reporting time, and the ability to test a single specimen for both C. trachomatis and N. gonorrhoeae. Also, the positive public health impacts of a higher number of cases being diagnosed are important for tackling increasingly difficult-to-treat gonorrhea infections.

Universal screening using NAATs for samples from nongenital sites, particularly those from men who have sex with men, is encouraged, to minimize the morbidity associated with gonorrhea and potentially to enhance control of HIV transmission (5). However, there may be issues with the specificity of NAATs performed on samples from these anatomic sites due to the potential presence of commensal Neisseria species (6, 7).

Since 2009, C. trachomatis testing in our laboratory has been performed on the cobas 4800 using the C. trachomatis/N. gonorrhoeae PCR test (Roche Diagnostics, New Zealand), which simultaneously detects C. trachomatis and N. gonorrhoeae. However, the N. gonorrhoeae test was not initially reported, and culture was maintained for detection of gonorrhea. Recent studies report improved clinical and analytical specificity of the cobas 4800 N. gonorrhoeae assay over previous assays (8, 9), indicating that N. gonorrhoeae confirmatory testing may not be required on some specimens (10). This study compared the cobas 4800 N. gonorrhoeae test to culture, for genital and nongenital specimens, to investigate whether its performance would be acceptable without supplementary confirmatory testing in our low-prevalence population.

MATERIALS AND METHODS

Study design.

We retrospectively compared culture and cobas 4800 N. gonorrhoeae results obtained for patients from whom 2 specimens were collected; a cervical or urethral swab taken by a physician for microbial culture, and a physician-taken cervical, urethral, rectal, or throat swab or a urine sample or self-taken vulvovaginal swab for C. trachomatis PCR between 1 January 2011 and 13 September 2011. N. gonorrhoeae PCR results were obtained from the raw data (extensible markup language [XML] files) generated by the cobas 4800 for all C. trachomatis specimens tested, with the assistance of Roche Diagnostics, New Zealand.

Testing methods.

Amies medium swabs and direct cultures were inoculated onto New York City (NYC) or Thayer Martin agar (Fort Richard Laboratories) and incubated in 5% CO₂ at 37°C for 48 h. Isolates suspected of being N. gonorrhoeae were identified by Gram staining and oxidase testing and confirmed using BactiCard (Remel, Lenexa, KS), and/or MicroTrak Direct fluorescence antibody test (Syva, Palo Alto, CA). Specimens submitted for routine C. trachomatis NAAT analysis were tested on the cobas 4800 as per the manufacturers' protocol (11) with the exception that swab specimens were vortexed for 1 min and the swabs removed prior to testing. Physician-collected nongenital swab specimens were processed in the same manner as genital swabs. The recommended procedure for collection of rectal swabs is via a proctoscope, and for pharyngeal specimens, swabs should be taken from the back of the throat/tonsillar area.

Testing for N. gonorrhoeae was simultaneously performed by the cobas 4800; however, the results for N. gonorrhoeae were not reported clinically during the study period. Any specimen with a nonvalid result (i.e., failure due to a pipetting error or a clot in the specimen or an invalid result due to amplification failure) was treated with Sputasol (1.4% dithiothreitol [DTT]; Oxoid, Basingstoke, United Kingdom) and retested as previously described (12).

Urine specimens were not cultured for N. gonorrhoeae in our laboratory but were tested for C. trachomatis by PCR; therefore, when a urine specimen was sent for C. trachomatis testing, an accompanying cervical or urethral culture swab result was compared (referred to as urogenital specimens) (n = 18,247).

Assay performance was further investigated using 97 local isolates grown from directly inoculated plates (from Wellington sexual health clinics) or from swabs from the rectum (n = 24), throat (n = 32), urethra (n = 26), penis (n = 2), and cervix (n = 13) that were received in Amies transport medium and inoculated onto NYC agar.

Analytical specificity panel.

Analytical specificity was evaluated using 6 gonococcal and 34 commensal or pathogenic bacterial and fungal isolates grown from reconstituted freeze-dried organisms from the Institute of Environmental Science and Research, Limited (ESR), culture collection. Gonococcal isolates included antibiotic-sensitive and -resistant strains: ATCC 43069 (medium quality control strain), ATCC 49226 (antibiotic sensitivity reference strain), AGSP QC 00/3 (ciprofloxacin MIC, 2.0 mg/liter), AGSP QC 36 (penicillin MIC, 2.0 mg/liter), AS 84/417 (NZ plasmid), AS 91/376 (tetracycline MIC > 16 mg/liter; penicillinase producing). N. meningitidis isolates included a representative strain from each of groups A, B, and C. Also included were Acinetobacter calcoaceticus, Candida albicans, Candida glabrata, Citrobacter freundii, Enterococcus faecalis, Enterococcus faecium, Escherichia coli, Fusobacterium necrophorum, Gardnerella vaginalis, Haemophilus influenzae (n = 3), Haemophilus parainfluenzae, Klebsiella pneumoniae, Lactobacillus acidophilus, Micrococcus luteus, Neisseria cinerea, N. flava, N. lactamica, N. meningitidis (n = 3), N. sicca, N. subflava, Proteus mirabilis, Serratia marcescens, Streptococcus agalactiae, S. pneumoniae, S. viridans, Staphylococcus aureus (n = 2), S. epidermis, S. pyogenes, and S. saprophyticus. Organisms were grown on tryptic soy agar with sheep blood, supplemented chocolate agar, or Sabouraud dextrose agar with antibiotics (Fort Richard Laboratories, Auckland, NZ) under optimal conditions (36°C and 5% CO₂ or 36°C and O₂ or under anaerobic conditions [ANO₂] for 24 to 48 h).

A pure isolate of each organism was inoculated into sterile saline to 0.5 McFarland standard turbidity (approximately 1 × 10⁸ CFU/ml of bacteria). A 50-μl portion of the resulting mixture was added to a 4.2-ml tube of cobas PCR medium. Panels included 2 to 5 organisms per tube of cobas PCR medium.

Confirmatory testing.

Where results were discordant between culture and PCR, available specimens were sent for testing at the Queensland Pediatric Infectious Diseases (QPID) laboratory, Brisbane, Australia. DNA was extracted with a MagNA Pure LC system (Roche) using a total nucleic acid isolation kit (Roche) and then sent for confirmatory testing by a N. gonorrhoeae duplex PCR test targeting the porA pseudogene and opa genes (8, 13).

Discrepancy analysis.

This study was a retrospective audit; therefore, some specimens with discrepant results were not available for testing. A board-certified microbiologist determined the expected results for these specimens by reviewing available clinical information under the following criteria. Specimens were ruled positive if there was previous or subsequent testing history of an N. gonorrhoeae-positive result (by culture and/or PCR), clinical details from the referrer indicated probable N. gonorrhoeae infection, the N. gonorrhoeae PCR specimen tested positive by supplementary testing for porA/opa gene targets, a companion specimen tested N. gonorrhoeae positive (by culture and/or PCR), or if the patient was a sexual contact of a patient diagnosed with N. gonorrhoeae infection. Specimens were ruled negative if there was no clinical history/possibility of infection or if culture was performed on an inappropriate anatomical site (e.g., external genitalia). As culture was considered the gold standard for this evaluation, where no information was available, discrepant PCR results were ruled false.

RESULTS

Between 1 January 2011 and 13 September 2011, 27,585 specimens were tested on the cobas 4800 for C. trachomatis, of which 187 (0.68%) were positive for N. gonorrhoeae during our retrospective analysis. During the same period, 144/18,789 (0.77%) (unpaired) cultures grew N. gonorrhoeae.

C. trachomatis samples with valid results were separated into “orphan C. trachomatis specimens” (where there was no request for N. gonorrhoeae culture [n = 10,185]) and “study specimens,” which were accompanied by a request for N. gonorrhoeae culture (n = 17,400), and direct culture plates (n = 97). As multiple sites were tested for some patients, there were 18,913 specimens with comparable test results, including 16,692 genital swabs, 1,555 urine specimens, and 531 rectal, 19 throat, and 84 eye swabs. Nongenital sites are not validated for testing on the cobas 4800 by the manufacturer. However, a significant proportion of specimens from sexual health clinics are from these sites; therefore, it was important to analyze assay performance in these specimen types. A total of 79/187 (42.2%) patients infected with N. gonorrhoeae were found to be coinfected with C. trachomatis at the time of testing. In contrast, only 79/2,165 (3.6%) of patients infected with C. trachomatis were positive for N. gonorrhoeae.

Samples with failed or invalid cobas 4800 results were excluded from the analysis of assay performance.

Assay performance.

The performance of the cobas 4800 N. gonorrhoeae assay was assessed by comparing valid N. gonorrhoeae PCR results with microbial culture for 17,400 specimens tested on the same day.

These data showed 122 concordant N. gonorrhoeae positive results and a further 37/17,400 (0.2%) discrepant results. Of the discrepant cobas specimens, 35 were PCR positive and culture negative (5 urine specimens and 24 cervical, 1 urethral, 1 penile, 1 vaginal, and 3 rectal swabs) and 2 were PCR negative and culture positive (both urine specimens).

Before the resolution of discrepant results, the sensitivity, specificity, and positive and negative predictive values for the performance of the cobas 4800 CT/NG test for urogenital specimens were 98.5% (95% confidence interval, 94.6% to 99.8%), 99.8% (99.7% to 99.9%), 80% (73.0% to 85.9%), and 100% (99.9% to 100%), respectively. For nongenital specimens the sensitivity, specificity, and positive and negative predictive values were 100% (71.5% to 100%), 99.5% (98.7% to 99.9%), 78.6% (49.2% to 95.3%), and 100% (99.4% to 100%), respectively.

Further testing and analysis.

There were 10 samples with discrepant results (N. gonorrhoeae positive by PCR but negative by culture) available for confirmatory PCR testing at QPID. The cobas N. gonorrhoeae results were considered correct if the sample tested positive for the porA and/or opa targets. In total, 9 samples were positive by these criteria, while 1 was not confirmed. All remaining discrepant results were evaluated by the clinical microbiologist with regard to history, clinical details, and companion specimen results.

Of the 37 discrepant samples, 27 positive PCR results were confirmed, while 2 results were ruled false negative and 8 were ruled false positive.

In total, there were 164 samples with concordant N. gonorrhoeae-positive results and 18,739 samples with concordant negative results (Table 1). The assay performance was recalculated and showed that for urogenital specimens, the sensitivity, specificity, and positive and negative predictive values were 98.7% (95% confidence interval, 95.4% to 99.8%), 100% (99.9% to 100%), 95.6% (91.1% to 98.2%), and 100% (99.9% to 100%), respectively. For nongenital specimens the sensitivity, specificity, and positive and negative predictive values were 100% (75.3% to 100%), 99.8% (99.1% to 100%), 92.9% (66.1% to 99.8%), and 100% (99.4% to 100%), respectively (Table 2).

Table 1.

N. gonorrhoeae PCR results compared to culture following discrepancy analysis and resolution

Sample	PCR result	No. of patients (M/F)a
		Culture positive	Culture negative
Urogenital	+	151 (79/72)	7 (F)
	−	2 (F)	18,087 (16,517/1570)
Rectal	+	12 (1/11)	1 (M)
	−	0	518 (24/494)
Throat	+	1 (M)	0
	−	0	50 (9/41)
Eye	+	0	0
	−	0	84 (11/73)

^aM, male; F, female.

Table 2.

Performance of the cobas 4800 N. gonorrhoeae assay by specimen type (after discrepancy resolution)^a

Specimen	n	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Urogenital
All	18,247	98.7 (95.4–99.8)	100 (99.9–100)	95.6 (91.1–98.2)	100 (99.9–100)
Female	16,605	97.5 (91.4–99.7)	100 (99.9–100)	91.9 (83.9–96.7)	100 (99.9–100)
Male	1,642	100 (95.0–100)	100 (99.7–100)	100 (95.0–100)	100 (99.7–100)
Nongenital
All	666	100 (75.3–100)	99.8 (99.1–100)	92.9 (66.1–99.8)	100 (99.4–100)
Female	86	100 (2.5–100)	100 (95.7–100)	100 (2.5–100)	100 (95.7–100)
Male	580	100 (73.5–100)	99.8 (99.0–100)	92.3 (64.0–99.8)	100 (99.3–100)
Total	18,913	98.8 (95.7–99.9)	100 (99.9–100)	95.3 (91.0–98.0)	100 (99.9–100)

^a95% confidence intervals are provided in parentheses.

A separate analysis of performance in the 564 urine specimens from females and 950 from males showed that the cobas 4800 N. gonorrhoeae test has a sensitivity of only 86.7% for female urine specimens compared to cervical or urethral swab culture, while it has 100% sensitivity for male urine specimens compared to urethral swab culture (Table 3).

Table 3.

Performance of the cobas 4800 N. gonorrhoeae assay in male and female urine specimens

Urine specimen source	n	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Females	570	86.7 (59.5–98.3)	100 (99.3–100)	100 (75.3–100)	99.6 (98.7–100)
Males	985	100 (91.0–100)	100 (99.6–100)	100 (91.0–100)	100 (99.6–100)

Missed opportunities for screening and diagnosis.

A total of 10,185 (37%) of C. trachomatis specimens in the study were not accompanied by a culture swab for gonorrhoea and were excluded from the analysis of assay performance. However, of these specimens, 43 were N. gonorrhoeae positive by PCR (8 vaginal swabs, 6 cervical swabs, 1 penile swab, 1 urethral swab, and 27 urine specimens), with 36 of these 43 being “noninvasive” specimen types. Only 1,389 (7.4%) of gonorrhoea culture specimens were not accompanied by a swab for C. trachomatis PCR.

During the period of the study, the number of extra infections that were identified by PCR was 65 (35%). These results were referred to the clinical microbiologist for interpretation.

Specificity.

The analytical specificity of the cobas 4800 C. trachomatis/N. gonorrhoeae assay was first determined for local gonococcal isolates, including both antibiotic-resistant and -sensitive strains, including a N. gonorrhoeae prolyl iminopeptidase (PIP)-negative clinical isolate (14). All were identified correctly as positive for N. gonorrhoeae. A further 34 reference laboratory organisms (including nongonococcal neisseriae and commensal and pathogenic bacterial and fungal isolates) were tested and correctly identified as negative by this assay.

DISCUSSION

For routine supplementary confirmatory testing not to be necessary, the performance indicators of specificity and PPV should reach >95% and >90%, respectively, which can be difficult in a low-prevalence population (15, 16). As prevalence decreases, PPV decreases, and therefore, the likelihood of a false-positive result increases and the effect of test specificity is magnified. Our test population exhibited a low frequency of N. gonorrhoeae infections as determined by culture (0.77%, data not shown), which is comparable to the reported national test positivity rate of 0.8% in New Zealand (4). Previous reports have shown that 35 to 41% of patients may be coinfected with C. trachomatis and N. gonorrhoeae (17), and we found that 42.2% of those infected with N. gonorrhoeae had a concurrent C. trachomatis infection. These high rates of coinfection confirm the importance of testing for both STIs, and the ability to test both organisms from a single specimen is a benefit of multiplex NAAT tests.

The performance of the cobas 4800 N. gonorrhoeae assay was assessed by comparison with culture in 18,905 paired specimens obtained from 17,400 visits, and we initially found 37 (0.2%) discrepant results between PCR and culture. Ten of these samples were available for supplementary confirmatory testing, of which 9 were positive for both opa and porA, and 1 did not give consistent results (presumably due to low organism load). After supplementary testing and clinical review of the remaining discrepant samples, 10 were deemed to have false N. gonorrhoeae PCR results (8 false positives, 2 false negatives).

The performance indicators calculated from these data reached >95% sensitivity and specificity and >90% PPV and NPV. Notably, our PPV reached 95.6% in urogenital specimens and 92.9% in nongenital specimens. However, it should be noted that a limitation and potential bias of this study was that the PCR and culture were, for the most part, not performed on the same sample type (i.e., PCR was performed on urine samples and bacterial culture on genital swabs). Nevertheless, our findings are supported by two recent studies which showed that the cobas N. gonorrhoeae test had lower levels of false positivity than predecessor assays (9) and does not require a confirmation test for N. gonorrhoeae in urine specimens (10).

Despite recent calls to move to NAAT routinely for pharyngeal and rectal testing, it is still recommended that positive results from these sites be confirmed by supplemental assays (5). The presence of potentially cross-reacting commensal Neisseria species in nongenital sites may compromise the specificity of NAATs, and the cobas 4800 CT/NG test is not commercially validated for these specimen types. While our results showed that that there was no cross-reactivity with either local or reference laboratory strains of commensal Neisseria species, it should be noted that we tested only limited numbers of throat samples (n = 51) compared to rectal samples (n = 531) and had no positive eye swabs in the validation. Therefore, supplementary testing of cobas 4800 N. gonorrhoeae-positive results for throat and eye samples is still warranted (13).

While different sample types were used for much of the PCR and culture-based testing, the results suggest that if PCR were the gold standard test against which the sensitivity of culture was assessed, the sensitivity of culture for urogenital and nongenital specimens would be low, at 80% and 79%, respectively. The poor sensitivity of culture may contribute not only to a higher prevalence of gonorrhea but also to higher rates of HIV transmission (5). The limited sensitivity of culture compared with NAAT, particularly for pharyngeal and rectal samples, is well substantiated (6, 7, 18).

Self-taken swabs and urine samples are now widely used for C. trachomatis testing, but in our laboratory, culture for N. gonorrhoeae was carried out only on cervical, urethral, throat, and rectal swabs. The comparative difficulty of obtaining invasive specimens may have led to a decrease in testing for N. gonorrhoeae.

This is evidenced by the one-third of patients who provided “noninvasive” specimens for C. trachomatis PCR testing but no culture specimen during the study period. Indeed, 43 of these patients were found to be positive for gonorrhoea by N. gonorrhoeae PCR, with the majority of these results (36/43) coming from “noninvasive” specimen types.

The cobas 4800 N. gonorrhoeae PCR assay has several limitations. As in previous studies on C. trachomatis/N. gonorrhoeae NAAT assays, our data show that a urine specimen alone is not sufficiently sensitive (86.7%) to screen for N. gonorrhoeae in females (19, 20). We have previously shown that interference from mucopurulent discharge in N. gonorrhoeae-positive patients causes failed cobas 4800 results (12), and this has also been reported by Hopkins et al. (10). Most failed results can be resolved by treatment with Sputasol (1.4% DTT; Oxoid) (12). Methods to overcome this technical problem are essential to the reliable use of the cobas 4800 C. trachomatis/N. gonorrhoeae assay when secondary assays are not available. Extensively drug-resistant (XDR) N. gonorrhoeae strains displaying high-level resistance to third-generation cephalosporins (ceftriaxone and cefixime) may pose a problem for transferring N. gonorrhoeae diagnosis to NAAT methods and empirically treating patients. Therefore, enhanced surveillance of antibiotic susceptibility is recommended (21, 22). However, maintaining a viable organism for this testing is not possible in the high-salt cobas PCR collection medium. Until molecular methods for detection of gene sequences that confer resistance are validated for routine laboratory use, testing by both culture and PCR of N. gonorrhoeae is required to continue surveillance of antibiotic susceptibility (23). The challenge in a diagnostic laboratory arises when it is not economically viable to provide both tests. The question of whether diagnostic sensitivity is subordinate to antimicrobial resistance surveillance in this situation remains to be answered.

In summary, this audit comprehensively evaluated the performance of the cobas 4800 CT/NG test for N. gonorrhoeae compared to microbial culture. The cobas 4800 CT/NG test diagnosed 33% more urogenital and 25% more rectal N. gonorrhoeae infections than culture, and, based on performance indicators, neither urogenital nor rectal specimens require supplementary confirmatory testing in our low-prevalence population.