Evaluation of Six Commercial Nucleic Acid Amplification Tests for Detection of Neisseria gonorrhoeae and Other Neisseria Species

Abstract

Molecular detection of Neisseria gonorrhoeae in extragenital samples may result in false-positive results due to cross-reaction with commensal Neisseria species or Neisseria meningitidis. This study examined 450 characterized clinical culture isolates, comprising 216 N. gonorrhoeae isolates and 234 isolates of nongonococcal Neisseria species (n = 218) and 16 isolates of other closely related bacteria, with six commercial nucleic acid amplification tests (NAATs). The six NAATs tested were Gen-Probe APTIMA COMBO 2 and APTIMA GC, Roche COBAS Amplicor CT/NG and COBAS 4800 CT/NG tests, BD ProbeTec GC Qx amplified DNA assay, and Abbott RealTime CT/NG test. All assays except COBAS Amplicor CT/NG test where four (1.9%) isolates were not detected showed a positive result with all N. gonorrhoeae isolates (n = 216). Among the 234 nongonococcal isolates examined, initial results from all assays displayed some false-positive results due to cross-reactions. Specifically, the COBAS Amplicor and ProbeTec tests showed the highest number of false-positive results, detecting 33 (14.1%) and 26 (11%) nongonococcal Neisseria isolates, respectively. On the first testing, APTIMA COMBO 2, APTIMA GC, Abbott RealTime, and Roche COBAS 4800 showed lower level of cross-reactions with five (2.1%), four (1.7%), two (1%), and two (1%) of the isolates showing low-level positivity, respectively. Upon retesting of these nine nongonococcal isolates using freshly cultured colonies, none were positive by the APTIMA COMBO 2, Abbott RealTime, or COBAS 4800 test. In conclusion, the COBAS Amplicor and ProbeTec tests displayed high number of false-positive results, while the remaining NAATs showed only sporadic low-level false-positive results. Supplementary testing for confirmation of N. gonorrhoeae NAATs remains recommended with all samples tested, in particular those from extragenital sites.

INTRODUCTION

Neisseria gonorrhoeae causes the second most prevalent bacterial sexually transmitted infection (STI) in men and women globally (42). As clinical signs of gonococcal infection may overlap with those of other STIs, laboratory testing is crucial for appropriate diagnosis and subsequent adequate treatment. The accurate diagnosis of gonorrhea relies on laboratory tests that are sensitive, specific, reproducible, and robust because of nonspecific clinical manifestations or lack of symptoms in women as well as men (43). Nucleic acid amplification tests (NAATs) have a number of recognized advantages over other diagnostic assays, which include increased sensitivity of detection and ease of sample collection (including use of self-collected samples) and transport, compared to culture-based methods (13, 26, 27, 31, 32). A number of commercial NAAT systems and assays developed “in-house” are currently in use for the detection of urogenital gonococcal infection, and their use has seen many of the anticipated benefits of NAATs. However, with increasing use of commercial and “in-house” NAAT systems over time and in different geographical settings, the need for both a considered approach to the application of NAATs and for an awareness of their limitations has arisen. These considerations include the sensitivity and specificity of the primary NAAT screening test, and where used, supplementary “confirmatory” tests as well as the prevalence of infection in various population groups (18). A number of assays have been shown to cross-react with other Neisseria species (10, 20, 23, 29, 36). The Centers for Disease Control and Prevention (CDC) (Atlanta, Georgia) and the Australian Public Health Laboratory Network have proposed a number of testing algorithms for confirmation of STIs by NAATs, which require the use of additional or supplementary assays (12, 28, 43). There is increasing evidence for the rise in infection in extragenital sites, which include the rectum and oropharynx, particularly in population groups of men who have sex with men (38). Commercial NAAT systems currently on the market have not been cleared by the FDA for diagnosis of specimens from the rectum, pharynx, or conjunctiva; however, as detection of N. gonorrhoeae by NAAT is more sensitive than culture (43), laboratories continue to offer these tests to diagnose extragenital gonorrhea (4). Extragenital sites carry a number of commensal Neisseria species and commonly Neisseria meningitidis, which due to having a high nucleic acid homology to N. gonorrhoeae may cross-react in the NAAT assay utilized (9). As cross-reactions occur in screening assays, steps to include at least one supplementary assay to confirm positive results using either a commercial or in-house-developed assay have been recommended for an accurate diagnosis (10, 18, 28). Most studies evaluating the sensitivity and specificity of the gonococcal NAAT assays utilize clinical samples whereby the gold standard for confirmation of N. gonorrhoeae is not culture, and instead a consensus NAAT gold standard is used (1). In some cases, cross-reactions are not evident, particularly if the gold standard NAAT(s) also cross-reacts with the isolate, giving a false-positive result in the assay.

In this study, we evaluated six commercial NAAT assay systems for their ability to detect N. gonorrhoeae or cross-react with Neisseria species or other closely related bacteria. This is the most comprehensive study that has addressed the issue of NAAT specificity for the detection of N. gonorrhoeae since the study published in 2003 by Palmer et al. (23). This is also the first study that thoroughly compares the specificity of the latest generation N. gonorrhoeae NAATs.

MATERIALS AND METHODS

Bacterial isolates.

The performance of the nucleic acid amplification tests (NAATs) was evaluated using 234 nongonococcal Neisseria isolates or closely related bacterial isolates, including 14 Moraxella catarrhalis isolates, 2 Moraxella osloensis isolates, 1 Neisseria animalis isolate, 1 Neisseria caviae isolate, 14 Neisseria cinerea isolates, 1 Neisseria elongata isolate, 1 Neisseria flava isolate, 7 Neisseria flavescens isolates, 4 Neisseria gonorrhoeae subsp. kochii isolates, 30 Neisseria lactamica isolates, 75 Neisseria meningitidis (serogroup A, B, C, W135 and Y) isolates, 11 Neisseria mucosa isolates, 1 Neisseria pharyngis isolate, 5 Neisseria polysacchareae isolates, 16 Neisseria sicca isolates, 41 Neisseria subflava isolates, 1 Neisseria weaveri isolate, and 9 isolates of commensal Neisseria species not identified to the species level. In addition, 216 well-characterized ATCC isolates (n = 8) and clinical isolates (n = 208) of N. gonorrhoeae from Asia, Asia-Pacific, Australia, North America, South America, Africa, and Europe were used for confirmation of assay performance. Overall, 300 of these isolates were obtained from the Neisseria Reference Laboratory at the World Health Organization Collaborating Centre for Sexually Transmitted Disease (STD) in Sydney, Australia, and 150 of these isolates were from the Swedish Reference Laboratory for Pathogenic Neisseria, Örebro, Sweden.

Sample preparation and testing.

All bacterial isolates were maintained at −70°C in nutrient broth with 20% glycerol and as required were recovered on GC lysed blood agar (LBA) plates. Although the cryocultures were prepared from a single colony culture, to ensure purity for testing, a single colony was again subcultured to another LBA plate. After incubation, 2 to 5 colonies of each isolate were suspended in 500 μl of phosphate-buffered saline (PBS) (Oxoid, Hampshire, England). An aliquot of 100 μl of suspension was immediately placed in each of the manufacturers' collection tubes: Multi-Collect (Abbott Molecular Inc., Des Plaines, IL) containing 1.2 ml of transport medium, APTIMA Swab Specimen Transport tube (Gen-Probe, San Diego, CA) containing 2.9 ml of transport medium, CT/GC Qx Swab Diluent (Becton Dickinson, Sparks MD) containing 2 ml of transport medium, STD Swab Specimen and Transport medium (Roche Molecular Systems Inc., Branchburg, NJ) containing 1 ml of transport medium, and COBAS PCR medium (Roche Molecular Systems Inc.) containing 4.3 ml of transport medium.

All inoculated transport tubes were tested in a blind manner without prior knowledge of the organism's identity and processed according to the respective manufacturer's recommendations for the available platform. Briefly, Multi-Collect (Abbott Molecular) was tested using an m2000sp instrument for sample preparation and an m2000rt instrument for real-time PCR amplification and detection of N. gonorrhoeae (Abbott). APTIMA Swab Specimen Transport Tube (Gen-Probe) was tested using the APTIMA COMBO 2 and APTIMA GC assays on the semiautomated DTS system (Gen-Probe). STD Swab Specimen and Transport Medium (Roche Molecular Systems) was tested using COBAS Amplicor instrument (Roche). COBAS PCR medium (Roche Molecular Systems) was tested using the automated COBAS 4800 (Roche). CT/GC Qx Swab Diluent (Becton Dickinson) was tested using BD ProbeTec GC Qx Amplified DNA assay on the Viper platform (Becton Dickinson). The summary of technologies used and possible cross-reacting species are shown in Table 1.

Table 1.

Nucleic acid amplification tests (NAATs) used in this study and their target regions used for detection of N. gonorrhoeae

Manufacturer [reference(s)]	Assay name	Amplification technology	Gene target	Previously reported cross-reacting speciesa
Abbott (19)	RealTime CT/NG test	Real-time PCR	opa (multicopy)	None
Becton Dickinson (2, 16, 35, 37)	BD ProbeTec GC Qx amplified DNA assay	Strand displacement amplification	Pilin (multicopy; different sequence from that used in the BD ProbeTec ET system)	N. cinera, N. lactamica, N. sicca, N. flavescens, and N. subflava
Gen-Probe (7, 8, 21)	APTIMA COMBO 2 (AC2) assay	Transcription-mediated amplification	16S rRNA (multicopy)	None
Gen-Probe (22)	APTIMA GC assay	Transcription-mediated amplification	16S rRNA (multicopy; different than the AC2 target)	None
Roche (2, 3, 5, 6, 14, 23, 35, 36)	COBAS Amplicor CT/NG test	PCR with endpoint detection	Cytosine DNA methyltransferase (single copy)	N. cinerea, N. flavescens, N. lactamica, N. subflava, and N. sicca
Roche (25)	COBAS 4800 CT/NG test	Real-time PCR	Direct repeat region DR9 (multicopy)	None

^aPreviously reported cross-reacting species are from reference 23.

All results were interpreted per the manufacturer's recommendations, and strict procedures were followed to avoid specimen contamination and carryover.

RESULTS

The results from each nucleic acid amplification test (NAAT) were matched with the corresponding organism whose identification had been confirmed by reference laboratories in Australia or Sweden, and these results are summarized in Table 2. Among 216 Neisseria gonorrhoeae isolates, all assays, except the COBAS Amplicor test, which did not identify four isolates (1.9%), detected all isolates. One isolate of N. gonorrhoeae was initially negative on the Abbott RealTime test; however, when it was retested, it showed a strong positive signal.

Table 2.

Positivity rate for detection of N. gonorrhoeae and other Neisseria species and closely related species by various gonococcal NAATs

Isolate tested	No. of specimens tested	No. of positive specimens or parameter valuea by the following NAAT:
		RealTime CT/NG	ProbeTec GC Qx	APTIMA COMBO 2	APTIMA GC	COBAS Amplicor	COBAS 4800
No. of Neisseria gonorrhoeae isolates (% detected)	216	215 (99.5)	216 (100)	216 (100)	216 (100)	212 (98.1)	216 (100)
Total no.	216
Isolates of other Neisseria species and closely related species
Moraxella catarrhalis	14	0	0	0	0	2	0
Moraxella osloensis	2	0	0	0	0	1	0
Neisseria animalis	1	0	0	0	0	1	0
Neisseria caviae	1	0	0	0	0	1	0
Neisseria cinerea	14	0	5	0	0	2	0
Neisseria elongata	1	0	0	0	0	0	0
Neisseria flava	1	0	0	0	0	0	0
Neisseria flavescens	7	0	1	0	0	2	0
Neisseria gonorrhoeae subsp. kochii	4	0	0	0	0	0	0
Neisseria lactamica	30	0	18	0	0	1	1d
Neisseria meningitidis	75	1b	1	4c	3	13	0
Neisseria mucosa	11	1b	1	0	0	0	0
Neisseria pharyngis	1	0	0	0	0	1	0
Neisseria polysacchareae	5	0	0	0	0	1	0
Neisseria sicca	16	0	0	1c	0	4	0
Neisseria subflava	41	0	0	0	0	3	1d
Neisseria weaver	1	0	0	0	0	0	0
Other commensal Neisseria species	9	0	0	0	1	1	0
Total no.	234
Initial no. of false-negative isolates (%)		1 (0.5)	0 (0)	0 (0)	0 (0)	4 (1.9)	0 (0)
Initial no. of false-positive isolates (%)		2 (1.0)	26 (11.0)	5 (2.1)	4 (1.7)	33 (14.1)	2 (1.0)
Final no. of false-negative isolates (%)		0 (0)	0 (0)	0 (0)	0 (0)	4 (1.9)	0 (0)
Final no. of false-positive isolates (%)		0 (0)	26 (11.0)	0 (0)	4 (1.7)	33 (14.1)	0 (0)

^aThe number of positive specimens is shown unless specified otherwise (i.e., number of Neisseria gonorrhoeae isolates and initial and final numbers of false-negative and false-positive isolates).

^bLow positive signal was obtained. Retesting of these isolates from culture resulted in negative results.

^cEquivocal or low positive signal obtained. Retesting of all of these isolates from culture resulted in negative results.

^dLow positive signal was obtained. Retesting of these isolates from culture resulted in negative results.

Among 234 isolates of nongonococcal Neisseria species or closely related bacterial species, initial testing showed that all assays gave some cross-reaction, resulting in false-positive results. The COBAS Amplicor and BD ProbeTec assays produced the highest number of false-positive results with 33 (14.1%) and 26 (11.1%) of the isolates, respectively, showing cross-reactions. The COBAS Amplicor assay showed cross-reactions with virtually 14 of 18 defined nongonococcal Neisseria species with the highest number of false-positive results demonstrated with N. meningitidis (13/75 isolates), N. sicca (4/16 isolates), and N. subflava (3/41 isolates). The BD ProbeTec assay primarily cross-reacted with N. lactamica (18/30 isolates) and N. cinerea (5/14 isolates). Cross-reactions with one isolate each of N. flavescens, N. meningitidis, and N. mucosa were also detected.

The Abbott RealTime assay initially showed cross-reactions with two Neisseria isolates (0.9%), namely, N. meningitidis and N. mucosa. However, when these bacterial species were retested (all companies were provided the results of the assays and accordingly had the opportunity to retest isolates; however, only three company assays were retested) using a new culture, cross-reactivity was not evident on subsequent testing.

APTIMA COMBO 2 and APTIMA GC initially showed cross-reactions with five (2.1%) and four (1.7%) isolates, respectively. APTIMA COMBO 2 cross-reacting samples included N. meningitidis (4/75 isolates) and N. sicca (1/16 isolates). All of these cross-reacting isolates showed an equivocal or low positive result, and when retested using a new culture, none showed cross-reactions. With the APTIMA GC, cross-reactions occurred with 3/75 N. meningitidis and with one commensal Neisseria isolate not identified to the species level. These isolates were not retested using APTIMA GC.

The COBAS 4800 assay showed cross-reactions with two (0.9%) of the isolates tested, one N. lactamica and the other N. subflava. Upon retesting from freshly cultured colonies, both isolates gave negative results in the assay.

DISCUSSION

Reliable, valid, sensitive, and specific laboratory assays are crucial for the accurate diagnosis of gonorrhea and for the appropriate clinical care of patients. Many commercial and in-house-developed nucleic acid amplification test (NAAT) systems are available and frequently used for gonococcal diagnostics; however, for a number of these assays, several issues have been reported in regard to false-positive results (5, 23, 34, 36) and false-negative results (17, 33, 39). Initial evaluations of developed NAATs often show good sensitivity and specificity; however, with prolonged diagnostic laboratory use, it has been shown that some assays show cross-reactions with similar or identical sequences which may be present in the genomes of closely related bacterial species (false positivity) or suboptimal sensitivity due to alteration or deletion of a target sequence in some gonococcal strains (false negativity) circulating in the population. As the distribution of N. gonorrhoeae in a population group is a nonrandom event, false-positive and false-negative rates will also vary due to the prevalence and distribution of the subtypes cross-reacting with or lacking the target, respectively, in the specific population examined. N. gonorrhoeae exhibits a natural competence for transformation, and there is frequent genetic exchange with other Neisseria species (15). This may lead to commensal Neisseria species acquiring gonococcal sequences and cause false positivity (and vice versa). Thus, false-positive results have been frequently reported in commercial assays such as Roche COBAS Amplicor and BD ProbeTec ET (5, 23, 34, 36) as well as in some “in-house” assays (41). The use of supplementary testing to confirm gonococcal NAAT positive results has been suggested and is now generally considered best practice (12, 28). The supplementary testing may include other commercial or “in-house” assays. Assays developed “in-house” that target the single-copy porA pseudogene and/or the multicopy opa gene (30, 40) have limited data available; however, as with all other NAATs, only with testing of large number of samples from various geographical regions, can the true specificity of these assays be evaluated.

With the increasing need to use NAAT systems for detection of N. gonorrhoeae in samples from extragenital sites, such as the pharynx and rectum, which harbor a higher number of commensal Neisseria species and possibly N. meningitidis, the performance characteristics of the NAAT platforms and assays need to be strictly evaluated. This is best done by utilizing clinical samples and well-characterized isolates of different Neisseria species. Some of these previous studies have also evaluated the latest generation gonococcal NAAT systems, such as Abbott RealTime CT/NG assay on m2000 (19), Gen-Probe APTIMA COMBO 2 (22), Roche COBAS 4800 CT/NG test (25), and ProbeTec GC Qx assay (24), all of which have shown excellent concordance compared to an alternative NAAT target. However, no previous study has substantially challenged sensitivity and especially specificity of all these commercial assays, using a broad, heterogeneous panel of gonococcal strains, as well as samples spiked with diverse commensal Neisseria species, different N. meningitidis strains, and other closely related bacterial species.

In the present study, a large number of characterized isolates, including 216 N. gonorrhoeae isolates and 234 isolates from nongonococcal Neisseria and closely related species was examined using all the latest generation gonococcal NAAT systems. Overall, each of the assays detected all N. gonorrhoeae isolates, with the exception of the COBAS Amplicor test where four isolates were not detected. The internal control in each of these four samples was positive, indicating that inhibition was not the cause for the lack of detection. For the assay evaluation, all tests were performed using the same stock of organisms to avoid target sequence variation in some isolates which may result in false negativity as has been shown with other targets (17, 39).

Among 234 isolates of nongonococcal Neisseria or other closely related species, two assays detected a substantially higher number of nongonococcal isolates, the COBAS Amplicor assay with 33/234 (14.1%) and ProbeTec GC Qx assay with 26/234 (11.0%) false-positive results. The COBAS Amplicor assay showed the highest level of cross-reaction, with all except five represented nongonococcal species displaying false positivity. With the ProbeTec GC Qx assay, cross-reactions occurred with N. lactamica (18/30 isolates) and N. cinerea (5/14 isolates). As strains of most of these Neisseria species are present in extragenital samples, these results clearly highlight that the use of such assays should be restricted to genital samples. It is highly recommended that if such assays are used, all positive results should be confirmed by appropriate supplementary tests to avoid the reporting of false-positive results.

The Abbott RealTime CT/NG assay also initially showed false-positive results for N. meningitidis (1/75 isolates) and N. mucosa (1/11 isolates); however, when these two isolates were retested from fresh cultures, both gave a negative result. Similarly, COBAS 4800 initially reacted positive with two isolates, 1 isolate of 30 N. lactamica isolates and 1 isolate of 41 N. subflava isolates. Similarly, both of these isolates were also negative when retested from a fresh culture. The crossing point for each of these cross-reacting isolates occurred after cycle 35, indicating a low-level cross-reaction, which may suggest a balancing between positive and negative results. APTIMA COMBO 2 and APTIMA GC were also initially positive in five and four isolates of nongonococcal Neisseria species, respectively, with four of the five isolates and three of the four isolates, respectively, being N. meningitidis isolates. When retested from an additional fresh culture sample, all subsequently showed negative results. These isolates all displayed a low positive signal with both assays in the initial testing, and accordingly indicated low-level cross-reaction. Of the four N. meningitidis isolates, three gave initially low or equivocal cross-reactivity by both APTIMA COMBO 2 and APTIMA GC. Consequently, if a laboratory utilized one of these assays as the initial screen and the other as the supplementary test, false-positive results would have been reported. Laboratories obtaining low signal on all these four assays may need to retest the samples in duplicate to ensure the signal obtained is not due to cross-reactivity.

As clearly shown in the present study, there was great variation in the performance of the various NAATs evaluated. Some assays are substantially more likely to be cross-reactive to nongonococcal Neisseria species, while other assays may show only a low level of cross-reaction. The issue of nonrepeatability of the low positive cross-reaction results across some of the cross-reactive isolates may relate to mispriming in a proportion of amplification reactions resulting in a low-level cross-reaction with a commensal Neisseria isolate.

The use of a supplementary test directing amplification of a different sequence target to that of the screening assay is an alternative which has been suggested in different countries (11, 12, 28). Hence, testing of the initial single specimen with a supplementary test targeting a different sequence than that used in the screening assay would significantly increase the confidence in accurately reporting an N. gonorrhoeae-positive result. Therefore, it is recommended that a positive result should be reported only when the specimen gives a positive result in both the screening assay and supplementary tests. If the specimen is positive only in the screening assay and negative in the supplementary test, it should be reported as negative for N. gonorrhoeae.

In this study, compared to the older version of gonococcal NAATs, cross-reacting nongonococcal isolates tested by the latest generation of gonococcal NAATs (from Roche, Abbott, and Gen-Probe) resulted in substantially lower levels of false positivity. Although it is likely that in a clinical specimen, in particular those from the pharynx and rectum, rare strains of different nongonococcal Neisseria species may also react with these assays producing a false-positive result, which could be the case in both the screening and supplementary assay and hence result in a false-positive gonorrhea diagnosis. Assays that are highly cross-reactive, such as the COBAS Amplicor and BD ProbeTec tests, should not be used as a supplementary assay, and if they are used as the primary screening assay for extragenital site samples, their results should be confirmed using two strictly validated and quality-assured supplementary assays to reduce the likelihood of reporting false-positive results due to the presence of cross-reacting isolates.