Comparison of illumigene Group A Streptococcus Assay with Culture of Throat Swabs from Children with Sore Throats in the New Zealand School-Based Rheumatic Fever Prevention Program

Abstract

Group A streptococcal (GAS) pharyngitis is a particularly important condition in areas of New Zealand where the incidence of acute rheumatic fever remains unacceptably high. Prompt diagnosis and treatment of GAS pharyngitis are cornerstones of the Rheumatic Fever Prevention Programme, but these are hindered by the turnaround time of culture. Tests with excellent performance and rapid turnaround times are needed. For this study, throat swabs (Copan ESwabs) were collected from schoolchildren self-identifying with a sore throat. Samples were tested by routine culture and the illumigene GAS assay using loop-mediated isothermal amplification. Discrepant results were resolved by retesting of the same specimen by an alternative molecular assay. Seven hundred fifty-seven throat swab specimens were tested by both methods. The performance characteristics of the illumigene assay using culture on blood agar as the “gold standard” and following discrepancy analysis were as follows: sensitivity, 82% and 87%, respectively; specificity, 93% and 98%, respectively; positive predictive value, 61% and 88%, respectively; and negative predictive value, 97% and 97%, respectively. In our unique setting of a school-based throat swabbing program, the illumigene assay did not perform quite as well as described in previous reports. Despite this, its improved sensitivity and rapid turnaround time compared with those of culture are appealing.

INTRODUCTION

Group A streptococcal (GAS) throat infections are particularly significant, as a subgroup of people (typically children) develop acute rheumatic fever (ARF) or acute poststreptococcal glomerulonephritis as a result of such infections. Although treatment of GAS pharyngitis with appropriate antibiotics markedly reduces the risk of ARF, New Zealand continues to have high rates of ARF compared with those in other developed nations (1). In 2011, the New Zealand government announced a target for a reduction in the national incidence of ARF by two-thirds, from 4.2 per 100,000 people in 2011 to 1.4 per 100,000 people by 2017 (2). To achieve this goal, the Rheumatic Fever Prevention Program (RFPP) was initiated with an emphasis on the timely detection and treatment of GAS pharyngitis. The program focuses on improved access to throat swabbing services for school-aged children who are considered to be at the highest risk of ARF, specifically, Māori and Pacific children residing in areas of high socioeconomic deprivation (3). A large component of the intervention has been the school-based throat swabbing clinics for children who self-present with sore throats during the school day. As the signs and symptoms of bacterial and viral pharyngitis overlap and differentiation on clinical grounds is difficult, the collection of a throat swab specimen for culture is the current “gold standard” for diagnosing GAS pharyngitis (4). However, it is an imperfect test for use in school-based programs, as results are not available until the following day at the earliest (and up to 72 h after swabbing), and widespread swabbing such as that seen in this program puts a considerable resource strain on community laboratories. Given these limitations, tests that have more rapid turnaround times and that offer laboratory efficiencies are worth investigating.

The illumigene group A streptococcus assay (Meridian Bioscience, Inc., Cincinnati, OH) uses the loop-mediated isothermal amplification (LAMP) technology to detect the GAS pyrogenic exotoxin B (speB) gene, and it has FDA clearance for diagnosing GAS pharyngitis. The assay is semiautomated and has a turnaround time of less than an hour.

The aim of the laboratory-based study described here was to assess the performance of the illumigene GAS assay for the detection of GAS in throat swabs from symptomatic children in the unique setting of the school-based component of the RFPP.

MATERIALS AND METHODS

The study was an extension of the already established throat swabbing service in Auckland, New Zealand, schoolchildren at risk of ARF and did not deviate from routine clinical care. The study ran at a single South Auckland primary school, and the participating children (aged 5 to 11 years) were those who had already consented/assented to the school-based public health intervention. Each classroom is visited by a health worker daily (Monday to Friday), and children are asked to self-identify as having a sore throat. Those children then have a throat swab specimen collected for routine microbiological culture. As there was no deviation from routine clinical care, a specific consent or new ethics approval was not required for this study.

Throat swab specimens were collected using Copan ESwabs (Copan Diagnostics, Inc., Corona, CA). The ESwab is a nylon-flocked swab designed to minimize entrapment of the specimen. The swab specimens were collected and immediately placed into 1 ml liquid Amies transport medium. Although this was a deviation from the illumigene assay manufacturer's instructions, the ESwab's design facilitates elution of the bacteria collected on the swab into the liquid transport medium. The swabs were kept at room temperature and transported to the laboratory, arriving there by up to 8 h following collection. On receipt in the laboratory, the specimens were processed by trained laboratory technicians. The sample was vortexed for 10 s before 50 μl of transport medium was removed for the illumigene assay, and then 50 μl of transport medium was removed for culture. Culture methods were routine with inoculation onto Columbia sheep blood agar with 3% salt medium (Fort Richard, Auckland, New Zealand) and incubation in CO₂ at 37°C for up to 48 h. Presumptive GAS colonies (based on colonial morphology) were identified by matrix-assisted laser desorption ionization–time of flight mass spectrometry (MALDI-TOF MS; Biotyper, version 3.1; Bruker Daltonics) with a score of >2.0, and the results were reported qualitatively. Isolation of group C/G streptococci (identification by MALDI-TOF MS) was also reported. With the exception of the inoculation with transport medium rather than with the swab, illumigene testing was performed according to the manufacturer's instructions. Briefly, 50 μl of transport medium was added to the sample preparation tube and vortexed for 10 s. Following vortexing, 10 drops of the specimen were transferred to a heat treatment tube and incubated at 95°C for 10 min. Then, 50 μl of lysate was transferred to the test and control chambers. The test device was then inserted into the illumipro-10 incubator/reader and amplification was initiated. After 40 min, the amplified product was detected by the presence of turbidity and read by the illumipro-10 incubator/reader.

Discrepancy testing.

The DNA extracted from the ESwab eluate with discordant results (illumigene assay positive, culture negative) was tested at the Ann & Robert H. Lurie Children's Hospital of Chicago, IL, by real-time PCR, performed on a LightCycler instrument with Roche analyte-specific reagents for GAS (Roche Diagnostics, Indianapolis, IN) as previously reported (5). A 198-bp fragment of the ptsI (phosphotransferase) gene of GAS was amplified and detected.

Culture-positive/illumigene assay-negative samples were retested by both methods. If the illumigene assay result remained negative, a dilution (0.5 McFarland standard) was made from a representative GAS colony and tested by the illumigene assay. It was intended for culture-negative/illumigene assay-positive samples to undergo repeat culture, but this was not universally done due to error.

Sample size.

Past studies have demonstrated the excellent sensitivity of the illumigene test (5–7). The expected rate of positivity for GAS was 14% (from school swabbing data). Therefore, in order to obtain 100 positive swabs, it was calculated that 720 swabs would be required.

Data analysis.

Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated first using culture as the gold standard. However, it is acknowledged that the current gold standard is culture from a swab and not liquid transport medium. Then, after discrepancy testing, they were recalculated using an estimate of GAS status where a true-positive result was either a positive result by culture or positive results by both molecular assays. A sample was considered to have a true-negative result when the culture result was negative and the specimen was either negative or positive by the illumigene assay but negative by PCR.

RESULTS

A total of 757 throat swab specimens were tested by both the illumigene assay and routine culture. Ninety-two (12.2%) were positive for GAS by culture, 57 (7.5%) were positive for group C/G streptococcus by culture, and 608 (80.3%) were culture negative. A total of 124 (16.4%) swab specimens were positive by the illumigene assay, and of these, 75 (60.5%) were also culture positive. A total of 633 (83.6%) were negative by the illumigene assay, and of these, 616 (97.3%) were also culture negative (Table 1). Using culture as the gold standard, the prevalence of GAS was 12%, and the sensitivity, specificity, positive predictive value, and negative predictive values for the illumigene GAS assay were 82%, 93%, 61%, and 97%, respectively.

TABLE 1.

Performance of illumigene assay and bacterial culture for detection of GAS in throat swabs using culture results as the gold standard

Method	No. of samples with the following resulta:				% sensitivity (95% CI)	% specificity (95% CI)	% PPV (95% CI)	% NPV (95% CI)
	TPb	FP	TN	FN
Culture	92	0	665	0	100	100	100	100
illumigene	75	49	616	17	81.5 (72.0–88.9)	92.6 (90.4–94.5)	60.5 (51.3–69.1)	97.3 (95.7–98.4)

^aTP, true positive; FP, false positive; TN, true negative; FN, false negative.

^bA true-positive result was defined as culture positivity for GAS.

Discrepant results.

There were 49 throat swab specimens negative for GAS by culture but illumigene assay positive. Two were found to be culture positive on repeat testing. Of the remaining 47 swab specimens, the DNA extracted from 46 was further tested and GAS DNA was identified in 34 samples, 1 of which also had group C/G streptococcus DNA identified. Eight samples had no streptococcal DNA, while four had only group C/G streptococcus DNA identified (Table 2).

TABLE 2.

Discrepancy testing results

Original illumigene assay resulta and repeat assay result	No. of samples
False positive (n = 49)
Culture positive on repeat testing	2
Positive by secondary molecular method	34
Negative by secondary molecular method	12
Isolate was lost and not further tested	1
False negative (n = 17)
illumigene assay positive on repeat testing	10
illumigene assay positive on broth from isolate	6
Isolate was lost and not further tested	1

^aOriginal illumigene assay result when culture was used as the gold standard.

There were 17 throat swab specimens GAS negative by the illumigene assay and positive by culture. Of these, 7 remained negative by the illumigene assay on repeat testing of the stored transport medium, while 10 were positive when the illumigene assay was repeated. Of the seven that remained negative by the illumigene assay, six had GAS identified on reculture. All of these were positive by the illumigene assay when tested using 50 μl of a 0.5 McFarland suspension.

Following discrepancy analysis, the recalculated GAS prevalence was 17%. The recalculated sensitivity, specificity, and positive and negative predicative values were 73%, 100%, 100%, and 95%, respectively, for culture and 87%, 98%, 88%, and 97%, respectively, for the illumigene assay (Table 3).

TABLE 3.

Performance of illumigene assay and bacterial culture for detection of GAS in throat swabs using composite gold standard^a for culture-positive specimens and culture result as gold standard for culture-negative specimens

Method	No. of samples with the following resultb:				% sensitivity (95% CI)	% specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
	TPc	FP	TN	FN
Culture	93	0	631	34	73.2 (64.5–80.5)	100 (99.2–100)	100 (98.1–100)	94.8 (92.9–96.4)
illumigene	109	15	633	17	86.5 (80.0–91.7)	97.6 (96.0–98.6)	87.9 (80.5–92.8)	97.3 (95.6–98.4)

^aThe composite gold standard was either a positive culture result or positive results by two molecular tests.

^bTP, true positive; FP, false positive; TN, true negative; FN, false negative.

^cA true-positive result was defined as positivity for GAS by culture or two molecular tests.

DISCUSSION

This study took place in the unique environment of a developed country tackling pockets of a high incidence of ARF with a targeted throat swabbing program for the detection of GAS in children self-identifying as having a sore throat. We found the illumigene GAS assay to have a sensitivity superior to that of culture for the detection of GAS in throat swabs, but its specificity was reduced.

Overall, the illumigene assay detected more GAS-positive specimens than culture, although 17 specimens were negative, despite a positive culture result on initial testing. The finding of 10 samples initially illumigene assay negative but positive on repeat testing suggests an issue with test reproducibility as well as sensitivity. The GAS isolates from six throat swab specimens that were repeatedly illumigene assay negative did test positive when a diluted solution was made from the isolate from the culture plate. Thus, the false-negative results may have been due to the bacterial load in the sample being lower than the lower limit of detection, which is reported in the package insert to be 400 CFU (8). The variability seen among some specimens may be due to sampling error, with bacteria being unevenly spread throughout the sample. However, specimens were vortexed before sampling to reduce the likelihood of this. It is possible that use of the liquid medium led to dilution of GAS, but the samples with false-negative results by the illumigene assay were culture positive, indicating that the bacterial load was sufficient for culture. However, the use of the transport medium rather than the swab was a deviation from the manufacturer's recommendations, and we acknowledge that this may have impacted the sensitivity of the test. We do not expect the delay of up to 8 h to have impacted the illumigene assay results, as the manufacturer states that the samples are stable at room temperature for up to 48 h from the time of swabbing to the time of performance of the illumigene assay (8), although it is acknowledged that this is based on rayon swabs. Following discrepancy analysis, the illumigene assay sensitivity (87%) remained inferior to that described in recent publications, which have reported sensitivities of greater than 98% (5–7).

Current guidelines recommend backup throat swab culture for negative rapid antigen detection tests for GAS, as their sensitivity and NPV are insufficient to rule out GAS pharyngitis (4). Here we report an NPV for the illumigene assay of 97%. While it could be argued that samples with negative illumigene assay results should also be cultured, this would be impractical in our laboratory, given the sheer volume of samples received and the ongoing resource constraints.

Not unusually for a molecular assay, the illumigene assay was less specific than culture, with a PPV of 87.9% (95% confidence interval [CI], 80.5% to 92.8%). Again the performance is not as robust as that previously reported (specificities, >96%) (5–7). We hypothesize that the lower PPV seen here was due, in part, to our study population of children who were well enough to be at school and self-identified with a sore throat. In contrast, the previously published studies examined throat swabs from children presenting to the emergency room with pharyngitis. Thus, the pretest probability in the school population could be lower because some children are likely to be carriers with a GAS burden lower than that in emergency room patients presenting with more acute and dramatic symptoms.

A further explanation for the false-positive results is that speB may not be specific to GAS; isolates of group C/G streptococcus and the Streptococcus anginosus group in India have been identified to carry speB (9). Interestingly, the isolates in four of our samples that were illumigene assay positive and culture negative were identified to be group C/G streptococcus by PCR. Lastly, the sensitivity of our culture method may have been impacted by the volume of transport medium used (50 μl) and the incubation in CO₂ (10).

The clinical impact of false-positive test results in the context of RFPP may be less concerning than that in routine primary care (11). In all likelihood, the children in RFPP are being treated for GAS pharyngeal colonization as well as infection (as they are in routine primary care [4, 12]). Although this is unnecessary antibiotic therapy, it may be reducing the rates of carriage and transmission of GAS in the community, and thus, among children at high risk of ARF, the benefits of additional antibiotic therapy may outweigh the slight risks.

The significant advantage that the illumigene assay offers RFPP, in addition to enhanced sensitivity, is a considerably improved turnaround time, enabling same-day reporting of results and antibiotic prescription, with the consequent reduced period of infectivity. However, use of illumigene or other molecular assays does not result in an isolate on which antibiotic susceptibility tests can be performed. This has little practical impact in our setting, as GAS is universally susceptible to penicillins and cephalosporins, and macrolides are employed only for those with a significant beta-lactam allergy. In addition, the rate of erythromycin resistance in our community remains low (approximately 4%).

This study has some limitations. Our methods deviated from the illumigene assay manufacturer's recommendations, in that the transport medium rather than the swab was tested. However, the FDA submission does report acceptable performance with liquid Amies transport medium. Discrepancy PCR testing was done on extracted eluate, and we cannot rule out the possibility of specimen contamination. We did not report GAS growth quantitatively and so were not able to establish the impact of the bacterial load on illumigene assay sensitivity. The DNA from one specimen was not referred for further molecular testing, which was in error. Clinical data were not collected and so could not be used to evaluate the impact of the severity of illness or concurrent antibiotic administration on test performance.

In summary, the illumigene GAS test offers the community a laboratory molecular test for the detection of GAS in throat swabs with a rapid turnaround time. The assay identifies more true-positive results for GAS at the cost of a slight drop in specificity (100 to 98%) compared to that of culture.

Funding Statement

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.