Comparison of Test Specificities of Commercial Antigen-Based Assays and In-House PCR Methods for Detection of Rotavirus in Stool Specimens

Abstract

Seven commercial rotavirus antigen assays were compared with in-house PCR methods for detecting rotavirus in stool specimens. The assay sensitivities were 80% to 100%, while the specificities were 54.3% for one commercial immunochromatographic (ICT) method and 99.4% to 100% for other assays. Thus, except for one commercial ICT, all the assays were generally reliable for rotavirus detection.

TEXT

An estimated 453,000 children die annually of severe rotavirus-related gastroenteritis, with most deaths occurring in developing countries (1). In the prevaccine era, rotavirus caused 10,000 hospitalizations, 22,000 emergency department visits, and 115,000 general practice consultations in Australian children aged <5 years (2). Our recent findings (3) raised concerns regarding the specificity of the Vikia Rota-Adeno assay (bioMérieux, France) for detecting rotavirus in stool samples and followed an unexplained increase in positive results in a highly vaccinated population in which surveillance had previously shown rotavirus vaccine to have been highly effective in significantly reducing rotavirus notifications and rotavirus-related hospitalizations (4, 5). In that study (3), we found that, of 81 available stool specimens submitted for diagnostic testing (collected between July 2011 and August 2012) and reported as positive using the Vikia kit, only 28% to 37% could be confirmed as positive using additional real-time reverse transcriptase (RT)-PCR and enzyme-linked immunosorbent assay (ELISA)-based testing. The results were highly suggestive of an unacceptably low specificity in the Vikia rotavirus immunochromatographic (ICT) assay. In this follow-up study, we sought to examine whether false positivity in the Vikia kit is an ongoing problem and to assess the performance of a wider range of ICT and ELISA rotavirus detection methods.

Convenience sampling of stool specimens submitted from patients with acute gastroenteritis to the publicly funded Central Microbiology Laboratory of Pathology in Queensland, Brisbane, Australia, for rotavirus testing occurred between July 2012 and June 2013. Samples were tested initially for rotavirus using the Vikia ICT method. Only samples with sufficient volumes for subsequent testing were included in the study. These were stored at −20°C until they underwent further testing by the additional assays. Overall, 182 stool samples from patients up to 94 years of age (median, 11 years; mean, 28 years) were included; the samples were from 101 males and 81 females. There were Vikia rotavirus-positive (n = 92) and Vikia rotavirus-negative (n = 90) specimens in this sample. We tested these specimens with six additional commercial rotavirus tests (three ICT kits and three ELISAs) and three in-house real time RT-PCR assays (Tables 1 and 2). All the ICT assays and ELISAs were performed according to their manufacturer's instructions. Performance characteristics according to the kit inserts are listed in Table S1 in the supplemental material. The RT-PCR methods comprised two TaqMan-based real-time RT-PCR assays (NVP3-PCR and JVK-PCR) and a conventional PCR (VP6 RT-PCR) and were performed as described previously (3). The oligonucleotide primers and probes used in the real-time RT-PCR assays are provided in Table S2 in the supplemental material. In order to confirm the initial Vikia assay results, we retested all specimens with the Vikia assay according to the manufacturer's instructions. The test performance characteristics from this study for each assay are reported as their sensitivity, specificity, and true-positive and true-negative proportions. The 95% confidence intervals for each of these values were calculated using Stata version 12 (Stata Corp, College Station, TX). The Children's Health Queensland Human Research Ethics Committee approved the study.

TABLE 1.

Summary of results for all 182 samples

No. of specimens with indicated result (total no) at initial testing using the Vikia kit	Vikia		ICT assay			ELISA			Rota PCR			Reference resulta
	Initial	Repeat	SD Bioline	CerTest	QuickStripe	Rotaclone	Ridascreen	Prospect	NVP3 (CT value)b	JVK (CT value)b	VP6
Positive for rotavirus (92)
2	Detected	NDc	ND	ND	ND	ND	ND	ND	ND
67	Detected	Detected	ND	ND	ND	ND	ND	ND	ND	ND	ND	ND
1	Detected	Detected	Detected	ND	ND	ND	ND	ND	ND	ND	ND	ND
1	Detected	Detected	ND	ND	ND	Detected	ND	ND	ND	ND	ND	ND
1	Detected	Detected	ND	ND	ND	ND	ND	ND	ND	Detected (41.1)	ND	ND
2	Detected	Detected	ND	ND	ND	ND	ND	ND	ND	ND	Detected	ND
1	Detected	Detected	ND	ND	ND	ND	ND	Detected	Detected (38.5)	ND	Detected	Detected
1	Detected	Detected	Detected	Detected	Detected	Detected	ND	Detected	Detected (33.2)	Detected (34.1)	Detected	Detected
16	Detected	Detected	Detected	Detected	Detected	Detected	Detected	Detected	Detected (26.8–34.5)	Detected (25.9–34.3)	Detected	Detected
Negative for rotavirus (90)
2	ND	Detected	ND	ND	ND	ND	ND	ND	ND	ND	ND	ND
1	ND	ND	ND	ND	ND	ND	ND	ND	Detected (40.8)	ND	ND	ND
2	ND	ND	ND	ND	ND	ND	ND	ND	Detected (36.0–38.2)	Detected (37.2–37.4)	Detectedd	Detected
2	ND	ND	ND	ND	ND	ND	ND	ND	ND	ND	Detected	ND
83	ND	ND	ND	ND	ND	ND	ND	ND	ND	ND	ND	ND

^aClassified as a true positive (TP) if rotavirus was detected in three or more tests, otherwise classified as a true negative (TN).

^bCycle threshold (C_T) values; ranges obtained for positive results in the real-time PCR methods (NVP3 and JVK) are provided in parentheses.

^cND, not detected.

^dA VP6 amplicon from one of these two samples in which there were discrepant positive results between PCR- and antigen-based assays was also sequenced and confirmed as being consistent with rotavirus.

TABLE 2.

Performance characteristics of each rotavirus assay applied to 182 samples from Queensland between July 2012 and June 2013^a

Assay type	TP (n)	TN (n)	FP (n)	FN (n)	TP/RefP (sensitivity [%] [95% CI])b	TN/RefN (specificity [%] [95% CI])b	TP/TP + FP (% TP [95% CI])b	TN/TN + FN (% TN [95% CI])b
Vikia initial	18	88	74	2	90.0 (68.3–98.8)	54.3 (46.3–62.2)	19.6 (12.0–29.1)	97.8 (92.2–99.7)
Vikia repeat	18	88	74	2	90.0 (68.3–98.8)	54.3 (46.3–62.2)	19.6 (12.0–29.1)	97.8 (92.2–99.7)
SD Bioline	17	161	1	3	85.0 (62.1–96.8)	99.4 (96.6)	94.4 (72.7–99.9)	98.2 (94.7–99.6)
CerTest	17	162	0	3	85.0 (62.1–96.8)	100.0 (97.7)	100.0 (80.5)	98.2 (94.8–99.6)
QuickStripe	17	162	0	3	85.0 (62.1–96.8)	100.0 (97.7)	100.0 (80.5)	98.2 (94.8–99.6)
Rotaclone	17	161	1	3	85.0 (62.1–96.8)	99.4 (96.6)	94.4 (72.7–99.9)	98.2 (94.7–99.6)
Ridascreen	16	162	0	4	80.0 (56.3–94.3)	100.0 (97.7)	100.0 (79.4)	97.6 (93.9–99.3)
ProSpecT	18	162	0	2	90.0 (68.3–98.8)	100.0 (97.7)	100.0 (81.5)	98.8 (95.7–99.9)
ROTA NVP3	20	161	1	0	100.0 (83.2)	99.4 (96.6)	95.2 (76.2–99.9)	100.0 (97.7)
ROTA JVK	19	161	1	1	95.0 (75.1–99.9)	99.4 (96.6)	95.0 (75.1–99.9)	99.4 (96.6)
ROTA VP6	20	158	4	0	100.0 (83.2)	99.4 (93.8–99.3)	83.3 (62.6–95.3)	100.0 (97.7)

^aRefP, positive by the reference standard criteria; RefN, negative by the reference standard criteria; CI, confidence interval (considered a true positive if rotavirus was detected in three or more tests [RefP], otherwise considered a true negative [RefN]); TP, true positive; TN, true negative; FP, false positive; FN, false positive.

^bWhere the percentage point estimate is 100%, the lower 97.5% confidence limit is provided.

The results for the 182 specimens with each kit and the associated performance characteristics are summarized in Tables 1 and 2, respectively. Overall, there was close agreement between all methods with the exception of the Vikia assay. Specifically, 67 of the 90 (74%) samples that were positive in the Vikia test were negative in every other assay evaluated upon retesting (Table 1). When the Vikia results were excluded, only 12 (6.6%) of the 182 samples provided discrepant results among the remaining ICT, ELISA, and PCR methods. The performance characteristics (Table 2) were determined on the basis of a reference standard whereby samples that provided positive results in three or more methods were considered true positives; all other samples were considered true negatives. On the basis of this standard, the sensitivities of the kits ranged from 80% to 100%. Specificity was lowest for the Vikia kit at 54.3%, whereas observed specificities for the remaining methods were 99.4% to 100%. None of the assays achieved the sensitivities described in their kit inserts. However, this is most likely due to the use of PCR assays in this study, whereas most antigen-based assays would have been validated using other antigen-based methods. In fact, including PCR test results within the reference standard may be viewed as having negatively biased the sensitivity values for other the assays, as PCR can detect low-level virus shedding from infection weeks earlier and unrelated to the current illness (6). This is further reflected in those samples providing negative results by one or more antigen-based methods but being positive by real-time PCR, with their cycle threshold (C_T) values exceeding 33 cycles (Table 1). These particular C_T values were among the highest observed and indicated a low viral load. Another limitation of our study was that there were only 20 true-positive samples, and this may have influenced the certainty around sensitivity calculations, as shown by the broad 95% confidence intervals associated with these data. In contrast, except for the Vikia method, the specificities for all the other commercial methods were comparable with those reported by the manufacturers.

These data show that the specificity problems observed previously with the Vikia assay (3) remain and that the same problems are not evident with the other ICT or ELISA methods we studied. Based on our results, PCR provided the best overall sensitivity and specificity. While the antigen tests were not as sensitive as PCR, they, excluding the Vikia, were highly specific. We therefore agree with a recent international study of childhood diarrhea evaluating molecular-based detection techniques, which found that antigen testing remained suitable for rapidly diagnosing rotavirus infection in clinical samples (7).