Abstract
This study shows that the clinical performance and reproducibility of the cobas 4800 HPV test for high-risk human papillomavirus (HPV) detection fulfill the criteria as formulated in international guidelines of HPV test requirements for cervical screening purposes. Accordingly, the cobas 4800 HPV test can be considered clinically validated for cervical screening.
TEXT
A critical feature for high-risk human papillomavirus (hrHPV) tests used for cervical screening is their clinical accuracy for detection of high-grade cervical intraepithelial neoplasia (CIN) and cervical cancer (CIN2+). For primary screening, an hrHPV test should have a balanced clinical sensitivity and specificity to allow effective detection of CIN2+ and minimize follow-up procedures of HPV test-positive women without CIN2+. Furthermore, high intra- and interlaboratory reproducibility is required to ensure reliable performance of the test in clinical practice. Both the high-risk HPV Hybrid Capture 2 method (HC2) and GP5+/6+-PCR-enzyme immunoassay (EIA) fulfill these specifications and are considered clinically validated for screening purposes (4). In order to facilitate the validation procedure of any novel, candidate hrHPV assay without the necessity of performing large, prospective screening trials, specific criteria have recently been outlined by an international consortium based on clinical equivalence analysis (3, 4). Here, we evaluated the recently FDA-approved cobas 4800 HPV test (1) according to this validation strategy. The cobas 4800 HPV test features automated sample preparation combined with real-time PCR technology to detect 14 hrHPV genotypes. PCR amplification and detection occur in a single tube, where probes with four different reporter dyes track the different targets in the multiplex reaction: (i) HPV16, (ii) HPV18, (iii) 12 hrHPV types (i.e., HPV31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, and -68) as a pool, and (iv) β-globin as the control for extraction and amplification adequacy.
The clinical performance of the cobas 4800 HPV test was assessed relative to HC2, which detects 13 hrHPV types (i.e., HPV16, -18, -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, and -68), in a cohort of screening participants originally screened by cytology and HC2 cotesting (intervention group of the VUSA-Screen trial) (5, 6). A detailed description of the VUSA-Screen trial, including the referral policy and follow-up procedure, has been published previously (6). Informed consent was obtained from all study participants, and this study followed the local ethical guidelines of the medical center.
A total of 860 archived cervical scrapings were selected comprising (i) a set for clinical sensitivity analysis of 60 representative scrapes from women (median age of 35 [range, 29 to 60] years) who had histologically confirmed CIN2+ (i.e., 23 with CIN2, 33 with CIN3, 1 with adenocarcinoma, and 3 with squamous cell carcinoma) diagnosed within a median follow-up time of 3.3 (range, 0 to 32.6) months and detected through either HC2 or cytology or both (hereafter referred to as cases) and (ii) a set for clinical specificity analysis of 800 representative scrapes from women (median age of 45 [range, 29 to 61] years) without CIN2+ diagnosis within a median follow-up time of 24.6 (range, 0 to 54.1) months (hereafter referred to as controls). HC2 and cytology data of these scrapings were retrieved from the trial database. Of the cases, 17 (28.3%) had normal cytology and 55 (91.7%) were HC2 positive. Of the controls, 789 (98.6%) had normal cytology, 11 had a borderline or mild dyskaryosis reading (i.e., equaling that of ASC-US [atypical squamous cells of undetermined significance]/ASC-H [atypical squamous cells, cannot rule out a high-grade lesion]/LSIL [low-grade intraepithelial lesion]), and 45 (5.6%) had a positive HC2 test.
Cervical scrapes (1/10 of the original sample) were used for the cobas 4800 HPV test according to recommendations of the manufacturer (Roche) in a blinded fashion, and results were compared to HC2 data afterward. A valid cobas 4800 HPV test result was obtained for all samples (Table 1). Of the cases, 54 (90%) had a positive cobas 4800 HPV test result. Of the controls, the cobas 4800 HPV test gave a positive hrHPV result for 43 (5.4%) samples. Detailed cobas 4800 HPV test results are given in Table 2. Agreement between the cobas 4800 HPV test and HC2 was strong, i.e., 97.3% (778/800; 95% confidence interval [95% CI], 95.9 to 98.2; kappa value, 0.74) and 98.3% (59/60; 95% CI, 89.1 to 99.8; kappa value, 0.90) for controls and cases, respectively.
Table 1.
cobas 4800 HPV test findings among 860 scrapings collected in the baseline round in the VUSA-Screen trial, stratified by case/control status
| Status | cobas 4800 HPV test resulta | No. of specimens analyzed by HC2b that are HPV: |
Total no. of specimens | |
|---|---|---|---|---|
| Negative | Positive | |||
| Control (<CIN2) | hrHPV negative | 745 | 12e | 757 |
| hrHPV positive | 10d | 33 | 43 | |
| Total | 755 | 45 | 800 | |
| Case (CIN2+) | hrHPV negative | 5 | 1c | 6 |
| hrHPV positive | 0 | 54 | 54 | |
| Total | 5 | 55 | 60 | |
Sensitivity and specificity values of the cobas 4800 HPV test for CIN2+ were 90% (54/60 specimens; 95% CI, 82.4 to 97.6) and 94.6% (95% CI, 93.0 to 96.2), respectively.
Sensitivity and specificity values of HC2 for CIN2+ were 91.7% (95% CI, 84.7 to 98.7) and 94.4% (95% CI, 92.8 to 96.0), respectively.
The discrepant result may relate to an initial workup failure. The leftover cobas-extracted DNA of this sample also tested negative by the GP5+/6+-PCR-EIA-reverse line blot (RLB) method (8), whereas both the cobas 4800 HPV test and GP5+/6+-PCR-EIA-RLB were positive on a renewed cobas DNA extract of the original cervical scrape sample (i.e., positive for HPV16, -18, and another type and positive for HPV16, -18, and -31, respectively). By comparison, all double HC2/cobas 4800 HPV test-negative cases were negative by GP5+/6+-PCR throughout, both on the original cobas DNA extracts and on extracts derived from an extra cobas DNA extraction procedure.
Of the 10 HC2-negative/cobas 4800 HPV test-positive specimens, 8 (80%) were hrHPV positive by GP5+/6+-PCR, and 2 were negative. Five of the 8 GP5+/6+-PCR-positive specimens revealed HPV16 alike the cobas 4800 HPV test, whereas three others contained HPV35, HPV45, and HPV66, respectively. In the cobas 4800 HPV test, the latter three tested positive for “other hrHPV.”
Of the 12 HC2-positive/cobas 4800 HPV test-negative specimens, 7 were positive following GP5+/6+-PCR-RLB for genotypes that could not be detected by the cobas 4800 HPV test, i.e., HPV53 (n = 1 specimen), HPV67 (n = 1 specimen), and HPVlrX (n = 5 specimens), and 3 were positive with the hrHPV EIA probe but untypeable (HPVhrX). The remaining two samples were GP5+/6+-PCR positive for HPV52 (n = 1 specimen) and HPV59 (n = 1 specimen).
Table 2.
Detailed results of the cobas 4800 HPV test in relation to HC2 stratified by case/control status
| Status | cobas 4800 HPV test result | No. of specimens analyzed by Hybrid Capture 2 that are HPV: |
Total no. of specimens | |
|---|---|---|---|---|
| Negative | Positive | |||
| Control (<CIN2) | HPV negative | 745 | 12 | 757 |
| HPV16 | 5 | 8 | 13 | |
| HPV18 | 0 | 3 | 3 | |
| Other hrHPV | 5 | 16 | 21 | |
| HPV16 and other hrHPV | 0 | 3 | 3 | |
| HPV18 and other hrHPV | 0 | 3 | 3 | |
| Total | 755 | 45 | 800 | |
| Case (CIN2+) | HPV negative | 5 | 1 | 6 |
| HPV16 | 0 | 24 | 24 | |
| HPV18 | 0 | 5 | 5 | |
| Other hrHPV | 0 | 16 | 16 | |
| HPV16 and other hrHPV | 0 | 9 | 9 | |
| Total | 5 | 55 | 60 | |
Clinical sensitivity and specificity values of the cobas 4800 HPV test were compared to those of HC2 by using a noninferiority score test (software R), using a sensitivity threshold for CIN2+ of at least 90% and a specificity threshold for CIN2+ of at least 98% relative to that of HC2 (4). Both clinical sensitivity and specificity for CIN2+ of the cobas 4800 HPV test were noninferior to those of the HC2 using the predetermined thresholds (P = 0.022 and P = 0.0009, respectively). Whereas some minor differences at the analytical level of HC2 and the cobas 4800 HPV test assays were seen (Table 1), these had no major impact on their clinical performances. Our findings of comparable performance of the cobas 4800 HPV test and HC2 are in line with previous studies (2, 7).
To determine intralaboratory and interlaboratory agreement of the cobas 4800 HPV test, 3 portions (1/10 of the original sample each) of a set of 553 samples, 194 of which tested positive by HC2, were independently subjected to the cobas 4800 HPV test. The first two portions were tested within the same laboratory (Lab A; VU University Medical Center [VUmc]; Amsterdam, The Netherlands) at an interval of 8 to 10 weeks, and the third was tested in a different laboratory (Lab B; Radboud University Nijmegen Medical Center, Nijmegen, The Netherlands). Different assay lots were used in the different laboratory settings.
The cobas 4800 HPV test displayed sufficient intralaboratory reproducibility in time (Table 3) and interlaboratory reproducibility (Table 4). Both fulfilled the validation guidelines in that the lower confidence bound of percentage of agreement was higher than 87%, and the corresponding kappa value was higher than 0.5 (4). Intralaboratory agreement as pointed out by kappa statistics was higher than interlaboratory agreement. It is difficult to find out what variable might be responsible for this difference, since the cobas 4800 HPV test procedure involves a sequence of several consecutive steps, including sampling processing, DNA extraction, and real-time PCR analysis. Since different lots of reagents were used in Lab A and Lab B, these differences may simply reflect subtle differences between different lots, though within acceptable limits. The cobas 4800 HPV test provides HPV16 and -18 genotyping, which next to the pooled hrHPV results turned out to be reproducible in this study also (Table 3). Yet, the value of HPV16/18 genotyping in screening is still a matter of debate.
Table 3.
Intralaboratory agreementa
| First-run cobas 4800 HPV test results | No. of scrapings in second run that were hrHPV: |
ND | Total no. of specimens | |||
|---|---|---|---|---|---|---|
| Negative | Positive |
|||||
| HPV16b | HPV18c | Other HPV | ||||
| hrHPV negative | 358 | 1 | 3 | 3 | 365 | |
| hrHPV positive | ||||||
| HPV16 positiveb | 2 | 73 | 75 | |||
| HPV18 positivec | 1 | 16 | 1 | 1 | 19 | |
| Other HPV positive | 2 | 86 | 88 | |||
| ND | 3 | 2 | 1 | 6 | ||
| Total | 366 | 74 | 16 | 92 | 5 | 553 |
Overall HPV test agreement (i.e., hrHPV positive/negative) of 98.3% (534/543 specimens; 95% CI, 96.8 to 99.1). Kappa value of 0.96. Genotyping agreement (i.e., HPV16, -18, other) of 98.2% (533/543 specimens; 95% CI, 97.1 to 99.3). Kappa value of 0.96. ND, not determined.
Either combined or not combined with other hrHPV types.
If combined HPV16/18 positive, the specimen is counted among HPV16 positives.
Table 4.
Interlaboratory agreementa
| Lab A cobas 4800 HPV test results | No. of scrapings in Lab B that were hrHPV: |
ND | Total no. of specimens | |||
|---|---|---|---|---|---|---|
| Negative | Positive |
|||||
| HPV16b | HPV18c | Other HPV | ||||
| hrHPV negative | 334 | 4 | 2 | 22 | 3 | 365 |
| hrHPV positive | ||||||
| HPV16 positiveb | 1 | 73 | 1 | 75 | ||
| HPV18 positivec | 19 | 19 | ||||
| Other HPV positive | 4 | 1 | 82 | 1 | 88 | |
| ND | 4 | 2 | 6 | |||
| Total | 339 | 81 | 22 | 106 | 5 | 553 |
Lab A, results from the first portion that was analyzed at VUmc; Lab B, results from the third portion that was analyzed at the Radboud University Nijmegen Medical Center. Overall HPV test agreement (i.e., hrHPV positive/negative) of 94.6% (513/542 specimens; 95% CI, 92.4 to 96.2), with a kappa value of 0.88. Genotyping agreement (i.e., HPV16, -18, other) of 93.7% (530/542 specimens; 95% CI, 91.7 to 95.7). Kappa value of 0.88. ND, not determined.
Either combined or not combined with other hrHPV types.
If combined HPV16/18 positive, the specimen is counted among HPV16 positives.
In conclusion, the cobas 4800 HPV test fulfills all requirements as defined in the international guidelines (4) to consider this assay clinically validated for screening purposes.
Acknowledgments
P.J.F.S. provides occasional consultation to Roche and Gen-Probe, and C.J.L.M.M. is on the scientific advisory board of Qiagen. D.A.M.H. has received a speaker's fee from Roche. C.J.L.M.M., P.J.F.S., and D.A.M.H. are shareholders of Self-Screen BV, a recent spinoff company of VU University Medical Center. All other authors declare that they have no conflicts of interest.
The cobas 4800 HPV test reagents were kindly provided by Roche. The funding sources did not have any influence on the design of the study or on analysis of the results.
Footnotes
Published ahead of print on 31 August 2011.
The authors have paid a fee to allow immediate free access to this article.
REFERENCES
- 1. Castle P. E., et al. 2009. Evaluation of a prototype real-time PCR assay for carcinogenic human papillomavirus (HPV) detection and simultaneous HPV genotype 16 (HPV16) and HPV18 genotyping. J. Clin. Microbiol. 47:3344–3347 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Mateos M. L., de Chacon A. J., Rodriguez-Dominguez M., Sanz I., Rubio M. D. 2011. Evaluation of a prototype real-time PCR assay for the separate detection of human papilloma virus genotypes 16 and 18 and other high risk human papillomavirus in cervical cancer screening. Enferm. Infecc. Microbiol. Clin. 29:411–414 [DOI] [PubMed] [Google Scholar]
- 3. Meijer C. J., Berkhof H., Heideman D. A., Hesselink A. T., Snijders P. J. 2009. Validation of high-risk HPV tests for primary cervical screening. J. Clin. Virol. 46(Suppl. 3):S1–S4 [DOI] [PubMed] [Google Scholar]
- 4. Meijer C. J., et al. 2009. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int. J. Cancer 124:516–520 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Rijkaart D. C., et al. 2011. Evaluation of 14 triage strategies for HPV DNA-positive women in population-based cervical screening. Int. J. Cancer [Epub ahead of print.] doi:10.1002/ijc.26056 [DOI] [PubMed] [Google Scholar]
- 6. Rijkaart D. C., et al. 2009. Comparison of HPV and cytology triage algorithms for women with borderline or mild dyskaryosis in population-based cervical screening (VUSA-Screen study). Int. J. Cancer 126:2175–2181 [DOI] [PubMed] [Google Scholar]
- 7. Stoler M. H., et al. 2011. High-risk human papillomavirus testing in women with ASC-US cytology: results from the ATHENA HPV study. Am. J. Clin. Pathol. 135:468–475 [DOI] [PubMed] [Google Scholar]
- 8. van den Brule A. J., et al. 2002. GP5+/6+ PCR followed by reverse line blot analysis enables rapid and high-throughput identification of human papillomavirus genotypes. J. Clin. Microbiol. 40:779–787 [DOI] [PMC free article] [PubMed] [Google Scholar]