Abstract
The Validation of Human Papillomavirus (HPV) Genotyping Tests (VALGENT) studies offer an opportunity to clinically validate HPV assays for use in primary screening for cervical cancer and also provide a framework for the comparison of analytical and type-specific performance. Through VALGENT, we assessed the performance of the cartridge-based Xpert HPV assay (Xpert HPV), which detects 14 high-risk (HR) types and resolves HPV16 and HPV18/45. Samples from women attending the United Kingdom cervical screening program enriched with cytologically abnormal samples were collated. All had been previously tested by a clinically validated standard comparator test (SCT), the GP5+/6+ enzyme immunoassay (EIA). The clinical sensitivity and specificity of the Xpert HPV for the detection of cervical intraepithelial neoplasia grade 2 or higher (CIN2+) and CIN3+ relative to those of the SCT were assessed as were the inter- and intralaboratory reproducibilities according to international criteria for test validation. Type concordance for HPV16 and HPV18/45 between the Xpert HPV and the SCT was also analyzed. The Xpert HPV detected 94% of CIN2+ and 98% of CIN3+ lesions among all screened women and 90% of CIN2+ and 96% of CIN3+ lesions in women 30 years and older. The specificity for CIN1 or less (≤CIN1) was 83% (95% confidence interval [CI], 80 to 85%) in all women and 88% (95% CI, 86 to 91%) in women 30 years and older. Inter- and intralaboratory agreements for the Xpert HPV were 98% and 97%, respectively. The kappa agreements for HPV16 and HPV18/45 between the clinically validated reference test (GP5+/6+ LMNX) and the Xpert HPV were 0.92 and 0.91, respectively. The clinical performance and reproducibility of the Xpert HPV are comparable to those of well-established HPV assays and fulfill the criteria for use in primary cervical cancer screening.
INTRODUCTION
Molecular human papillomavirus (HPV) testing is being used increasingly for cervical cancer screening and for the management of women with minor cytological abnormalities, given the sensitivity and objectivity of this approach (1–3). As a consequence, the community is faced with an expanding portfolio of HPV tests which vary with respect to target, type range, chemistry, and level of automation, and many of these tests are not associated with published, peer-reviewed evidence of performance (4). If HPV tests are to be used for the secondary prevention of cervical cancer, it is essential that they be clinically validated, and this is particularly relevant, given that HPV infection often clears without any associated morbidity.
International criteria have been established to evaluate the appropriateness of a new high-risk (HR) HPV DNA assay based on noninferior sensitivity and specificity compared to those of a clinically validated comparator assay and on high reproducibility (5). While they are not entirely perfect, they at least represent a consistent standard/benchmark via which performance can be assessed. One issue with validating an assay according to these criteria is that capturing representative samples which allow verification of noninferior accuracy can be logistically challenging.
The Validation of HPV Genotyping Tests (VALGENT) framework is an international collaboration designed to facilitate the clinical validation and comparison of HPV assays that offer genotyping capability (6). One of VALGENT's objectives is to allow the assessment of HPV assays according to the aforementioned clinical accuracy criteria, through the use of continuous samples from women participating in screening enriched with samples associated with cytopathological abnormalities.
The Xpert HPV assay (Cepheid, Sunnyvale, CA, USA) is a PCR amplification assay which detects 14 HR HPV types, offers limited genotyping (of HPV16 and HPV18/45 as a duplex), and can provide a result in around 1 h from sample addition. It differs from many competitor HPV assays in that the extraction and amplification processes are contained within an individual cartridge with minimal operator input other than addition of 1 ml of (unmanipulated) original sample (7). The initial reports on performance have been favorable when it is compared to FDA-approved assays in both primary screening contexts and colposcopy settings (7, 8). However, further data on performance pertaining to the Meijer criteria are lacking as are data on the performance/concordance of the type-specific aspects of the assay. The purpose of the present analysis was to address these gaps.
MATERIALS AND METHODS
Sample collection.
Samples used for the present analysis constitute the VALGENT-2 panel. A detailed description of this panel has been published previously (9, 10). In brief, archived samples were collated at the cytopathology laboratory at the Royal Infirmary of Edinburgh in Scotland, which is one of eight National Health Service (NHS) laboratories that serve the Scottish Cervical Screening program and processes around 70,000 samples per year. All samples were collected in PreservCyt liquid (Hologic, Bedford, MA, USA) from August 2012 to October 2012. The panel contained 1,000 consecutive samples from the routinely screened population (the Scottish screening set) and 300 cytologically abnormal samples (the Scottish enrichment set). With respect to age, as Scotland initiates screening at age 20, 419 samples were from women aged <30 years, and 881 were from women aged 30 or older.
Ethical approval.
A favorable ethical opinion for the project was provided by the West of Scotland Research Ethics Committee 4, reference 11/WS/0038.
Annotation of samples. (i) HPV status.
All samples had been tested with a GP5+/6+ PCR enzyme immunoassay (PCR-EIA), which was used as the standard comparator assay for clinical performance measurement, as per the Meijer criteria (5), and which used 0.5 ml of sample input. The amplicon generated via the GP5+/6+ PCR-EIA was also subjected to genotyping by the LMNX genotyping kit HPV GP HR (GP5+/6+ LMNX; Labo Biomedical Products, Rijswijk, The Netherlands), which uses Luminex xMAP technology for genotyping of the following HPV types: 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, 68, 26, 53, 73, and 82. The performance of the GP5+/6+ EIA and the genotyping provided by the GP5+/6+ LMNX on the VALGENT-2 panel was described in detail previously by Geraets et al. (10). Both assays were performed at DDL Diagnostic Laboratory, Rijswijk, The Netherlands. The GP5+/6+ EIA and GP5+/6+ LMNX assays were performed between April and May 2013 and April and September 2013, respectively.
The Xpert HPV testing was performed at the Scottish HPV Reference Laboratory in Edinburgh from April to August 2014, according to the manufacturer's instructions. Briefly, the assay is CE marked and detects 14 high-risk (HR) HPV types (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68), which are detected simultaneously via amplification of the E6 and E7 genes in five fluorescent channels (HPV16, HPV18/45, HPV31/33/35/52/58, HPV51/59, and HPV39/56/66/68a). The assay also incorporates a human control gene (hydroxymethylbilane synthase [HMBS]) as a sample and amplification validity check. A total of 1 ml of sample(s) was added to the cartridge before placement in the Cepheid GeneXpert system.
(ii) Underlying pathology.
Cytology findings were reported according to the British Society for Clinical Cytopathology (BSCC) reporting guidelines, with cervical intraepithelial neoplasia (CIN) nomenclature used to classify histological outcomes (11–13). Women with abnormal cytology results were managed according to guidelines defined by the United Kingdom NHS Cervical Screening Programme modified for use in Scotland (13). Colposcopically directed biopsy samples were taken as routinely indicated. Clinical management was not influenced by HPV status.
Age-specific prevalence of HR HPV according to Xpert HPV.
The prevalence and 95% confidence intervals (CI) of HR HPV as measured by the Xpert HPV were assessed in 5-year age bands within the Scottish screening set.
Assessment of clinical performance and comparison with a clinically validated SCT.
High-grade disease was classed as histologically confirmed CIN2 or higher (CIN2+) within 18 months of sample collection in the screening and enrichment set combined (n = 101). No or low-grade disease was assumed when a woman either had 2 consecutive cytologically negative samples across two screening rounds (average of 3 years and 11 months) or had CIN1 or less (≤CIN1) after having a positive cytology screen (n = 842). The sensitivity and specificity of the Xpert HPV at the level of CIN2+ or CIN3+ were assessed for the overall sample set and also for women aged 30 and older separately. CIN3+ incorporated any cancer and also high-grade glandular intraepithelial neoplasia. The relative sensitivity and specificity of the Xpert HPV were compared to those of the standard comparator test (SCT) to determine whether these measures were lower than 0.90 and 0.98, respectively. Noninferiority was assessed by a one-sided statistical test for matched data (14). The null hypothesis of inferiority of the Xpert HPV was rejected if P for noninferiority was < 0.05. The χ2 test of McNemar (McN) was used to assess differences between matched proportions and PMcN of > 0.05 indicated that the sensitivity (or specificity) of the Xpert HPV was not significantly different from that of the GP5+/6+ EIA.
Aggregation of VALGENT-2 data with existing United Kingdom clinical data set.
The sample size for VALGENT-2 was computed to assess the performance of assays in the Scottish screening context where screening is initiated at age 20 years. To bolster data on the performance of the Xpert HPV in women aged 30 years and older, VALGENT data were combined with another United Kingdom screening-based data set described in Cuzick et al. (7). As the Meijer 2009 criteria are based on women aged >30 years, this aggregation (using two United Kingdom data sets) ensured the requisite number of women/outcomes to align with said criteria. The combined data set is referred to as the “United Kingdom aggregated” data set. In Cuzick et al. (7), 3,408 samples obtained from the United Kingdom Cervical Screening Programme were tested with the Xpert HPV, the Cobas HPV test (Roche Molecular Systems, Pleasanton, CA, USA), and the Hybrid Capture 2 HPV test (hc2) (Qiagen Ltd., Manchester, United Kingdom), the latter two of which are clinically validated HPV screening tests.
Inter- and intralaboratory reproducibility and type-specific agreement.
Inter- and intralaboratory reproducibilities were obtained according to the Meijer criteria, which specify that a minimum of 500 samples are assessed, of which 30% are positive. Intralaboratory testing took place at the AML laboratory (Antwerp, Belgium) and interlaboratory testing took place between AML and the laboratory of the University Hospital of Ghent (Ghent, Belgium). A total of 510 samples collated at the AML laboratory, which had previously been tested with an in-house multiplex real-time PCR (15, 16), were assessed. For the validation criteria to be satisfied, the 95% lower confidence bound of both agreements should be >87%, with a kappa value ≥0.5 (5).
Agreement between the Xpert HPV and the GP5+/6+ LMNX for the detection of HPV16 and HPV18/45 was evaluated using Cohen's kappa statistic. HPV18/45 was reported as a combination by the Xpert HPV; “agreement” with the GP5+/6+ LMNX was satisfied if the latter assay was positive for HPV18 and/or for HPV45.
Figure S1 in the supplemental material provides an overview of the discrete sample set(s) used for the analyses described above.
RESULTS
Demographic, clinical, and technical characteristics of the VALGENT-2 panel.
When the Scottish screening and enriched populations are considered together, age ranged from 15 to 65 years, with 881 samples from women older than 30 years. The average ages in the screening and the enrichment sets were 38 years (range, 18 to 68 years) and 31 years (range, 19 to 62 years), respectively.
In the screening set, 10.2% of samples were cytologically abnormal, and 9.2% and 1% had low-grade and high-grade abnormalities, respectively. In the enrichment set, samples were proactively selected for abnormality and incorporated 100 samples with borderline nuclear change, 100 with low-grade dyskaryosis, and 100 with high-grade dyskaryosis (moderate) or worse. Four samples were considered invalid with the Xpert HPV by generating a double-negative (HPV and housekeeping control) result. Three of 4 were from the screening set, and 1 of 4 was from the enrichment set. These samples were excluded from the prevalence and accuracy assessments. Of the 1,296 evaluable samples, outcomes were available for 943 and 101 were associated with histologically confirmed CIN2+ (55 of which were CIN3+), whereas 842 were associated with no disease (i.e., two consecutive negative cytology results or biopsy-proven ≤CIN1).
HR HPV prevalence as measured by the Xpert HPV in women aged 20 to 60 years presenting for routine cervical screening in Scotland.
The HR HPV prevalences by the Xpert HPV overall and in women over 30 were 18.0% and 11.1%, respectively. Totals of 34 HPV16 and 29 HPV18/45 infections (3.4% and 2.9%, respectively) were detected, and the prevalence of HR HPV infection by 5-year age group in the screening population is presented in Fig. 1.
FIG 1.
HR HPV prevalence using the Xpert HPV in the routinely screened population in Scotland (aged 20 to 60).
Clinical performance of Xpert HPV for the detection of CIN2+.
Agreement between the Xpert HPV and the GP5+/6+ EIA, stratified by disease outcomes (CIN2+, CIN3+, and ≤CIN1), for the VALGENT-2 data set is presented in Table 1 for all ages and for women 30 years or older. The Xpert HPV detected 94% (95% CI, 86 to 98%) of CIN2+ lesions and 98% (95% CI, 90 to-100%) of CIN3+ lesions among all screened women and 93% (95% CI, 80 to 98%) of CIN2+ lesions and 96% (95% CI, 79 to 100%) of CIN3+ lesions in women 30 years and older. The specificities for identifying women with ≤CIN1 were 83% (95% CI, 80 to 85%) and 88% (95% CI, 86 to 91%) in all women and in women 30 years and older, respectively. Table 2 details the absolute and relative accuracies of the Xpert HPV compared to those of the SCT for both the VALGENT data set and the aggregated data set. In addition, cross-tabulations of the Xpert HPV versus the SCT are provided in Table 3 for the aggregated data set. The sensitivity and specificity of the Xpert HPV were not significantly different from those for the GP5+/6+ EIA (95% CI around the relative accuracy measures always included unity and PMcN was never significant) (Table 2). Noninferior sensitivity and specificity of the Xpert HPV compared to the GP5+/6+ EIA were demonstrated for all outcomes, except for women 30 years and older, where the Xpert HPV was inferior with respect to sensitivity for CIN2+ and CIN3+. However, by aggregating the VALGENT-2 data with those of Cuzick et al. (7), the hypothesis of inferiority was rejected for women aged ≤30 years.
TABLE 1.
HR HPV positivity of the Xpert HPV versus the GP5+/6+ EIA in women presenting for routine screening in Scotland (i.e., VALGENT-2 data set) for whom outcome data were availablea
| Age and outcome | GP5+/6+ EIA result | Xpert HPV result |
||
|---|---|---|---|---|
| + | − | Total | ||
| All | ||||
| CIN2+ | + | 93 | 2 | 95 |
| − | 2 | 4 | 6 | |
| Total | 95 | 6 | 101 | |
| CIN3+ | + | 54 | 0 | 54 |
| − | 0 | 1 | 1 | |
| Total | 54 | 1 | 55 | |
| ≤CIN1 | + | 116 | 25 | 141 |
| − | 30 | 671 | 701 | |
| Total | 146 | 696 | 842 | |
| ≥30 yr | ||||
| CIN2+ | + | 36 | 1 | 37 |
| − | 2 | 2 | 4 | |
| Total | 38 | 3 | 41 | |
| CIN3+ | 23 | 0 | 23 | |
| − | 0 | 1 | 1 | |
| Total | 23 | 1 | 24 | |
| ≤CIN1 | + | 55 | 14 | 69 |
| − | 21 | 562 | 583 | |
| Total | 76 | 576 | 652 | |
Cross-tabulations are stratified according to women having CIN2+, CIN3+, or two consecutive negative cytology results and/or ≤CIN1. The top portion contains women of all ages, whereas the bottom section contains women ≥30 years of age.
TABLE 2.
Clinical performance of the Xpert HPV compared to a standard, clinically validated comparator test (GP5+/6+ EIA or hc2)
| Age group | Measure | Outcome (No.) | Absolute accuracy (% [95% CI]) |
Relative accuracy: Xpert HPV vs SCT (95% CI) | PMcN | Pn.inf | |
|---|---|---|---|---|---|---|---|
| Xpert HPV | SCT | ||||||
| VALGENT 2 | |||||||
| All ages | Sensitivity | CIN2+ (101) | 94.1 (87.5–97.8) | 94.1 (87.5–97.8) | 1.000 (0.960, 1.042) | 1.00 | 0.0017 |
| Sensitivity | CIN3+ (55) | 98.2 (90.3–100) | 98.2 (90.3–100) | 1.000 | 1.00 | 0.0072 | |
| Specificity | ≤CIN1 (842) | 82.7 (79.9–85.1) | 83.2 (80.6–85.7) | 0.993 (0.972–1.014) | 0.50 | <0.0001 | |
| ≥30 yr | Sensitivity | CIN2+ (41) | 90.2 (76.9–97.3) | 92.7 (80.1–98.5) | 0.974 (0.889–1.066) | 1.00 | 0.0951 |
| Sensitivity | CIN3+ (24) | 95.8 (78.9–99.9) | 95.8 (78.9–99.9) | 1.000 | 1.00 | 0.055 | |
| Specificity | ≤CIN1 (652) | 88.3 (85.6–90.7) | 89.4 (86.8–91.7) | 0.988 (0.968–1.008) | 0.24 | <0.0001 | |
| Aggregated set | |||||||
| All ages | Sensitivity | CIN2+ (180) | 96.1 (92.2–984) | 96.1 (92.2–98.4) | 1.000 (0.978–1.023) | 1.00 | <0.0001 |
| Sensitivity | CIN3+ (102) | 99.0 (94.7–100) | 99.0 (94.7–100) | 1.000 | 1.00 | 0.0004 | |
| Specificity | ≤CIN1 (4,171) | 82.5 (81.3–83.6) | 82.1 (80.9–83.3) | 1.001 (0.992–1.011) | 0.76 | <0.0001 | |
| ≥30 yr | Sensitivity | CIN2+ (68) | 92.6 (837–97.6) | 94.1 (85.6–98.4) | 0.984 (0.931–1.040) | 1.00 | 0.019 |
| Sensitivity | CIN3+ (38) | 97.4 (86.2–99.9) | 97.4 (86.2–99.9) | 1.000 | 1.00 | 0.021 | |
| Specificity | ≤CIN1 (3,206) | 88.1 (86.9–89.2) | 87.5 (86.3–88.6) | 1.006 (0.997–1.016) | 0.18 | <0.0001 | |
a Absolute sensitivities of the tests for CIN2+ or CIN3+ and specificity for ≤CIN1 are presented, as is the relative accuracy of the Xpert HPV compared to the comparator tests. The statistical tests in the last two columns verify differences (P for the McNemar test [PMcN] or noninferiority [Pn.inf]). The upper six rows show accuracy measures according to the VALGENT-2 set where the comparator test was GP5+/6+ EIA. The aggregated data set incorporates VALGENT-2 and data from Cuzick et al. (7) where the comparator test was hc2.
TABLE 3.
HR HPV positivity of the Xpert HPV versus a clinically validated comparator testa
| hc2 or GP5+/6+ EIA result | Xpert HPV result (no.) |
Total | |
|---|---|---|---|
| + | − | ||
| CIN2+, age ≥30 yr | |||
| + | 62 | 2 | 64 |
| − | 1 | 3 | 4 |
| Total | 63 | 5 | 68 |
| <CIN2+, age ≥30 yr | |||
| + | 300 | 100 | 400 |
| − | 82 | 2,724 | 2,806 |
| Total | 382 | 2,824 | 3,206 |
| CIN2+, all ages | |||
| + | 171 | 2 | 173 |
| − | 2 | 5 | 7 |
| Total | 173 | 7 | 180 |
| <CIN2, all ages | |||
| + | 611 | 130 | 741 |
| − | 125 | 3,305 | 3,430 |
| Total | 736 | 3,435 | 4,171 |
Aggregated data are from VALGENT-2 and Cuzick et al. (7). Data are stratified according to the presence or absence of underlying disease and separately for women of all ages and women aged ≥30 years.
Type-specific agreement between the Xpert HPV and the GP5+/6+ LMNX assay.
In the screening and enrichment sets combined, 118 HPV16 infections were detected by the Xpert HPV compared to 110 by the GP5+/6+ LMNX. A total of 106 samples were HPV16 positive for both tests, whereas 12 were HPV16 positive for the Xpert HPV only and 4 were positive for the GP5+/6+ LMNX only. The overall concordance for HPV16 was 98.8% (95% CI, 98.0 to 99.3%) and the kappa value was 0.923 (95% CI, 0.886 to 0.960). Concordance for the presence of HPV18/45 was 99.2% (95% CI, 98.5 to 99.6%) with a kappa value of 0.915 (95% CI, 0.865 to 0.965).
Inter- and intralaboratory agreement for HR HPV testing with Xpert HPV.
A total of 510 samples were assessed for intra- and interlaboratory reproducibilities. The overall intralaboratory concordance of HR HPV positivity was 96.9% (95% CI, 95.0 to 98.2%) with a kappa value of 0.925 (95% CI, 0.888 to 0.961), whereas the interlaboratory concordance between the initial testing in Antwerp and retesting in Ghent was 97.8% (95% CI, 96.2 to 98.9%) with a kappa value of 0.948 (95% CI, 0.917 to 0.978).
DISCUSSION
The Xpert HPV is a cartridge-based test which detects 14 HR HPV types and offers concurrent limited typing capability. It is a rapid, technically undemanding assay and integrates extraction and detection within an individual cartridge. Different levels of instrument throughput are available for the assay, from single-module systems to 80-module systems, enabling applications within point-of-care settings to high-throughput centralized service laboratories.
As the Xpert HPV is a relatively new assay, there are, understandably, fewer data on clinical performance than for more established tests. This said, the data available thus far have been encouraging. Einstein et al. (8) assessed the clinical performance of the Xpert HPV on 697 samples obtained from colposcopy referral populations across 7 U.S. sites, with performance compared to the Cobas HPV test and the hc2. The Xpert HPV showed sensitivity for CIN2+ comparable to the COBAS HPV test and the hc2, whereas the highest specificity was conferred by the hc2 followed by the Xpert HPV and then the Cobas HPV test, leading the authors to conclude that the Xpert HPV performance was “comparable to that of currently available clinically validated tests.”
A further analysis of this colposcopy study was performed by Castle et al. (17). Here, the authors assessed assay agreement across two samples and demonstrated high agreement of 95%. This observation of high agreement reconciles with the results of the present study where inter- and intralaboratory agreements were also high.
To our knowledge, this is the first study to assess the performance of the Xpert HPV using the Meijer criteria, and, accordingly, the Xpert HPV fulfills the criteria with respect to sensitivity and specificity relative to a clinically validated standard comparator test and also inter- and intralaboratory reproducibilities. This assessment builds on the previous work of Cuzick et al. (7), where a total of 3,408 prospective samples derived from women presenting for routine cervical screening within the United Kingdom program and collated across 3 sites were tested with the Xpert HPV, hc2, and Cobas HPV tests. The respective sensitivities of the assays for CIN2 + were 98.7%, 97.5%, and 98.7% with specificities of 82.3%, 82.7%, and 82.3%. Indeed, aggregation of these data with those from the VALGENT-2 series allowed more precise assessment of the performance of the Xpert HPV in women older than 30, an important consideration given that many HPV-based primary screening protocols stipulate 30 as a minimum age for application. Consistent with previous work (9), the data also demonstrate the high prevalence of HR HPV in the United Kingdom, particularly in young women, emphasizing the need for appropriate triage strategies in an era of HPV primary screening (18).
Like many other HPV assays, the Xpert HPV offers limited typing capability (HPV16 and HPV18/45). Type-specific agreement for HPV16 and HPV18/45 between the Xpert HPV and the GP5+/6+ LMNX was high, although because the Xpert HPV does not delineate between 18 and 45 separately, the concordance between it and the LMNX, which does provide individual resolution, is somewhat artificial. Further outputs from the VALGENT studies will include further comparisons for all assays used within the projects so that type-specific discordances and their relevance can be examined more comprehensively.
There are caveats to the analysis. As the time frame of testing for the GP5+/6+ EIA, LMNX, and Xpert HPV were not exactly matched, it is feasible that this may have influenced the results. Furthermore, storage of VALGENT-2 samples prior to testing may affect assay performance to an extent. However, if storage were to have a deleterious impact on assay detection, lower sensitivity may be anticipated, whereas the sensitivity of the Xpert HPV for CIN2+ and CIN3+ was high and equivalent to or higher than that described in the prospective series (7, 8). In addition, as biopsies were only indicated as a consequence of preceding abnormal cytology, some CIN2+ may have been missed, although further longitudinal follow-up of the cohort, which is planned, will address this.
To conclude, the clinical performance and reproducibility of the Xpert HPV are comparable to those of standard comparator tests (the hc2 and the GP5+/6+ EIA), which demonstrated better protection against cervical cancer than cytology results alone (5). Therefore, the Xpert HPV may be added to the list of HPV assays considered validated for primary screening for cervical cancer (19).
Supplementary Material
ACKNOWLEDGMENTS
M.A. was supported by the seventh framework program of DG Research of the European Commission, through the COHEAHR Network (grant 603019) and by the Joint Action CANCON, which has received funding from the European Union in the framework of the Health Programme (2008 to 2013). This work made use of the Scottish HPV Archive, a resource for research (www.shine.mvm.ed.ac.uk/archive.shtml) set up through a program grant from the Chief Scientist Office of the Scottish Government (CZB/4/658).
We are grateful to colleagues in the cytology department and the Scottish HPV Reference Laboratory at the Royal Infirmary of Edinburgh, including Nicola Paton and Daniel Guerendiain, for support with data collation and testing. Thanks are also due to NRS Lothian Bioresource (formerly SAHSC Bioresource) for support with sample capture and governance.
Although industrial funding was used for this evaluation, it was investigator-led and the analysis was performed independently. K.C. has received project funding and/or consumables to carry out assay evaluations from HOLOGIC, Qiagen, Roche, NorChip, Cepheid, GeneFirst, GSK, and Abbott in the past. J.C. has received research funding (to the Wolfson Institute of Preventive Medicine) from Qiagen, BD, Abbott, Hologic, Trovagene, OncoHealth, Genera, and Cepheid and has been on speakers' bureaus/advisory boards for BD, Abbott, Hologic, Trovagene, and Cepheid. The Scientific Institute of Public Health, where M.A. is employed, received support from Cepheid for methodological and statistical work as foreseen in the VALGENT Network (6). E.P. provides sporadic scientific advice for Roche and Fujirebio. D.V.B. has received travel grants from Cepheid and Hologic. The other authors declare no conflicts of interest.
Author contributions were as follows: K.C. was the local (NHS Lothian) principal investigator for the study and created the manuscript drafts. D.G. and W.Q. were responsible for delivery and analysis of the sample panel with the standard comparator test. J.C. and L.C. contributed data and analytical support with respect to the project outlined in Cuzick et al. (7). C.M. organized and delivered laboratory testing using the Xpert assay. D.V.B. and E.P. performed the intra- and interlaboratory testing. M.A. is the chief investigator of the VALGENT projects and performed the statistical analysis. All authors provided critical comment to manuscript drafts.
Funding Statement
Project funding for the work was received from Cepheid; analysis was performed independently. VALGENT is an independent research collaboration set up to compare HPV tests where manufacturers can participate.
Footnotes
Supplemental material for this article may be found at http://dx.doi.org/10.1128/JCM.00897-16.
REFERENCES
- 1.Arbyn M, Ronco G, Anttila A, Meijer CJ, Poljak M, Ogilvie G, Koliopoulos G, Naucler P, Sankaranarayanan R, Peto J. 2012. Evidence regarding human papillomavirus testing in secondary prevention of cervical cancer. Vaccine 30(Suppl 5):F88−F99. doi: 10.1016/j.vaccine.2012.06.095. [DOI] [PubMed] [Google Scholar]
- 2.Arbyn M, Roelens J, Simoens C, Buntinx F, Paraskevaidis E, Martin-Hirsch PP, Prendiville WJ. 2013. Human papillomavirus testing versus repeat cytology for triage of minor cytological cervical lesions. Cochrane Database Syst Rev 3:CD008054. doi: 10.1002/14651858.CD008054.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cuschieri K, Bhatia R, Cruickshank M, Hillemanns P, Arbyn M. 2016. HPV testing in the context of post-treatment follow up (test of cure). J Clin Virol 76(Suppl 1):S56–S61. doi: 10.1016/j.jcv.2015.10.008. [DOI] [PubMed] [Google Scholar]
- 4.Poljak M, Kocjan BJ, Oštrbenk A, Seme K. 2016. Commercially available molecular tests for human papillomaviruses (HPV): 2015 update. J Clin Virol 76(Suppl 1):S3–S13. doi: 10.1016/j.jcv.2015.10.023. [DOI] [PubMed] [Google Scholar]
- 5.Meijer CJ, Berkhof J, Castle PE, Hesselink AT, Franco EL, Ronco G, Arbyn M, Bosch FX, Cuzick J, Dillner J, Heideman DA, Snijders PJ. 2009. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int J Cancer 124:516–520. doi: 10.1002/ijc.24010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Arbyn M, Depuydt C, Benoy I, Bogers J, Cuschieri K, Schmitt M, Pawlita M, Geraets D, Heard I, Gheit T, Tommasino M, Poljak M, Bonde J, Quint W. 2016. VALGENT: a protocol for clinical validation of human papillomavirus assays. J Clin Virol 76(Suppl 1):S14–S21. doi: 10.1016/j.jcv.2015.09.014. [DOI] [PubMed] [Google Scholar]
- 7.Cuzick J, Cuschieri K, Denton K, Hopkins M, Thorat MA, Wright C, Cubie H, Moore C, Kleeman M, Austin J, Ashdown-Barr L, Hunt K, Cadman L. 2015. Performance of the Xpert HPV assay in women attending for cervical screening. Papillomavirus Res 1:32–37. doi: 10.1016/j.pvr.2015.05.002. [DOI] [Google Scholar]
- 8.Einstein MH, Smith KM, Davis TE, Schmeler KM, Ferris DG, Savage AH, Gray JE, Stoler MH, Wright TC Jr, Ferenczy A, Castle PE. 2014. Clinical evaluation of the cartridge-based GeneXpert human papillomavirus assay in women referred for colposcopy. J Clin Microbiol 52:2089–2095. doi: 10.1128/JCM.00176-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Cuschieri K, Geraets DT, Moore C, Quint W, Duvall E, Arbyn M. 2015. Clinical and analytical performance of the Onclarity HPV assay using the VALGENT framework. J Clin Microbiol 53:3272–3279. doi: 10.1128/JCM.01366-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Geraets DT, Cuschieri K, de Koning MN, van Doorn LJ, Snijders PJ, Meijer CJ, Quint WG, Arbyn M. 2014. Clinical evaluation of a GP5+/6+-based Luminex assay having full high-risk human papillomavirus genotyping capability and an internal control. J Clin Microbiol 52:3996–4002. doi: 10.1128/JCM.01962-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Smith JHF, Patnick J (ed). 2013. Achievable standards, benchmarks for reporting and criteria for conducting cervical cytopathology, 2nd ed NHS Cancers Screening Programme, Sheffield, United Kingdom: http://www.cancerscreening.nhs.uk/cervical/publications/nhscsp01.html. [Google Scholar]
- 12.Luesley D, Leeson S (ed). 2010. Colposcopy and programme management, 2nd ed NHS Cancers Screening Programme, Sheffield, United Kingdom: http://www.cancerscreening.nhs.uk/cervical/publications/nhscsp20.html. [Google Scholar]
- 13.Hirschowitz L. (ed). 2012. Histopathology reporting in cervical screening—an integrated approach, 2nd ed NHS Cancers Screening Programme, Sheffield, United Kingdom: http://www.cancerscreening.nhs.uk/cervical/publications/cc-04.html. [Google Scholar]
- 14.Tang NS, Tang ML, Chan IS. 2003. On tests of equivalence via non-unity relative risk for matched-pair design. Stat Med 22:1217–1233. doi: 10.1002/sim.1213. [DOI] [PubMed] [Google Scholar]
- 15.Depuydt CE, Benoy IH, Beert JF, Criel AM, Bogers JJ, Arbyn M. 2012. Clinical validation of a type-specific real time quantitative human papillomavirus PCR to the performance of Hybrid Capture 2 for the purpose of cervical cancer screening. J Clin Microbiol 50:4073–4077. doi: 10.1128/JCM.01231-12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Micalessi IM, Boulet GA, Bogers JJ, Benoy IH, Depuydt CE. 2012. High-throughput detection, genotyping and quantification of the human papillomavirus using real-time PCR. Clin Chem Lab Med 50:655–661. [DOI] [PubMed] [Google Scholar]
- 17.Castle PE, Smith KM, Davis TE, Schmeler KM, Ferris DG, Savage AH, Gray JE, Stoler MH, Wright TC Jr, Ferenczy A, Einstein MH. 2015. Reliability of the Xpert HPV assay to detect high-risk human papillomavirus DNA in a colposcopy referral population. Am J Clin Pathol 143:126–133. doi: 10.1309/AJCP4Q0NSDHWIZGU. [DOI] [PubMed] [Google Scholar]
- 18.Wentzensen N, Schiffman M, Palmer T, Arbyn M. 2016. Triage of HPV positive women in cervical cancer screening. J Clin Virol 76(Suppl 1):S49–S55. doi: 10.1016/j.jcv.2015.11.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Arbyn M, Snijders PJ, Meijer CJ, Berkhof J, Cuschieri K, Kocjan BJ, Poljak M. 2015. Which high-risk HPV assays fulfil criteria for use in primary cervical cancer screening? Clin Microbiol Infect 21:817–826. doi: 10.1016/j.cmi.2015.04.015. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.