Abstract
The HPV-Risk assay is a novel real-time PCR assay targeting the E7 region of 15 high-risk human papillomavirus (HPV) types (i.e., HPV16, -18, -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, -67, and −68), and provides additional genotype information for HPV16 and HPV18. This study evaluated the clinical performance and reproducibility of the HPV-Risk assay with cervical scraping specimens and its utility with self-collected (cervico)vaginal specimens. The clinical performance of the HPV-Risk assay for cervical intraepithelial neoplasia of grade 2 or worse (CIN2+) with cervical scraping specimens was evaluated by a noninferiority analysis, relative to high-risk HPV GP5+/6+ PCR, following international guidelines for HPV test requirements for cervical cancer screening. The HPV-Risk assay showed clinical sensitivity for CIN2+ of 97.1% (95% confidence interval [CI], 89.1 to 99.3%; 67/69 samples) and a clinical specificity for CIN2+ of 94.3% (95% CI, 92.5 to 95.7%; 777/824 samples). The clinical sensitivity and specificity were noninferior to those of GP5+/6+ PCR (noninferiority score test, P = 0.006 and 0.0003, respectively). Intralaboratory reproducibility over time (99.5% [95% CI, 98.6 to 99.8%]; 544/547 samples, kappa = 0.99) and interlaboratory agreement (99.2% [95% CI, 98.6 to 99.8%]; 527/531 samples, kappa = 0.98) for the HPV-Risk assay with cervical scraping specimens were high. The agreement of the HPV-Risk assay results for self-collected (cervico)vaginal specimens and clinician-obtained cervical scraping specimens was also high, i.e., 95.9% (95% CI, 85.1 to 99.0%; 47/49 samples, kappa = 0.90) for self-collected lavage samples and 91.6% (95% CI, 84.6 to 95.6%; 98/107 samples, kappa = 0.82) for self-collected brush samples. In conclusion, the HPV-Risk assay meets the cross-sectional clinical and reproducibility criteria of the international guidelines for HPV test requirements and can be considered clinically validated for cervical screening purposes. The compatibility of the HPV-Risk assay with self-collected specimens supports its utility for HPV self-sampling.
INTRODUCTION
Persistent infection with high-risk human papillomavirus (HPV) is the causative agent for cervical cancer (1, 2). Testing for HPV DNA provides better protection against cervical cancer and its precursors, i.e., high-grade cervical intraepithelial neoplasia (CIN), compared to cytology (3–7). For primary cervical cancer screening, it is crucial that the HPV assays that are used are clinically validated to ensure optimal distinction between HPV infections associated with CIN grade 2 or worse (CIN2+) and clinically irrelevant transient HPV infections (8, 9).
A variety of HPV DNA detection assays are currently considered clinically validated with cervical scraping specimens for cervical cancer screening purposes. Validation has been based on either data from large prospective screening trials (i.e., high-risk HPV Hybrid Capture 2 [HC2] and GP5+/6+ PCR) (3, 5, 6, 10) or cross-sectional clinical equivalence analyses according to international guidelines (8, 9) for HPV DNA test requirements (11–13). In addition to clinician-based sampling, HPV self-sampling is an emerging effective strategy for cervical screening. Offering HPV testing on self-collected cervicovaginal specimens reattracts a substantial number of nonattendees into the screening program and effectively detects CIN2+ (14, 15). However, standardization of the collection device, HPV test, and sample preparation protocols is important to minimize variations in the CIN2+ sensitivity and specificity of HPV self-sampling (reviewed in reference 16). It must be realized that the use of an HPV test that is clinically validated for cervical scraping specimens does not automatically result in high clinical accuracy when it is applied to self-collected specimens (17). Therefore, a separate analysis of the candidate HPV test with self-collected samples, relative to its performance with cervical scraping specimens, is important to ensure suitability for HPV self-sampling. Given the potential variations in target cell yields between different self-samplers (16), such a comparative accuracy analysis ideally should be performed for each self-sampler type, in order to determine the best combination of self-sampler and validated HPV test.
Most assays validated for cervical screening purposes use PCR-based assays targeting regions within the HPV E1 or L1 open reading frames (11, 13, 18). However, malignant progression of cervical lesions is often associated with viral DNA integration into the genome of the host cell (19). Integration often takes place between the E1 and L1 regions, with a probability of interruption of the PCR target region. This may result in nondetection of HPV by E1- or L1-based assays (2, 19). A novel real-time PCR assay that targets the E7 region of high-risk HPV types is the HPV-Risk assay (Self-Screen BV, Amsterdam, The Netherlands). This assay was designed to detect clinically relevant infections with a total of 15 high-risk HPV types (i.e., HPV16, -18, -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, -67, and -68), chosen on the basis of their presence in other clinically validated HPV PCR assays (11, 13, 18), supplemented with types found to be stably integrated in cervical carcinoma cell lines. Compared to currently clinically validated HPV assays, this assay covers an extra type (i.e., HPV67), which was added on the basis of the following criteria: (i) it belongs to the alpha 9 species, which, in addition to HPV67, contains only established high-risk HPV types that are classified as class 1 carcinogens (20), (ii) it has been demonstrated in cervical carcinomas and several cases of cervical cancers worldwide (20, 21), and (iii) it has been found to be stably integrated in a cervical cancer cell line (22). The HPV-Risk assay simultaneously reports on a pool of non-HPV16/HPV18 high-risk HPV types and provides individual results for HPV16 and HPV18, the two most oncogenic genotypes (23, 24).
In this study, we report on the analytical and clinical performance of the HPV-Risk assay. The clinical analyses involved guideline-directed clinical validation and reproducibility analyses with cervical scraping specimens (8, 9). Furthermore, we compared the performance of the HPV-Risk assay with self-collected (cervico)vaginal samples with the performance with concomitantly clinician-obtained cervical scraping specimens. The self-collected samples were obtained with two different self-samplers, i.e., a brush-based device (25) and a lavage-based device (26).
MATERIALS AND METHODS
HPV-Risk assay.
The HPV-Risk assay (Self-Screen BV, Amsterdam, The Netherlands) is a multiplex real-time PCR-based assay designed for the clinical detection of high-risk HPV DNA in various clinical sample types. The HPV-Risk assay targets an ∼150-bp fragment of the E7 region of 15 (probably) high-risk HPV types (i.e., HPV16, -18, -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, -67, and -68) and detection is by hydrolysis probes with 3 spectrally unique fluorescent dyes, each representing different (pools of) targets. The three different (pools of) targets are HPV16, HPV18, and the 13 other high-risk HPV types combined. The human β-globin gene is detected in a fourth channel using a probe labeled with a different fluorescent dye and serves as internal control to determine the quality of the sample DNA and the presence of potential inhibitory substances. The HPV-Risk assay uses 5-μl input of the sample DNA and runs on an ABI7500 Fast Real-Time PCR system (Applied Biosystems) or an equivalent real-time PCR system with a run time of 1 h, according to the manufacturer's instructions. A sample was considered HPV positive when threshold cycle (CT) values for HPV16, HPV18, and/or other HPV types were <36. In cases in which no HPV signals were obtained, the sample was considered HPV negative when the CT value for the β-globin target was <33 (e.g., equals ∼25 cells per reaction). A sample was scored as invalid when the CT value for HPV was >36 and that for β-globin was >33.
Analytical sensitivity and specificity analyses.
DNAs from plasmids containing the complete genomes of HPV genotypes 6, 11, 16, 18, 26, 31, 33, 35, 39, 40, 42, 43, 45, 51, 52, 53, 56, 58, 59, 61, 66, 67, 68, and 70 were used as amplification targets in analytical experiments. To assess analytical sensitivity, serial 10-fold plasmid dilutions (ranging from 4.6 × 106 to 4.6 × 10−1 copies per reaction) of the 15 targeted HPV types were prepared in a background of 100 ng/reaction human placental DNA (Sigma-Aldrich), in order to mimic cervical specimens. The analytical sensitivities were tested in 6-fold replications and scored positive when at least 5/6 replicates tested positive. To assess analytical specificity, plasmid DNAs representing 46,000 copies of nontargeted HPV types (i.e., HPV6, -11, -26, -40, -42, -43, -53, -61, and -70) per reaction were used in a background of 300 pg human placental DNA (approximately 60 genome equivalents). Nonspecific targets giving positive signals in the HPV-Risk assay were further diluted to determine the limit of detection. In addition, DNA representing 10,000 copies of each of the three most common vaginal microorganisms (Chlamydia trachomatis, Neisseria gonorrhea, and Candida albicans) was analyzed.
DNA extraction.
DNA was extracted from clinical specimens using an automated silica-based extraction system (Macherey-Nagel, Düren, Germany), according to the instructions of the manufacturer, and DNA was stored at −20°C until further use. All PCR assays were performed with aliquots from the same DNA isolate.
High-risk HPV GP5+/6+ PCR (reference assay).
Standardized high-risk HPV DNA detection by the clinically validated GP5+/6+ PCR with enzyme immunoassay (EIA) read out was performed as described previously (18). Separate β-globin PCR testing for sample quality control was performed as described by de Roda Husman et al. (27). Input in the GP5+/6+ PCR and β-globin-PCR assays was 10 μl of the DNA isolate. Genotyping of the GP5+/6+ PCR products was performed with a bead-based array for high-risk HPV types (28) and a reverse line blot (RLB) analysis for low-risk types (18).
Cervical scraping specimens for clinical validation and reproducibility analyses.
DNA isolates from a total of 1,444 clinician-obtained cervical scraping specimens collected in PreservCyt medium (Hologic) in a population-based screening setting in the Utrecht and North Holland region of The Netherlands were used for clinical validation and reproducibility analyses. All scraping specimens were tested with both the high-risk HPV GP5+/6+ PCR assay (serving as a reference) and the HPV-Risk assay. The scraping specimens included 70 samples from women with histologically confirmed CIN2+ (i.e., 29 CIN2 cases, 37 CIN3 cases, and 4 squamous cell carcinoma [SCC] cases) for clinical sensitivity analysis. Of these samples, 50 (71%) had abnormal cytological findings. The remaining 20 samples (29%) were cytomorphologically normal but tested HPV positive by the high-risk HPV GP5+/6+ PCR assay. The median age at diagnosis was 39 years (range, 30 to 60 years). Another 824 cervical scraping specimens were also used for clinical specificity analysis, representing consecutive samples from women with normal cytological findings who were without evidence of CIN2+ in up to 2 years of follow-up monitoring. The median age of these women was 41 years (range, 30 to 60 years). For analysis of intralaboratory reproducibility over time and interlaboratory agreement, a final series of 550 cervical scraping specimens was used. These represented a selected set, of which 30% (165/550 specimens) tested positive in the high-risk HPV GP5+/6+ PCR assay. Of these scraping specimens, two portions were independently analyzed, after a >8-week interval, by different technicians in the Department of Pathology, VU University Medical Center (Amsterdam, The Netherlands) (hereafter referred to as laboratory 1), and a third portion was tested blindly in the Department of Medical Microbiology, Maastricht University Medical Center (Maastricht, The Netherlands) (hereafter referred to as laboratory 2). All portions were analyzed using different batches of the HPV-Risk assay.
Self-sampled specimens and corresponding clinician-collected cervical scraping specimens.
DNA isolated from two types of self-sampled specimens from women visiting an outpatient clinic (25, 26) were used to compare the performance of the HPV-Risk assay with these sample types relative to that of the HPV-Risk assay with the corresponding clinician-collected cervical scraping samples. For comparison, the high-risk HPV GP5+/6+ PCR assay was also performed for the self-sampled specimens. The two series of self-sampled specimens included 62 Delphi Screener (Delphi Biosciences, Scherpenzeel, The Netherlands) cervicovaginal lavage samples, with concomitantly clinician-collected cervical scraping specimens collected in SurePath medium (BD), which were described by Brink et al. (26), and 112 self-collected vaginal brush samples collected using a VibaBrush (Rovers Medical Devices, Oss, The Netherlands), with concomitantly clinician-collected cervical scraping specimens collected in PreservCyt medium, which were described by Dijkstra et al. (25).
Statistical analyses.
Testing of the HPV-Risk assay was performed blinded for GP5+/6+ PCR results and cytological and histological outcomes, and data were correlated subsequently. The clinical sensitivity and specificity values for the HPV-Risk assay with cervical scraping specimens were compared with those for the GP5+/6+ PCR assay using a noninferiority score test, as described by Tang et al. (29), with a relative sensitivity threshold for CIN2+ of 90% and a relative specificity threshold for CIN2+ of 98% (8). For the intralaboratory reproducibility and interlaboratory agreement analyses of the HPV-Risk assay with cervical scraping specimens, the agreement and kappa values for samples with valid test results were determined. The 95% lower confidence bounds of the intralaboratory reproducibility and interlaboratory agreement values should both be ≥87%, with kappa values of >0.5 (8). For comparison of the HPV-Risk assay for self- and clinician-collected specimens and comparison of the HPV-Risk assay with the GP5+/6+ PCR assay for self-collected specimens, overall agreement and kappa values were determined for samples with valid test results.
Interassay and interspecimen genotype agreement was determined among HPV-positive samples. For this purpose, genotype results for GP5+/6+ PCR products were categorized as (i) HPV16, (ii) HPV18, (iii) other HPV types, including HPV31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, and/or -68 or (iv) HPV X. HPV X was scored when samples tested positive with the GP5+/6+ PCR EIA but yielded no genotype with the bead-based genotyping assay. Concordant genotype findings were defined as complete agreement between the two assays, compatible findings as having at least one genotype category in common, and discordant findings as no similarity between detected genotype categories. For calculations, we used SPSS (version 20) and Stata software. P values of <0.05 were considered statistically significant.
RESULTS
Analytical performance of the HPV-Risk assay.
The analytical sensitivity of the HPV-Risk assay was evaluated with 10-fold dilution series of cloned HPV genotypes (i.e., HPV16, -18, -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, -67, and -68). All targeted types demonstrated highly specific signals in the respective channel of detection (i.e., HPV16 in the HPV16 channel, HPV18 in the HPV18 channel, and the other targeted types in the “other HPV” channel). The limit of detection of the HPV-Risk assay for the targeted HPV types ranged from 460 copies/reaction for genotypes 16, 18, and 56 to 46,000 copies/reaction for genotypes 52, 58, 66, and 67 (Table 1).
TABLE 1.
Limits of detection of the HPV-Risk assay for different targets
| Target | Limit of detection (copies/reaction) |
|---|---|
| HPV16 | 460 |
| HPV18 | 460 |
| HPV31 | 4,600 |
| HPV33 | 4,600 |
| HPV35 | 4,600 |
| HPV39 | 4,600 |
| HPV45 | 4,600 |
| HPV51 | 4,600 |
| HPV52 | 46,000 |
| HPV56 | 460 |
| HPV58 | 46,000 |
| HPV59 | 4,600 |
| HPV66 | 46,000 |
| HPV67 | 46,000 |
| HPV68 | 4,600 |
The analytical specificity of the HPV-Risk assay was assessed by testing cloned HPV DNA of genotypes 6, 11, 26, 40, 42, 43, 53, 61, and 70 using a high-copy number input of 46,000 copies/reaction. Only HPV70 displayed cross-reactivity, reflected by a signal in the other HPV channel. Further dilution revealed that the HPV70 genotype is detected at a minimal input of 17,000 copies/reaction. No cross-reactivity was observed with Chlamydia trachomatis, Neisseria gonorrhea, or Candida albicans.
Clinical validation of the HPV-Risk assay with clinician-collected cervical scraping specimens.
For clinical sensitivity and specificity analyses of the HPV-Risk assay, a series of 894 cervical scraping specimens (70 from women with CIN2+ and 824 from women without evidence of CIN2+) were tested, of which 99.9% (893/894 samples) gave valid test results (Table 2). One sample, from a woman diagnosed with CIN2, repeatedly tested invalid in the HPV-Risk assay. The clinical sensitivity for CIN2+ of the HPV-Risk assay was 97.1% (95% confidence interval [CI], 89.1 to 99.3%; 67/69 samples) (Table 2), and the clinical specificity for CIN2+ was 94.3% (95% CI, 92.5 to 95.7%; 777/824 samples) (Table 2). For comparison, these figures were 97.1% (95% CI, 89.1 to 99.3%; 67/69 samples) and 94.1% (95% CI, 92.2 to 95.5%; 775/824 samples), respectively, for the GP5+/6+ PCR assay. Both the clinical sensitivity and the specificity for CIN2+ of the HPV-Risk assay were noninferior to those of the GP5+/6+ PCR assay (noninferiority score test, P = 0.006 and 0.0003, respectively).
TABLE 2.
Comparison of the HPV-Risk assay and the GP5+/6+ PCR assay with clinician-collected cervical scraping specimens from population-based screening
| Population and HPV-Risk assay result | No. of specimens with GP5+/6+ PCR result |
P value for noninferiority score test | ||
|---|---|---|---|---|
| Negative | Positive | Total | ||
| Women without evidence of CIN2+ | ||||
| Negative | 768 | 9 | 777 | 0.0003 |
| Positive | 7 | 40 | 47 | |
| Total | 775 | 49 | 824 | |
| Women with CIN2+ | ||||
| Invalid | 1a | 1 | ||
| Negative | 1 | 1 | 2 | 0.006 |
| Positive | 1 | 66 | 67 | |
| Total | 2 | 68 | 70 | |
Not included in the noninferiority analysis.
In the sensitivity and specificity analyses, 106 samples tested positive with both assays. Of the double-positive (HPV-Risk assay and GP5+/6+ PCR assay) samples, 92.5% (98/106 samples) revealed concordant and 7.5% (8/106 samples) compatible genotyping results (see Table S1 in the supplemental material). Among the two cases with discrepant test results, one woman with CIN3 tested positive for other HPV with the HPV-Risk assay and negative with the GP5+/6+ PCR assay, and one woman with CIN2 tested negative with the HPV-Risk assay but positive with the GP5+/6+ PCR assay (HPV18 and HPV52). Another woman with CIN2 tested negative with both the HPV-Risk assay and the GP5+/6+ PCR assay.
Sixteen women without evidence of CIN2+ had discrepant results, nine of whom tested GP5+/6+ PCR positive/HPV-Risk assay negative and seven tested GP5+/6+ PCR negative/HPV-Risk assay positive. Of the nine GP5+/6+ PCR-positive/HPV-Risk assay-negative samples, 7 had signals just above the EIA threshold (all single infections, involving HPV16, -33, -35, -45, -59, -66, and HPV X). The remaining two samples had EIA values clearly above the threshold and contained HPV39 and HPV58. The seven GP5+/6+ PCR-negative/HPV-Risk assay-positive samples were all positive for other HPV; four had signals just above the HPV-Risk assay threshold. Reverse line blot analysis of GP5+/6+ PCR products revealed that 5 of the 7 samples contained HPV types reactive in the HPV-Risk assay but not detectable by high-risk HPV GP5+/6+ PCR (i.e., HPV67 [n = 2] and HPV70 [n = 3]). An additional sample showed a reverse line blot signal for HPV39 but, based on the EIA read out, this specimen was below the threshold for scoring the sample as high-risk HPV positive. Only one sample showed no genotype by reverse line blot analysis.
Intralaboratory reproducibility and interlaboratory agreement of HPV-Risk assay results.
The intralaboratory reproducibility of the HPV-Risk assay was determined with 550 cervical scraping specimens, of which 533 also had sufficient material for the interlaboratory agreement analysis. Valid HPV-Risk assay results were obtained for 547/550 samples (99.5%) tested in both series performed in laboratory 1 and for 531/533 samples (99.6%) tested in laboratory 2 (Table 3). The intralaboratory reproducibility was 99.5% (95% CI, 98.6 to 99.8%; 544/547 samples), with a kappa value of 0.99. The interlaboratory agreement was 99.2% (95% CI, 98.6 to 99.8%; 527/531 samples), and the kappa value was 0.98. Of the samples positive with the HPV-Risk assay in both runs in laboratory 1, 99.4% (154/155 samples) had concordant genotyping results and 0.6% (1/155 samples) compatible genotyping results (see Table S2A in the supplemental material). Also, the interlaboratory genotyping agreement was high, i.e., 97.2% (141/145 samples) had concordant results and 2.8% (4/145 samples) compatible results (see Table S2B in the supplemental material).
TABLE 3.
Intralaboratory reproducibility and interlaboratory agreement analyses of the HPV-Risk assay with cervical scraping specimens
| HPV-Risk assay laboratory 1/test 1 resulta | No. of specimens with HPV-Risk assay laboratory 1/test 2 resulta |
No. of specimens with HPV-Risk assay laboratory 2 resultb |
||||||
|---|---|---|---|---|---|---|---|---|
| Invalid | Negative | Positive | Total | Invalid | Negative | Positive | Total | |
| Invalid | 1 | 1 | 1 | 1 | ||||
| Negative | 2 | 389 | 3 | 394 | 1 | 382 | 2 | 385 |
| Positive | 155 | 155 | 2 | 145 | 147 | |||
| Total | 2 | 390 | 158 | 550 | 1 | 385 | 147 | 533 |
Agreement between laboratory 1/test 1 and laboratory 1/test 2 of 99.5% (95% CI, 98.6 to 99.8%; 544/547 samples), with a kappa value of 0.99.
Agreement between laboratory 1/test 1 and laboratory 2 of 99.2% (95% CI, 98.0 to 99.7%; 527/531 samples), with a kappa value of 0.98.
HPV-Risk assay with self-collected cervicovaginal lavage specimens.
The agreement between the results of the HPV-Risk assay with self-collected cervicovaginal lavage specimens and concomitantly clinician-collected cervical scraping specimens was determined for a set of women (n = 62) visiting an outpatient clinic. In addition, GP5+/6+ PCR was performed on the self-collected specimens for comparison (Table 4; also see Table S1 in the supplemental material). Of the samples with valid HPV-Risk assay results, 60.7% (37/61 samples) of the self-collected samples and 68.0% (34/50 samples) of the clinician-obtained cervical scraping specimens scored HPV positive with the HPV-Risk assay (Table 4). The overall agreement in HPV-Risk assay findings between the two sampling methods was 95.9% (95% CI, 85.1 to 99.0%; 47/49 samples), with a kappa value of 0.90. Similar agreement figures were found after stratification for CIN2+ or when comparing HPV-Risk assay results with GP5+/6+ PCR assay results for self-collected lavage samples (Table 4). Also the genotype agreement between HPV-Risk assays with the two sample types was high; 97.0% (32/33 samples) had identical and 3.0% (1/33 samples) compatible types (see Table S3A in the supplemental material). One woman with CIN3 who tested negative with the HPV-Risk assay with the self-collected sample but HPV16 positive with the clinician-obtained scraping specimen displayed a HPV16 signal in the self-collected sample just below the HPV-Risk assay threshold. A women with ≤CIN1 scored positive with the HPV-Risk assay (other HPV) with the self-collected sample and negative with the clinician-obtained scraping specimen. For this sample, GP5+/6+ PCR also yielded negative results for the self-collected sample.
TABLE 4.
Comparison of HPV-Risk assay results for self-collected lavage specimens and concomitantly clinician-obtained cervical scraping specimens, stratified for CIN2+
| Histology result | HPV-Risk assay result for self-collected lavage specimen | No. of clinician-obtained cervical scraping specimens with HPV-Risk assay resulta |
No. of self-collected lavage specimens with GP5+/6+ PCR resultb |
|||||
|---|---|---|---|---|---|---|---|---|
| Invalid | Negative | Positive | Total | Negative | Positive | Total | ||
| ≤CIN1 | Invalid | 1 | 1 | 1 | 1 | |||
| Negative | 9 | 13 | 22 | 22 | 22 | |||
| Positive | 3 | 1 | 12 | 16 | 1 | 15 | 16 | |
| Total | 12 | 15 | 12 | 39 | 24 | 15 | 39 | |
| CIN2+ | Negative | 1 | 1 | 2 | 1 | 1 | 2 | |
| Positive | 21 | 21 | 21 | 21 | ||||
| Total | 1 | 22 | 23 | 1 | 22 | 23 | ||
| Overall | Invalid | 1 | 1 | 1 | 1 | |||
| Negative | 9 | 14 | 1 | 24 | 23 | 1 | 24 | |
| Positive | 3 | 1 | 33 | 37 | 1 | 36 | 37 | |
| Total | 12 | 16 | 34 | 62 | 25 | 37 | 62 | |
Agreement between HPV-Risk assay results for self-collected and clinician-collected specimens was as follows: ≤CIN1, 96% (95% CI, 77 to 99%), 25/26 samples, kappa = 0.92; CIN2+, 96% (95% CI, 75 to 99%), 22/23 samples, kappa = 0.65; overall, 96% (95% CI, 85 to 99%), 47/49 samples, kappa = 0.90.
Agreement between HPV-Risk assay and GP5+/6+ PCR assay results was as follows: ≤CIN1, 97% (95% CI, 84 to 100%), 37/38 samples, kappa = 0.95; CIN2+, 96% (95% CI, 75 to 99%), 22/23 samples, kappa = 0.65; overall, 97% (95% CI, 88 to 99%), 59/61 samples, kappa = 0.93.
HPV-Risk assay with self-collected vaginal brush specimens.
The agreement between the HPV-Risk assay results for self-collected vaginal brush specimens and concomitantly clinician-obtained cervical scraping specimens for the same women was determined for a set of 112 women. For comparison, self-collected specimens were also tested with the GP5+/6+ PCR assay (Table 5; also see Table S1 in the supplemental material). The HPV-Risk assay scored 61.6% of the self-collected samples (69/112 samples) and 65.4% of the clinician-obtained scraping specimens (70/107 samples) positive (Table 5). Overall agreement in HPV-Risk assay findings between the two sampling methods was 91.6% (95% CI, 84.6 to 95.6%; 98/107 samples), with a kappa value of 0.82. Similar agreement figures were found after stratification for CIN2+ or when comparing HPV-Risk assay results with GP5+/6+ PCR assay results for self-collected brush samples. Genotype agreement between HPV-Risk assays with the two sample types was high; 87.7% (57/65 samples) had identical types and 12.3% (8/65 samples) compatible types (see Table S3B in the supplemental material). Nine women had discrepant HPV-Risk assay results for the self-collected and clinician-obtained samples (3 CIN2+ cases and 6 ≤CIN1 cases). For six of the women, the HPV status for the self-collected specimen was confirmed by the GP5+/6+ PCR assay (three HPV-negative cases and three HPV-positive cases. i.e., single infections, involving HPV16, HPV52, and HPV X). Furthermore, one GP5+/6+ PCR-negative/HPV-Risk assay-positive sample tested positive for other HPV but was just above the assay threshold, and two GP5+/6+ PCR-positive/HPV-Risk assay-negative samples displayed signals in the HPV-Risk assay just below the assay thresholds for the genotypes detected with GP5+/6+ PCR (i.e., HPV16 and other HPV, representing HPV33 by genotyping of the GP5+/6+ PCR product).
TABLE 5.
Comparison of HPV-Risk assay results for self-collected vaginal brush specimens and concomitantly clinician-obtained cervical scraping specimens, stratified for CIN2+
| Histology result | HPV-Risk assay result for self-collected brush specimen | No. of clinician-obtained cervical scraping specimens with HPV-Risk assay resulta |
No. of self-collected brush specimens with GP5+/6+ PCR resultb |
|||||
|---|---|---|---|---|---|---|---|---|
| Invalid | Negative | Positive | Total | Negative | Positive | Total | ||
| ≤CIN1 | Negative | 5 | 31 | 4 | 40 | 36 | 4 | 40 |
| Positive | 2 | 36 | 38 | 2 | 36 | 38 | ||
| Total | 5 | 33 | 40 | 78 | 38 | 40 | 78 | |
| CIN2+ | Negative | 2 | 1 | 3 | 2 | 1 | 3 | |
| Positive | 2 | 29 | 31 | 1 | 30 | 31 | ||
| Total | 4 | 30 | 34 | 3 | 31 | 34 | ||
| Overall | Negative | 5 | 33 | 5 | 43 | 38 | 5 | 43 |
| Positive | 4 | 65 | 69 | 3 | 66 | 69 | ||
| Total | 5 | 37 | 70 | 112 | 41 | 71 | 112 | |
Agreement between HPV-Risk assay results for self-collected and clinician-collected specimens was as follows: ≤CIN1, 92% (95% CI, 83 to 96%), 67/73 samples, kappa = 0.84; CIN2+, 91% (95% CI, 76 to 97%), 31/34 samples, kappa = 0.52; overall, 92% (95% CI, 85 to 96%), 98/107 samples, kappa = 0.82.
Agreement between HPV-Risk assay and GP5+/6+ PCR assay results was as follows: ≤CIN1, 92% (95% CI, 84 to 97%), 72/78 samples, kappa = 0.86; CIN2+, 94% (95% CI, 79 to 97%), 32/34 samples, kappa = 0.63; overall, 93% (95% CI, 86 to 96%), 104/112 samples, kappa = 0.86.
DISCUSSION
In this study, we reported on the analytical and clinical performance of a novel E7-based real-time PCR-based assay, i.e., the HPV-Risk assay. When applied to cervical scraping specimens, the HPV-Risk assay displayed similar clinical sensitivity and specificity for CIN2+ as did the high-risk HPV (L1-based) GP5+/6+ PCR. Together with the observed high intralaboratory reproducibility and interlaboratory agreement figures, with lower confidence bounds of >87%, and the kappa values of >0.5, these findings indicate that the HPV-Risk assay can be considered clinically validated for cervical cancer screening purposes, according to the international guidelines for HPV test requirements (8).
We also found a high agreement value for HPV (genotype) detection in self-collected and clinician-collected specimens, both for brush-based and lavage-based self-collected samples. These data support the utility of the HPV-Risk assay with these sample types. A limitation might be the number of women and the study population (i.e., outpatient clinic based), which might have hampered accurate assessment of the clinical performance of the HPV-Risk assay with self-collected specimens. Clinical validation of HPV assays for use with self-collected samples is important, given that variations in the clinical performance of HPV testing with self-collected samples have been described, likely reflecting the use of different combinations of HPV tests and collection devices (reviewed in references 16 and 30–32). For example, in a study in which two HPV tests were performed, one (Cervista) was clearly inferior with self-collected samples, compared with clinician-collected samples, in terms of CIN2+ detection, whereas the other test (PCR with matrix-assisted laser desorption ionization–time of flight mass spectrometry [MALDI-TOF MS] read out) revealed clinically comparable results for the two sample types (17). Similar results were obtained in another study, showing that the Hybrid Capture 2 assay was clinically inferior when performed on vaginal brush samples self-collected on an FTA cartridge (33).
Compared with other commercially available HPV tests that have been clinically validated for primary cervical screening with cervical scraping specimens according to the international guidelines (11–13, 34, 35), the HPV-Risk assay does not require specialized equipment for sample preparation, DNA extraction, or amplification. The assay can run on different real-time PCR platforms (e.g., Life Technologies and Bio-Rad) and is compatible with various collection media and different (automatic) DNA extraction procedures, as shown in this study.
In this study, a total of 127 women with CIN2+ were tested (i.e., 70 clinician-obtained cervical scraping specimens, 23 self-collected lavage specimens and concomitantly clinician-obtained cervical scraping specimens, and 34 self-collected brush specimens and concomitantly clinician-obtained cervical scraping specimens) with both the HPV-Risk assay and the GP5+/6+ PCR assay. Discrepant assay results were obtained for only six women, including one CIN2 case with an invalid HPV-Risk assay result. Three women tested GP5+/6+ PCR positive/HPV-Risk assay negative, i.e., one CIN2 case (HPV18 and HPV52) and two CIN3 cases (HPV16 and HPV X, i.e., a possible variant not detected with the genotyping assay). The latter two samples did yield signals just below the HPV-Risk assay threshold for HPV16 and HPV18/other HPV, respectively. The remaining two GP5+/6+ PCR-negative/HPV-Risk assay-positive cases both tested positive for other HPV and included a CIN2 case and a CIN3 case. Discrepant findings for control women without disease predominantly involved signals just above the threshold of either assay for all sample types. Collectively, these data suggest that these discrepant results are predominantly related to sampling issues for specimens with low viral loads.
Some degree of cross-reactivity with HPV70 at high copy numbers was observed. The latter can be explained by the high level of sequence homology (>95%) between genotypes 39, 68, and 70 from the alpha 7 clade in the targeted region of the E7 gene (data not shown). However, since HPV70 is considered probably carcinogenic, on the basis of epidemiological, phylogenetic, and functional studies, and the prevalence of HPV70 in the population is very low, the impact of this cross-reactivity on the performance of the assay will be very low (20, 36, 37).
In conclusion, the HPV-Risk assay meets the cross-sectional clinical and reproducibility criteria of the international guidelines for HPV test requirements and can be considered clinically validated for cervical screening purposes. The compatibility of the HPV-Risk assay with self-collected cervicovaginal lavage- and brush-based specimens supports its utility for HPV self-sampling.
Supplementary Material
ACKNOWLEDGMENTS
We thank V. Anneveld and J. Nelissen for excellent technical assistance, P. Wolffs for management assistance, and Self-Screen BV for supplying the HPV-Risk assay kits.
This work was supported by a Biotech Systems Platform (BSP) grant. The source of funding did not have any influence on the study design or the analysis of the results.
J.B. received speaker's fees from Qiagen, a company involved in HPV testing. D.A.M.H. is a shareholder and a member of the scientific staff of Self-Screen BV, a spin-off company of VU University Medical Center. All other authors declare that they have no conflicts of interest.
Footnotes
Published ahead of print 2 January 2014
Supplemental material for this article may be found at http://dx.doi.org/10.1128/JCM.03195-13.
REFERENCES
- 1.Munoz N, Bosch FX, de Sanjose S, Herrero R, Castellsague X, Shah KV, Snijders PJ, Meijer CJ. 2003. Epidemiologic classification of human papillomavirus types associated with cervical cancer. N. Engl. J. Med. 348:518–527. 10.1056/NEJMoa021641 [DOI] [PubMed] [Google Scholar]
- 2.Walboomers JM, Jacobs MV, Manos MM, Bosch FX, Kummer JA, Shah KV, Snijders PJ, Peto J, Meijer CJ, Munoz N. 1999. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J. Pathol. 189:12–19 [DOI] [PubMed] [Google Scholar]
- 3.Rijkaart DC, Berkhof J, Rozendaal L, van Kemenade FJ, Bulkmans NW, Heideman DA, Kenter GG, Cuzick J, Snijders PJ, Meijer CJ. 2012. Human papillomavirus testing for the detection of high-grade cervical intraepithelial neoplasia and cancer: final results of the POBASCAM randomised controlled trial. Lancet Oncol. 13:78–88. 10.1016/S1470-2045(11)70296-0 [DOI] [PubMed] [Google Scholar]
- 4.Sankaranarayanan R, Nene BM, Shastri SS, Jayant K, Muwonge R, Budukh AM, Hingmire S, Malvi SG, Thorat R, Kothari A, Chinoy R, Kelkar R, Kane S, Desai S, Keskar VR, Rajeshwarkar R, Panse N, Dinshaw KA. 2009. HPV screening for cervical cancer in rural India. N. Engl. J. Med. 360:1385–1394. 10.1056/NEJMoa0808516 [DOI] [PubMed] [Google Scholar]
- 5.Naucler P, Ryd W, Tornberg S, Strand A, Wadell G, Elfgren K, Radberg T, Strander B, Forslund O, Hansson BG, Rylander E, Dillner J. 2007. Human papillomavirus and Papanicolaou tests to screen for cervical cancer. N. Engl. J. Med. 357:1589–1597. 10.1056/NEJMoa073204 [DOI] [PubMed] [Google Scholar]
- 6.Mayrand MH, Duarte-Franco E, Rodrigues I, Walter SD, Hanley J, Ferenczy A, Ratnam S, Coutlee F, Franco EL. 2007. Human papillomavirus DNA versus Papanicolaou screening tests for cervical cancer. N. Engl. J. Med. 357:1579–1588. 10.1056/NEJMoa071430 [DOI] [PubMed] [Google Scholar]
- 7.Ronco G, Giorgi-Rossi P, Carozzi F, Confortini M, Dalla Palma P, Del Mistro A, Ghiringhello B, Girlando S, Gillio-Tos A, De Marco L, Naldoni C, Pierotti P, Rizzolo R, Schincaglia P, Zorzi M, Zappa M, Segnan N, Cuzick J. 2010. Efficacy of human papillomavirus testing for the detection of invasive cervical cancers and cervical intraepithelial neoplasia: a randomised controlled trial. Lancet Oncol. 11:249–257. 10.1016/S1470-2045(09)70360-2 [DOI] [PubMed] [Google Scholar]
- 8.Meijer CJ, Berkhof J, Castle PE, Hesselink AT, Franco EL, Ronco G, Arbyn M, Bosch FX, Cuzick J, Dillner J, Heideman DA, Snijders PJ. 2009. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int. J. Cancer 124:516–520. 10.1002/ijc.24010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Meijer CJ, Berkhof H, Heideman DA, Hesselink AT, Snijders PJ. 2009. Validation of high-risk HPV tests for primary cervical screening. J. Clin. Virol. 46(Suppl 3):S1–S4. 10.1016/S1386-6532(09)00540-X [DOI] [PubMed] [Google Scholar]
- 10.Ronco G, Giorgi-Rossi P, Carozzi F, Confortini M, Dalla Palma P, Del Mistro A, Gillio-Tos A, Minucci D, Naldoni C, Rizzolo R, Schincaglia P, Volante R, Zappa M, Zorzi M, Cuzick J, Segnan N. 2008. Results at recruitment from a randomized controlled trial comparing human papillomavirus testing alone with conventional cytology as the primary cervical cancer screening test. J. Natl. Cancer Inst. 100:492–501. 10.1093/jnci/djn065 [DOI] [PubMed] [Google Scholar]
- 11.Heideman DA, Hesselink AT, Berkhof J, van Kemenade F, Melchers WJ, Daalmeijer NF, Verkuijten M, Meijer CJ, Snijders PJ. 2011. Clinical validation of the cobas 4800 HPV test for cervical screening purposes. J. Clin. Microbiol. 49:3983–3985. 10.1128/JCM.05552-11 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hesselink AT, Heideman DA, Berkhof J, Topal F, Pol RP, Meijer CJ, Snijders PJ. 2010. Comparison of the clinical performance of PapilloCheck human papillomavirus detection with that of the GP5+/6+-PCR-enzyme immunoassay in population-based cervical screening. J. Clin. Microbiol. 48:797–801. 10.1128/JCM.01743-09 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hesselink AT, Meijer CJ, Poljak M, Berkhof J, van Kemenade FJ, van der Salm ML, Bogaarts M, Snijders PJ, Heideman DA. 2013. Clinical validation of the Abbott RealTime High Risk HPV assay according to the guidelines for human papillomavirus DNA test requirements for cervical screening. J. Clin. Microbiol. 51:2409–2410. 10.1128/JCM.00633-13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gok M, Heideman DA, van Kemenade FJ, Berkhof J, Rozendaal L, Spruyt JW, Voorhorst F, Belien JA, Babovic M, Snijders PJ, Meijer CJ. 2010. HPV testing on self collected cervicovaginal lavage specimens as screening method for women who do not attend cervical screening: cohort study. BMJ 340:c1040. 10.1136/bmj.c1040 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Gok M, van Kemenade FJ, Heideman DA, Berkhof J, Rozendaal L, Spruyt JW, Belien JA, Babovic M, Snijders PJ, Meijer CJ. 2012. Experience with high-risk human papillomavirus testing on vaginal brush-based self-samples of non-attendees of the cervical screening program. Int. J. Cancer 130:1128–1135. 10.1002/ijc.26128 [DOI] [PubMed] [Google Scholar]
- 16.Snijders PJ, Verhoef VM, Arbyn M, Ogilvie G, Minozzi S, Banzi R, van Kemenade FJ, Heideman DA, Meijer CJ. 2013. High-risk HPV testing on self-sampled versus clinician-collected specimens: a review on the clinical accuracy and impact on population attendance in cervical cancer screening. Int. J. Cancer 132:2223–2236. 10.1002/ijc.27790 [DOI] [PubMed] [Google Scholar]
- 17.Belinson JL, Du H, Yang B, Wu R, Belinson SE, Qu X, Pretorius RG, Yi X, Castle PE. 2012. Improved sensitivity of vaginal self-collection and high-risk human papillomavirus testing. Int. J. Cancer 130:1855–1860. 10.1002/ijc.26202 [DOI] [PubMed] [Google Scholar]
- 18.van den Brule AJ, Pol R, Fransen-Daalmeijer N, Schouls LM, Meijer CJ, Snijders PJ. 2002. GP5+/6+ PCR followed by reverse line blot analysis enables rapid and high-throughput identification of human papillomavirus genotypes. J. Clin. Microbiol. 40:779–787. 10.1128/JCM.40.3.779-787.2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Vinokurova S, Wentzensen N, Kraus I, Klaes R, Driesch C, Melsheimer P, Kisseljov F, Durst M, Schneider A, von Knebel Doeberitz M. 2008. Type-dependent integration frequency of human papillomavirus genomes in cervical lesions. Cancer Res. 68:307–313. 10.1158/0008-5472.CAN-07-2754 [DOI] [PubMed] [Google Scholar]
- 20.IARC Working Group on the Evaluation of Carcinogenic Risks to Humans 2009. Human papillomaviruses. IARC Monogr. 100B:255–313 [Google Scholar]
- 21.de Sanjose S, Quint WG, Alemany L, Geraets DT, Klaustermeier JE, Lloveras B, Tous S, Felix A, Bravo LE, Shin HR, Vallejos CS, de Ruiz PA, Lima MA, Guimera N, Clavero O, Alejo M, Llombart-Bosch A, Cheng-Yang C, Tatti SA, Kasamatsu E, Iljazovic E, Odida M, Prado R, Seoud M, Grce M, Usubutun A, Jain A, Suarez GA, Lombardi LE, Banjo A, Menendez C, Domingo EJ, Velasco J, Nessa A, Chichareon SC, Qiao YL, Lerma E, Garland SM, Sasagawa T, Ferrera A, Hammouda D, Mariani L, Pelayo A, Steiner I, Oliva E, Meijer CJ, Al-Jassar WF, Cruz E, Wright TC, Puras A, Llave CL, Tzardi M, Agorastos T, Garcia-Barriola V, Clavel C, Ordi J, Andujar M, Castellsague X, Sanchez GI, Nowakowski AM, Bornstein J, Munoz N, Bosch FX. 2010. Human papillomavirus genotype attribution in invasive cervical cancer: a retrospective cross-sectional worldwide study. Lancet Oncol. 11:1048–1056. 10.1016/S1470-2045(10)70230-8 [DOI] [PubMed] [Google Scholar]
- 22.Koopman LA, Szuhai K, van Eendenburg JD, Bezrookove V, Kenter GG, Schuuring E, Tanke H, Fleuren GJ. 1999. Recurrent integration of human papillomaviruses 16, 45, and 67 near translocation breakpoints in new cervical cancer cell lines. Cancer Res. 59:5615–5624 [PubMed] [Google Scholar]
- 23.Khan MJ, Castle PE, Lorincz AT, Wacholder S, Sherman M, Scott DR, Rush BB, Glass AG, Schiffman M. 2005. The elevated 10-year risk of cervical precancer and cancer in women with human papillomavirus (HPV) type 16 or 18 and the possible utility of type-specific HPV testing in clinical practice. J. Natl. Cancer Inst. 97:1072–1079. 10.1093/jnci/dji187 [DOI] [PubMed] [Google Scholar]
- 24.Berkhof J, Bulkmans NW, Bleeker MC, Bulk S, Snijders PJ, Voorhorst FJ, Meijer CJ. 2006. Human papillomavirus type-specific 18-month risk of high-grade cervical intraepithelial neoplasia in women with a normal or borderline/mildly dyskaryotic smear. Cancer Epidemiol. Biomarkers Prev. 15:1268–1273. 10.1158/1055-9965.EPI-05-0764 [DOI] [PubMed] [Google Scholar]
- 25.Dijkstra MG, Heideman DA, van Kemenade FJ, Hogewoning KJ, Hesselink AT, Verkuijten MC, van Baal WM, Boer GM, Snijders PJ, Meijer CJ. 2012. Brush-based self-sampling in combination with GP5+/6+-PCR-based hrHPV testing: high concordance with physician-taken cervical scrapes for HPV genotyping and detection of high-grade CIN. J. Clin. Virol. 54:147–151. 10.1016/j.jcv.2012.02.022 [DOI] [PubMed] [Google Scholar]
- 26.Brink AA, Meijer CJ, Wiegerinck MA, Nieboer TE, Kruitwagen RF, van Kemenade F, Fransen Daalmeijer N, Hesselink AT, Berkhof J, Snijders PJ. 2006. High concordance of results of testing for human papillomavirus in cervicovaginal samples collected by two methods, with comparison of a novel self-sampling device to a conventional endocervical brush. J. Clin. Microbiol. 44:2518–2523. 10.1128/JCM.02440-05 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.de Roda Husman AM, Snijders PJ, Stel HV, van den Brule AJ, Meijer CJ, Walboomers JM. 1995. Processing of long-stored archival cervical smears for human papillomavirus detection by the polymerase chain reaction. Br. J. Cancer 72:412–417 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Schmitt M, Bravo IG, Snijders PJ, Gissmann L, Pawlita M, Waterboer T. 2006. Bead-based multiplex genotyping of human papillomaviruses. J. Clin. Microbiol. 44:504–512. 10.1128/JCM.44.2.504-512.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Tang NS, Tang ML, Chan IS. 2003. On tests of equivalence via non-unity relative risk for matched-pair design. Stat. Med. 22:1217–1233. 10.1002/sim.1213 [DOI] [PubMed] [Google Scholar]
- 30.Bhatla N, Singla S, Awasthi D. 2012. Human papillomavirus deoxyribonucleic acid testing in developed countries. Best Pract. Res. Clin. Obstet. Gynaecol. 26:209–220. 10.1016/j.bpobgyn.2011.11.003 [DOI] [PubMed] [Google Scholar]
- 31.Cuzick J, Bergeron C, von Knebel Doeberitz M, Gravitt P, Jeronimo J, Lorincz AT, Meijer LM, Sankaranarayanan R, Snijders JF, Szarewski A. 2012. New technologies and procedures for cervical cancer screening. Vaccine 30(Suppl 5):F107–F116. 10.1016/j.vaccine.2012.05.088 [DOI] [PubMed] [Google Scholar]
- 32.Zhao FH, Lewkowitz AK, Chen F, Lin MJ, Hu SY, Zhang X, Pan QJ, Ma JF, Niyazi M, Li CQ, Li SM, Smith JS, Belinson JL, Qiao YL, Castle PE. 2012. Pooled analysis of a self-sampling HPV DNA test as a cervical cancer primary screening method. J. Natl. Cancer Inst. 104:178–188. 10.1093/jnci/djr532 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.de Bie RP, Schmeink CE, Bakkers JM, Snijders PJ, Quint WG, Massuger LF, Bekkers RL, Melchers WJ. 2011. The indicating FTA elute cartridge a solid sample carrier to detect high-risk HPV and high-grade cervical lesions. J. Mol. Diagn. 13:371–376. 10.1016/j.jmoldx.2011.02.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Heideman DA, Hesselink AT, van Kemenade FJ, Iftner T, Berkhof J, Topal F, Agard D, Meijer CJ, Snijders PJ. 2013. The Aptima HPV assay fulfills the cross-sectional clinical and reproducibility criteria of international guidelines for HPV test requirements for cervical screening. J. Clin. Microbiol. 51:3653–3657. 10.1128/JCM.01517-13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Belinson JL, Wu R, Belinson SE, Qu X, Yang B, Du H, Wu R, Wang C, Zhang L, Zhou Y, Liu Y, Pretorius RG. 2011. A population-based clinical trial comparing endocervical high-risk HPV testing using Hybrid Capture 2 and Cervista from the SHENCCAST II Study. Am. J. Clin. Pathol. 135:790–795. 10.1309/AJCPKA6ATAPBZ6JQ [DOI] [PubMed] [Google Scholar]
- 36.Doorbar J, Quint W, Banks L, Bravo IG, Stoler M, Broker TR, Stanley MA. 2012. The biology and life-cycle of human papillomaviruses. Vaccine 30(Suppl 5):F55–F70. 10.1016/j.vaccine.2012.06.083 [DOI] [PubMed] [Google Scholar]
- 37.Hiller T, Poppelreuther S, Stubenrauch F, Iftner T. 2006. Comparative analysis of 19 genital human papillomavirus types with regard to p53 degradation, immortalization, phylogeny, and epidemiologic risk classification. Cancer Epidemiol. Biomarkers Prev. 15:1262–1267. 10.1158/1055-9965.EPI-05-0778 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.