Abstract
Internationally comparable quality assurance of Human Papillomavirus (HPV) DNA detection and typing methods is essential for evaluation of HPV vaccines and effective monitoring and implementation of HPV vaccination programs. Therefore, the World Health Organization (WHO) HPV Laboratory Network (LabNet) designed an international proficiency study. Following announcement at the WHO website, the responding laboratories performed HPV typing using one or more of their usual assays on 43 coded samples composed of titration series of purified plasmids of 16 HPV types (HPV6, -11, -16, -18, -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, -66, and -68). Detection of at least 50 IU of HPV16 or HPV18 DNA and of 500 genome equivalents (GE) of the other 14 HPV types (in samples with single and multiple HPV types) was considered proficient. Fifty-four laboratories worldwide submitted a total of 84 data sets. More than 21 HPV-genotyping assays were used. Commonly used methods were Linear Array, Lineblot, InnoLiPa, Clinical Array, type-specific real-time PCR, PCR-Luminex and microarray assays. The major oncogenic HPV types (HPV16 and -18) were detected in 89.7% (70/78) and 92.2% (71/77) of the data sets, respectively. HPV types 56, 59, and 68 were the least commonly detected types (in less than 80% of the data sets). Twenty-eight data sets reported multiple false-positive results and were considered nonproficient. In conclusion, we found that international proficiency studies, traceable to international standards, allow standardized quality assurance for different HPV-typing assays and enable the comparison of data generated from different laboratories worldwide.
Human papillomavirus (HPV) infection has been established as the major cause of cervical cancer (2). Epidemiological studies have classified genital HPV types into high- and low-risk HPV types, reflecting their association with invasive cancer (19). The most important high-risk types, HPV16 and HPV18, account for about 70% of all invasive cervical cancers worldwide. The next most common HPV types on all continents are HPV31, -33, -35, -45, -52, and -58, found in approximately 20% of cervical cancers (19).
An accurate and internationally comparable HPV DNA detection and typing methodology is an essential component in the evaluation of HPV vaccines and in effective implementation and monitoring of HPV vaccination programs. The genotyping assays used today differ in their performance with regard to type-specific detection rates (10). As the methodology for quality assurance and evaluation of assay performance is not standardized, comparisons between different studies that use different assays is particularly difficult (10).
The World Health Organization (WHO) establishes international biological standard materials and reference reagents for substances of biological origin used in prophylaxis and in therapy or diagnosis of human diseases (http://www.who.int/biologicals/reference_preparations/en/). At the WHO meeting held in Geneva, Switzerland, from 15 to 17 August 2005, an expert group recommended the establishment of a global HPV laboratory network (HPV LabNet) to contribute to improving the quality of laboratory services for effective surveillance and HPV vaccination impact monitoring. Major activities within the HPV LabNet include the development of international standard reagents and standard operating procedures (SOPs) and the development of internationally comparable quality assurance methods (5, 26).
International proficiency panels are already widely used for several microorganisms, including hepatitis A, B, and C viruses; herpes simplex virus (HSV); and human immunodeficiency virus (HIV) (15, 18, 24). As there is no natural source of biological material that could be used to generate type-specific HPV international standards (ISs), the first WHO international collaborative study of the detection of HPV DNA examined the feasibility of using recombinant HPV DNA plasmids as standards, focusing on HPV16 and HPV18 (13). ISs of HPV16 and HPV18 DNA were established for detection and quantification of HPV16 and HPV18 DNA by the WHO Expert Committee on Biological Standardization in 2008 with assigned potency in international units (IU).
The international WHO proficiency study described in this report was based on a proficiency panel composed of purified plasmids containing the genomes of 14 oncogenic HPV types and 2 benign HPV types. As the amount of plasmid DNA was titrated in amounts traceable to the IS, the proficiency panel allowed an internationally standardized definition of assay sensitivity. Specificity was defined as an absence of incorrect typing. We also evaluated sample preprocessing with extraction controls of cervical cancer cell lines. The panel was distributed to 61 laboratories worldwide and analyzed using a range of HPV DNA-typing assays in a blinded manner. We report the results in terms of the ability of participating laboratories to correctly identify HPV types, grouped by methods performed, as well as the analytical sensitivity of detecting the HPV types included.
MATERIALS AND METHODS
Sources of panel material.
Complete genomes of HPV cloned into plasmid vectors had been provided to Lund University by the respective proprietors with written approval for their use in this proficiency panel: Ethel-Michele de Villiers (HPV types 6, 11, 16, 18, and 45), Gérard Orth (HPV types 33, 39, 66, and 68), Saul Silverstein (HPV type 51), Attila Lörincz (HPV types 31, 35, and 56), Wayne Lancaster (HPV type 52), and Toshihiko Matsukura (HPV types 58 and 59). The agreements allowed distribution of the plasmids only for the performance of this WHO proficiency study.
The nucleic acid sequences for each of the HPV genomes have been reported previously and are available in GenBank with the following accession numbers; HPV6, X00203; HPV11, M14119; HPV16, K02718; HPV18, X05015; HPV31, J04353; HPV33, M12732; HPV35, M74117; HPV39, M62849; HPV45, X74479; HPV51, M62877; HPV52, X74481; HPV56, X74483; HPV58, D90400; HPV59, X77858; HPV66, U31794; and HPV68, X67161.
Preparation and characterization of individual panel reagents.
HPV11 and HPV58 were originally cloned in the L1 gene and were therefore recloned so that the vector (pGEM4z) was positioned in the L2 (position 4781) and the E1 (position 1158) genes, respectively. For HPV35, two clones were included: HPV35-S, containing all the genes from L1 through E7, including nucleotides 5012 to 956, and HPV35-L, including nucleotides 956 to 5012. The plasmid used for HPV68 contained only the L1 gene. The DNA of each individual HPV genome was generated by overnight culture of transformed Escherichia coli and plasmid purification using a Qiagen Midi-prep kit. Optical density determinations were made at 260 nm and 280 nm to estimate the purity of the preparation. The sizes and purities of the plasmids were analyzed using agarose gel electrophoresis. The double-stranded-DNA (dsDNA) concentration was established using fluorimetric measurements by PicoGreen quantitations (PicoGreen dsDNA Quantitation Reagent; Molecular Probes, Inc., Eugene, OR). The purified plasmid bulks of HPV16 and HPV18 were tested in 10-fold serial dilutions in parallel with international standards for HPV16 (06/202) and HPV18 (06/206) distributed by NIBSC (Hertfordshire, United Kingdom), using a PCR Luminex assay to establish the amounts in international units by traceability of the amount of plasmids in the panel to the IS (16).
Panel composition and production.
Purified plasmids containing cloned genomic DNAs for HPV types 6, 11, 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68 were diluted to a stock concentration of 108 genome equivalents/μl in TE buffer (10 mM Tris, 1 mM EDTA, pH 8.0) to be used for preparation of 43 samples. Human placenta DNA (Sigma-Aldrich; no. 7011) at a concentration of 10 ng/μl was added to the TE buffer to mimic a molecular-matrix background that would typically be present in biological samples. Table 1 summarizes the composition of the panel. The different amounts of plasmid (5 to 500 GE or IU) were chosen to reflect the lower spectrum of the amounts of virus that would typically be present in clinical samples. For example, a study of virus quantities present in cervical samples from healthy HPV-positive women found an average of 18,000 GE of HPV16/100 ng input DNA (range, <300 to 14,000,000 GE/100 ng input DNA) (20). The 43 different panel samples were prepared by dilution of HPV recombinant DNA plasmid stock solution in TE buffer in the background of human placental DNA. Briefly, the HPV DNA plasmids were diluted 100-fold in TE-placenta buffer to 104 GE/μl, and further 10-fold dilutions were made to a final concentration of 1 IU/μl of HPV16 and HPV18; for the other HPV types included, 10 GE/μl was the final dilution. To ensure the high quality of the panel, two HPV types were diluted each day with an interval of at least 4 h in between. The samples containing multiple types were prepared from dilutions of 103 genome equivalents/μl. After production of each of the 43 reference samples, the preparation was dispensed in 100-μl volumes in 1.5 ml siliconized vials. The vials were labeled as WHO HPV DNA 2008 and randomly assigned numbers from 1 through 43. The panels were stored at +4°C before shipment to participating laboratories. The participants were instructed to perform HPV typing according to their standard methods using their standard sample input volumes.
TABLE 1.
HPV DNA proficiency panel composition and HPV testing results
| HPV type(s) | No. of HPV GE/5 μla | % Correct data setsb (no./total) |
|---|---|---|
| 16 | 500 | 94.9 (74/78) |
| 50 | 89.7 (70/78) | |
| 5 | 67.9 (53/78) | |
| 18 | 500 | 92.2 (71/77) |
| 50 | 92.2 (71/77) | |
| 5 | 59.7 (46/77) | |
| 6 | 500 | 91.3 (63/69) |
| 50 | 81.1 (56/69) | |
| 11 | 500 | 88.4 (61/69) |
| 50 | 94.2 (65/69) | |
| 31 | 500 | 86.4 (64/74) |
| 50 | 67.6 (50/74) | |
| 33 | 500 | 90.5 (67/74) |
| 50 | 86.5 (64/74) | |
| 35 | 500 | 86.5 (64/74) |
| 50 | 78.4 (58/74) | |
| 39 | 500 | 90.5 (38/42) |
| 50 | 69.0 (29/42) | |
| 45 | 500 | 89.0 (65/73) |
| 50 | 80.8 (59/73) | |
| 51 | 500 | 88.9 (64/72) |
| 50 | 75 (54/72) | |
| 52 | 500 | 85.1 (63/74) |
| 50 | 78.4 (58/74) | |
| 56 | 500 | 75.3 (55/73) |
| 50 | 68.5 (50/73) | |
| 58 | 500 | 90.5 (67/74) |
| 50 | 75.7 (56/74) | |
| 59 | 500 | 72.6 (53/73) |
| 50 | 65.7 (48/73) | |
| 66 | 500 | 84.6 (55/65) |
| 50 | 77.3 (51/65) | |
| 68 | 500 | 37.9 (11/29) |
| 50 | 34.4 (10/29) | |
| 16, 45, 52, 33 | 500 | 58.9 (46/78)c |
| 50 | 47.4 (37/78)c | |
| 11, 18, 31, 51 | 500 | 72.7(56/77)c |
| 50 | 59.7 (46/77)c | |
| 35, 39, 59, 66 | 500 | 59.7(44/74)d |
| 50 | 50 (37/74)d | |
| 6, 56, 58, 68 | 500 | 50.0(37/74)d |
| 50 | 41.9 (31/74)d | |
| None | 0 | 92.5 (74/80) |
| HPV16 cervical cancer cells | 2,000 cells | 34.3 (23/67) |
| HPV-negative cells | 0 | 65.7 (44/67) |
The plasmid concentration that is equivalent to 50 genome copies (IU) varied from 0.53 fg to 0.67 fg/5 μl because of small variations in the length of the HPV genome and the use of different cloning vectors. HPV68 had only an L1 plasmid, and the plasmid concentration equivalent to 50 genome copies was therefore 0.23 fg/5 μl. The background concentration of human DNA in all samples was 50 ng/5 μl.
Data sets detecting the correct type as claimed with no false-positive type detected. HPV16 cervical cancer cells had 3 false positives; HPV-negative cells had 6 false positives and 17 invalid results. The values corresponding to the highest concentration for each type are shown in boldface type.
Including data sets generated by type-specific HPV16/HPV18 PCR.
Data sets known not to detect HPV39 or HPV68 were considered correct when the other HPV types in the sample were detected.
Two different cell lines were used as controls for the extraction process in participating laboratories. The HPV-negative epithelial cell line C33A, derived from human cervical carcinoma, and the HPV16-positive epithelial cell line SiHa, derived from a squamous cell carcinoma, were purchased from the American Type Culture Collection and cultured in Dulbecco's modified Eagle medium (Gibco; no. 11960). The cells were diluted in PreserveCyt (Cytyc; no. 0234004) to a concentration of 400 cells/μl, and 100 μl of each preparation was dispensed in 1.5-ml vials and labeled WHO HPV DNA A and B.
Before distribution of the WHO HPV DNA proficiency panel, the samples were tested (blinded) at the WHO HPV LabNet Global Reference Laboratory (GRL) in Sweden and two other laboratories, namely, the German Cancer Research Center (DKFZ) in Heidelberg, Germany (Michael Pawlita), and the WHO HPV LabNet GRL at the Centers for Disease Control and Prevention (CDC) in the United States (Elizabeth Unger).
Technologies used for initial characterization of the panel. (i) GRL Sweden.
Three independent experiments testing each sample in duplicate were performed. Five microliters of panel sample DNAs was used for modified general primer (MGP) PCR as previously described (21). Ten microliters of PCR products was analyzed by multiplex genotyping using a Luminex-based assay as described previously (16, 17). HPV types 6, 11, 16, 18, 26, 30, 31, 33, 35, 39, 40, 42, 43, 45, 51, 52, 53, 54, 56, 58, 59, 66, 67, 68a, 69, 70, 73, 74, 82, 86, 89, 90, and 91 were distinguished. Appropriate negative and positive controls were used to monitor the performance of the method. DNA from the extraction controls A and B was extracted using a QIAamp DNA Mini and Blood kit (Qiagen) according to the manufacturer's instructions.
(ii) DKFZ.
A 10-μl DNA sample was amplified by the broad-spectrum GP5+/6+ primers as previously described (17). The PCR products were analyzed using bead-based multiplex genotyping as described previously (16). HPV types 6, 11, 16, 18, 26, 30, 31, 33, 35, 39, 42, 43, 44, 45, 51, 52, 53, 56, 58, 59, 66, 67, 68a, 68b (Me 180), 69, 70, 73, and 82 were distinguished. All samples were tested for human DNA with PCR primers amplifying part of the β-globin gene and a bead-coupled β-globin-specific probe used in the genotyping assay.
(iii) GRL CDC.
Ten microliters of sample DNAs was used in the 100-μl PCR; otherwise, the manufacturer's protocol for the Roche Linear Array, which is designed to detect 37 individual HPV types, 6, 11, 16, 18, 26, 31, 33, 35, 39, 40, 42, 45, 51, 52, 53, 54, 55, 56, 58, 59, 61, 62, 64, 66, 67, 68, 69, 70, 71, 72, 73, 81, 82, 83, 84, 89, and IS39, was followed. As the probe for detecting HPV52 cross-hybridizes to types 33, 35, and 58, the presence of HPV52 in samples with one or more of these three other types was tested with an HPV52-specific real-time PCR assay.
Organization of the study.
Participants in the study were recruited by advertisement at the WHO website. The panels were distributed from the WHO HPV LabNet GRL in Sweden at ambient temperature to 61 laboratories worldwide by WHO region: America Region, 16 laboratories; Africa Region, 1 laboratory; Eastern Mediterranean Region, 1 laboratory; European Region, 28 laboratories; Southeast Asia Region, 2 laboratories; and Western Pacific Region, 13 laboratories. The package also included a letter of instruction, as well as a form for reporting the results of the testing of the panel and technical information on the procedures to be performed. The laboratories were asked to submit the results of the tests performed to the WHO GRL in Sweden within 4 weeks of receipt of the specimens. The agreement included assigning the right to publish the data to the WHO, but it was agreed that only coded results from all laboratories would be presented, grouped by the methods performed.
All results submitted to the WHO HPV LabNet GRL Sweden were coded and analyzed anonymously. The data sets generated were designated numerically from 1 through 84. Individual results of the proficiency study were disclosed only to the participating laboratory that generated the data.
HPV technologies used by study participants.
The different HPV-typing methods that were used to generate results for the WHO LabNet proficiency study to detect HPV DNA (1, 3, 6-9, 17, 21, 22, 25) are summarized in Table 2.
TABLE 2.
Proficiency of detecting HPV types tested for, by laboratorya
| HPV assay typeb | No. of data sets | HPV region targeted (primer) | No. of proficient data sets |
||||
|---|---|---|---|---|---|---|---|
| 100% proficient | 99-90% proficient | 89-80% proficient | <80% proficient | Not proficient | |||
| All assays | 73 | L1/E1/E6/E7 | 19 | 10 | 5 | 11 | 28 |
| Linear Array (Roche) | 15 | L1 (PGMY) | 6 | 5 | 0 | 0 | 4 |
| CLART HPV 2 (Genomica) | 6 | L1 (PGMY) | 0 | 0 | 2 | 0 | 4 |
| InnoLiPA (Innogenetics) | 6 | L1 (SPF10) | 0 | 1 | 1 | 0 | 4 |
| PGMY-Lineblot | 6 | L1 (PGMY) | 1 | 1 | 0 | 2 | 2 |
| In-house type-specific PCR | 6 | L1/E6/E7 | 2 | 0 | 0 | 1 | 3 |
| DNA chip (Biocore) | 4 | L1 | 0 | 0 | 0 | 3 | 1 |
| In-house Lineblot (Snijders) | 4 | L1 (GP) | 0 | 1 | 0 | 2 | 1 |
| In house PCR Luminex | 4 | L1 (GP) | 3 | 0 | 0 | 0 | 1 |
| In house PCR Luminex | 4 | E6/E7 | 2 | 0 | 0 | 0 | 2 |
| In-house microarray | 3 | L1/E7 | 1 | 0 | 0 | 0 | 2 |
| PCR-RFLP | 3 | L1 | 0 | 0 | 0 | 2 | 1 |
| Microarray (Genetel) | 2 | L1 | 2 | 0 | 0 | 0 | 0 |
| DEIA LiPA assays (DDL) | 2 | L1 (SPF 10) | 0 | 0 | 0 | 0 | 2 |
| In house PCR EIA | 2 | L1 | 0 | 0 | 1 | 0 | 1 |
| Papillocheck microarray | 1 | E1 | 1 | 0 | 0 | 0 | 0 |
| Type-specific PCR (GenoID) | 1 | L1 | 0 | 1 | 0 | 0 | 0 |
| In-house PCR Luminex | 1 | L1 (PGMY-GP) | 0 | 0 | 0 | 1 | 0 |
| PCR Luminex (Multimetrix) | 1 | L1 (GP) | 0 | 0 | 1 | 0 | 0 |
| PCR EIA (GenoID) | 1 | L1 | 0 | 1 | 0 | 0 | 0 |
| In-house PCR sequencing | 1 | L1 (PGMY-GP) | 1 | 0 | 0 | 0 | 0 |
Restricted to assays testing for more than 12 types.
RFLP, restriction fragment length polymorphism; EIA, enzyme immunoassay.
Data analysis.
Criteria used for considering a data set to be proficient were as follows: (i) detection of at least 50 IU per 5 μl of HPV16 and HPV18 in both single and multiple HPV infections, (ii) detection of at least 500 GE per 5 μl of the other HPV types included in both single and multiple infections, and (iii) at most one false-positive result. These criteria were arrived at by a consensus opinion of international experts participating in an international WHO workshop in Geneva, Switzerland, in 2008 (5) and were based on a consideration of which performance requirements were required and realistic. A higher requirement for HPV16 and -18 was considered essential because of the pivotal role of these HPV types in causing cervical cancer.
Four data sets reporting results only as “high-risk” or “low-risk” HPV were not included in the overall performance analyses (one data set that used the Roche Amplicor assay, one data set that used the Seeplex HPV 4 ACE assay, and two data sets that used in-house PCR with agarose gel analysis).
RESULTS
Validation of HPV proficiency panel.
The results from the initial panel validation at the 2 GRLs and at DKFZ included qualitative characterization of HPV and human genomic DNAs. Two of these laboratories used Luminex-based assays with modified GP5+/6+ primers, and the third laboratory used Linear Array, which is based on PGMY primers, for the analyses. No false-positive HPV type was detected in the samples at any of the reference laboratories. HPV31 was not detected at the lowest concentration, when present together with other plasmids, in both laboratories that used GP5+/6+-based assays. HPV18 was not detected at the lowest concentration in the laboratory using Linear Array. All other HPV types were detected at the lowest concentration included in the panel, except HPV39 and HPV68, which could not be detected using Linear Array. The HPV39 plasmid used in the panel cannot be detected by systems using PGMY primers, as it was cloned into the vector at the binding site of one of the PGMY primers. Linear Array and other PGMY-based assays are designed to detect HPV68 subtype b and cannot detect the HPV68 prototype virus because of several mismatches.
All 3 reference laboratories detected HPV16 DNA in the DNA extraction control containing SiHa cells and had negative results in the negative control for DNA extraction (C33A cells).
The results from the reference laboratory evaluation advised that the panel performed as expected, and the panel was then distributed to participating laboratories worldwide.
Panel distribution and response.
Fifty-four of 61 participating laboratories, including the three laboratories that did the panel validation, submitted 84 data sets according to the timeline (Table 2). Two laboratories that responded after the deadline are not included in this report. Four data sets were generated using assays that did not discriminate specific HPV types and were therefore not included in the overall type-specific analyses presented here.
Some participating laboratories did not perform tests for typing of all HPV types included in the proficiency panel. Therefore, the denominator for the number of test results included in the analyses varies for the different HPV types. In 37 data sets, the results had been obtained using commercially available tests. The most commonly used assay was Linear Array (Roche), which was used to generate 15 data sets. Other widely used assays were CLART HPV 2 (Genomica), InnoLiPA (Innogenetics), PGMY-Lineblot, and in-house type-specific PCR, Luminex-, and microarray-based assays (Table 2). The participating laboratories included public health laboratories, research laboratories, diagnostic kit manufacturers, and vaccine companies. The number of samples analyzed for HPV per laboratory varied from 100 to 100,000 per year, with approximately 40% of the laboratories performing <2,000 HPV-typing tests per year and around 40% performing between 2,000 and 10,000 HPV typings per year.
Performance of HPV assays and participating laboratories.
Participating laboratories were requested to perform testing using their standard protocols. Accordingly, the input volumes of the DNA panel varied between 2 μl and 50 μl between laboratories. Data are presented by the lowest category of concentration (5, 50, or 500 GE or IU) proven to be detectable; e.g., a laboratory using a 2-μl input instead of a 5-μl input that does detect 2 GE is considered to be able to detect 5 GE. The sample containing 100 IU HPV16/μl was the sample that most data sets (94.9%) identified correctly (Table 1). Single HPV types in 100 GE/μl were correctly identified, without false-positive types detected, in an average of 84% of the data sets. HPV56 and -59 were correctly identified by less than 80% of the data sets; HPV68 was correctly identified by only 37.9% of laboratories. In the samples containing multiple HPV types, between 50% and 73% of the data sets could correctly identify the types. The negative-control sample containing only human genomic DNA was correctly identified as negative by 74 of 80 data sets.
The proficiencies in detecting HPV types (restricted to data sets testing for more than 12 HPV types) are shown in Table 2. Nineteen data sets were 100% proficient (detecting at least 50 IU of HPV16 and HPV18 in 5 μl and 500 GE in 5 μl of the other HPV types tested for [also when present together with other HPV types] without having more than one false-positive result). As the Linear Array assays used a large (50-μl) input volume, the Linear Array system did not test for the presence of amounts below 50 IU of HPV16 and HPV18 in 5 μl and 500 genome equivalents in 5 μl of the other HPV types. Two different Microarray assays were the commercial tests that had the highest number of proficient results (100%). Several in-house assays based on type-specific PCR and on general-primer PCR-Luminex were also 100% proficient.
The noncommercial PGMY-Lineblot assay was transferred to all WHO HPV LabNet members in 2008 in an effort to build up testing capacity and to evaluate the ease of technology transfer of this assay. The PGMY-Lineblot assay was used by seven members of the WHO HPV LabNet, but with 100% proficiency in only one laboratory. Only one laboratory (the originator) had been routinely using this assay before, and the other laboratories had recently set up the assay according to instructions. Indeed, when a subsequent, similar proficiency panel was sent to the WHO HPV LabNet members, two additional laboratories using PGMY-Lineblot were 100% proficient and one additional laboratory was 88% proficient (data not shown).
To be considered proficient in this study, no more than one false-positive sample per data set was acceptable. The numbers of false-positive HPV types detected per data set are shown in Table 3. Thirty-four of the 80 data sets did not have any false-positive results, whereas 12 data sets reported more than 3 false-positive results. Among these, 3 data sets reported false-positive HPV types in more than 15 samples. Data sets generated by the commercial tests CLART and InnoLiPA reported more than one false-positive sample in 4 out of 6 datasets. Several in-house assays, as well as some commercial assays that were performed by only one or only a few laboratories, reported no false-positive results at all.
TABLE 3.
False-positive HPV types detected, by assaya
| HPV assay typeb | No. of data sets | HPV region targeted (primer) | No. of data sets with false-positive samples |
||||
|---|---|---|---|---|---|---|---|
| 0 samples | 1 sample | 2 samples | 3 samples | >3 samples | |||
| All assays | 80 | L1/E1/E6/E7 | 34 | 16 | 9 | 9 | 12 |
| Linear Array (Roche) | 15 | L1 (PGMY) | 6 | 5 | 2 | 2 | 0 |
| CLART HPV 2 (Genomica) | 6 | L1 (PGMY) | 1 | 1 | 1 | 0 | 3 |
| InnoLiPA (Innogenetics) | 6 | L1 (SPF10) | 1 | 1 | 0 | 2 | 2 |
| PGMY-Lineblot | 6 | L1 (PGMY) | 3 | 0 | 0 | 3 | 0 |
| In-house type-specific PCR | 7 | L1/E6/E7 | 1 | 3 | 1 | 0 | 2 |
| In-house 16/18-specific PCR | 6 | E6/E7 | 5 | 0 | 1 | 0 | 0 |
| DNA chip (Biocore) | 4 | L1 | 1 | 2 | 0 | 1 | 0 |
| In-house Lineblot (Snijders) | 4 | L1 (GP) | 2 | 1 | 0 | 0 | 1 |
| In house PCR Luminex | 4 | L1 (GP) | 3 | 0 | 0 | 0 | 1 |
| In house PCR Luminex | 4 | E6/E7 | 2 | 0 | 1 | 0 | 1 |
| In-house microarray | 3 | L1/E7 | 0 | 1 | 1 | 0 | 1 |
| PCR-RFLP | 3 | L1 | 1 | 1 | 1 | 0 | 0 |
| Microarray (Genetel) | 2 | L1 | 2 | 0 | 0 | 0 | 0 |
| DEIA LiPA assays (DDL) | 2 | L1 (SPF 10) | 0 | 0 | 1 | 1 | 0 |
| In house PCR EIA | 2 | L1 | 0 | 1 | 0 | 0 | 1 |
| Papillocheck microarray | 1 | E1 | 1 | 0 | 0 | 0 | 0 |
| Type-specific PCR (GenoID) | 1 | L1 | 1 | 0 | 0 | 0 | 0 |
| In-house PCR Luminex | 1 | L1 (PGMY-GP) | 1 | 0 | 0 | 0 | 0 |
| PCR Luminex (Multimetrix) | 1 | L1 (GP) | 1 | 0 | 0 | 0 | 0 |
| PCR EIA (GenoID) | 1 | L1 | 1 | 0 | 0 | 0 | 0 |
| In-house PCR sequencing | 1 | L1 (PGMY-GP) | 1 | 0 | 0 | 0 | 0 |
Data including the 2 extraction control samples, which were not included in the proficiency evaluation.
RFLP, restriction fragment length polymorphism; EIA, enzyme immunoassay.
The lowest number of GE or IU of each HPV type included in the panel that was detected in both single and multiple infections by different assays is shown in Tables 4 and 5. HPV16 and HPV18 were the types detected at the lowest concentrations in most data sets. Only 1 and 3 data sets, respectively, could not detect the highest concentration of HPV16 and -18. In contrast, for HPV52, HPV59, and HPV56, there were 25, 19, and 17 data sets, respectively, that could not detect these viruses at the highest concentration (Table 4).
TABLE 4.
HPV IU/GE detected per 5 μl in both single and multiple infections by commercial assaysa
| HPV type | No. of HPV IU/GE | No. of positive results/no. of assays performed |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| All assays (%) | Linear Arrayb (Roche) | CLART HPV 2 (Genomica) | InnoLiPAc (Innogentics) | DNA chip (Biocore) | Microarray (Genetel) | Papillocheck microarray | Luminex (Multimetrix) | PCR-EIA (GenoID) | Type-specific PCR (GenoID) | ||
| 16 | 5 | 50/79 (63) | 7/15 | 4/6 | 2/6 | 4/4 | 2/2 | 1/1 | |||
| 50 | 69/79 (87) | 15/15 | 5/6 | 3/6 | 1/1 | 1/1 | |||||
| 500 | 78/79 (99) | 6/6 | 1/1 | ||||||||
| 18 | 5 | 41/78 (53) | 4/15 | 1/6 | 5/6 | 4/4 | 2/2 | 1/1 | 1/1 | ||
| 50 | 69/78 (88) | 14/15 | 6/6 | 1/1 | |||||||
| 500 | 75/78 (96) | 15/15 | nt | ||||||||
| 6 | 50 | 48/70 (69) | 6/15 | 6/6 | 6/6 | 4/4 | 2/2 | 1/1 | 1/1 | ||
| 500 | 62/70 (88) | 15/15 | 1/1 | ||||||||
| 11 | 50 | 56/70 (80) | 6/15 | 6/6 | 5/6 | 4/4 | 2/2 | 1/1 | 1/1 | 1/1 | 1/1 |
| 500 | 67/70 (96) | 14/15 | |||||||||
| 31 | 50 | 36/75 (48) | 6/15 | 3/6 | 5/6 | 2/2 | 1/1 | 1/1 | 1/1 | ||
| 500 | 61/75 (81) | 15/15 | 4/6 | ||||||||
| 33 | 50 | 55/75 (73) | 7/15 | 5/6 | 6/6 | 4/4 | 2/2 | 1/1 | 1/1 | 1/1 | |
| 500 | 70/75 (93) | 15/15 | 1/1 | ||||||||
| 35 | 50 | 50/75 (67) | 7/15 | 4/6 | 5/6 | 4/4 | 1/2 | 1/1 | 1/1 | 1/1 | |
| 500 | 65/75 (87) | 14/15 | 6/6 | 2/2 | 1/1 | ||||||
| 39 | 50 | 25/42 (60) | 5/6 | 1/4 | 1/2 | 1/1 | 1/1 | 1/1 | |||
| 500 | 38/42 (90) | Nt | nt | 6/6 | 3/4 | 2/2 | 1/1 | ||||
| 45 | 50 | 48/74 (65) | 7/15 | 1/6 | 2/6 | 4/4 | 1/1 | 1/1 | 1/1 | 1/1 | |
| 500 | 63/74 (85) | 15/15 | 2/6 | 5/6 | 2/2 | ||||||
| 51 | 50 | 49/73 (67) | 7/15 | 6/6 | 5/6 | 2/4 | 2/2 | 1/1 | 1/1 | 1/1 | 1/1 |
| 500 | 64/73 (88) | 15/15 | |||||||||
| 52 | 50 | 40/75 (53) | 3/15 | 4/6 | 1/6 | 2/4 | 2/2 | 1/1 | 1/1 | ||
| 500 | 50/75 (67) | 9/15 | 3/4 | 1/1 | 1/1 | ||||||
| 56 | 50 | 41/74 (55) | 4/15 | 1/6 | 6/6 | 2/2 | 1/1 | 1/1 | 1/1 | 1/1 | |
| 500 | 56/74 (76) | 14/15 | 2/6 | ||||||||
| 58 | 50 | 48/75 (64) | 7/15 | 5/6 | 1/6 | 3/4 | 2/2 | 1/1 | 1/1 | ||
| 500 | 68/75 (91) | 15/15 | 6/6 | 4/6 | 4/4 | 1/1 | 1/1 | ||||
| 59 | 50 | 42/74 (57) | 7/15 | 4/6 | 1/6 | 2/2 | 1/1 | 1/1 | 1/1 | ||
| 500 | 55/74 (74) | 15/15 | 1/1 | ||||||||
| 66 | 50 | 44/66 (67) | 6/15 | 6/6 | 6/6 | 1/2 | 1/1 | 1/1 | 1/1 | 1/1 | |
| 500 | 58/66 (88) | 14/15 | 2/2 | ||||||||
| 68 | 50 | 7/29 (24) | 1/2 | ||||||||
| 500 | 10/29 (34) | Nt | nt | 1/5 | 2/2 | nt | |||||
Includes laboratories with multiple false positives. Detection with an input volume of 50 μl was classified as data for the next 10-fold dilution compared to input with 5 μl. Input with 10 or 15 μl was classified as the same dilution compared to input with 5 μl.
Eight laboratories used a 50-μl input volume in Linear Array.
One InnoLiPA assay does not detect HPV 68.
TABLE 5.
HPV IU or GE detected per 5 μl in both single and multiple infections by in-house assaysa
| HPV type | No. of HPV IU/GE | No. of positive results/no. of assays performed |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PGMY-CHUV | Type-specific PCR | Lineblot | Luminex (GP) | Luminex (E6/E7) | Microarray | PCR-RFLP | DEIA LIPA | PCR-EIA | Luminex (PGMY-GP) | PCR sequencing | ||
| 16 | 5 | 4/7 | 6/7 | 2/4 | 3/4 | 3/4 | 2/3 | 1/3 | 2/2 | 1/2 | 1/1 | 1/1 |
| 50 | 6/7 | 3/4 | 4/4 | 3/3 | 2/3 | 2/2 | ||||||
| 500 | 7/7 | 7/7 | 4/4 | 4/4 | 3/3 | |||||||
| 18 | 5 | 2/7 | 5/7 | 1/4 | 4/4 | 2/4 | 2/3 | 2/2 | 1/1 | 1/1 | ||
| 50 | 6/7 | 4/4 | 4/4 | 3/3 | 1/2 | |||||||
| 500 | 7/7 | 2/3 | 2/2 | |||||||||
| 6 | 50 | 4/7 | 3/5 | 1/4 | 4/4 | 1/2 | 1/2 | 3/3 | 2/2 | 1/2 | 1/1 | 1/1 |
| 500 | 5/7 | 5/5 | 2/2 | |||||||||
| 11 | 50 | 5/7 | 4/5 | 4/4 | 4/4 | 1/2 | 2/2 | 3/3 | 2/2 | 2/2 | 1/1 | 1/1 |
| 500 | 7/7 | 5/5 | ||||||||||
| 31 | 50 | 5/7 | 2/4 | 1/4 | 2/4 | 1/3 | 1/3 | 2/2 | 2/2 | 1/1 | ||
| 500 | 5/7 | 6/7 | 4/4 | 4/4 | 3/4 | 3/3 | 2/3 | |||||
| 33 | 50 | 2/7 | 7/7 | 3/4 | 4/4 | 3/4 | 2/3 | 2/3 | 2/2 | 2/2 | 1/1 | |
| 500 | 5/7 | 4/4 | 3/3 | 3/3 | ||||||||
| 35 | 50 | 2/7 | 6/7 | 4/4 | 4/4 | 2/4 | 2/3 | 2/2 | 2/2 | 1/1 | 1/1 | |
| 500 | 5/7 | 4/4 | ||||||||||
| 39 | 50 | 4/6 | 4/4 | 3/4 | 2/3 | 2/2 | ||||||
| 500 | nt | 6/6 | 3/4 | 4/4 | 1/1 | nt | nt | |||||
| 45 | 50 | 6/7 | 5/7 | 4/4 | 4/4 | 4/4 | 2/3 | 2/2 | 1/1 | 1/1 | 1/1 | |
| 500 | 3/3 | |||||||||||
| 51 | 50 | 5/7 | 5/6 | 1/4 | 4/4 | 2/4 | 1/3 | 1/3 | 2/2 | 1/1 | 1/1 | |
| 500 | 7/7 | 6/6 | 3/4 | 3/4 | 1/1 | |||||||
| 52 | 50 | 5/7 | 7/7 | 4/4 | 4/4 | 3/3 | 2/2 | 1/1 | ||||
| 500 | 1/4 | |||||||||||
| 56 | 50 | 1/7 | 5/6 | 4/4 | 4/4 | 4/4 | 1/3 | 2/2 | 1/2 | 1/1 | 1/1 | |
| 500 | 4/7 | 2/2 | ||||||||||
| 58 | 50 | 5/7 | 7/7 | 3/4 | 3/4 | 2/4 | 2/3 | 2/3 | 2/2 | 1/1 | 1/1 | |
| 500 | 4/4 | 4/4 | 3/4 | 2/2 | ||||||||
| 59 | 50 | 6/7 | 5/7 | 3/4 | 3/4 | 2/4 | 2/3 | 1/2 | 1/1 | 1/1 | 1/1 | |
| 500 | 7/7 | 4/4 | 3/4 | 2/2 | ||||||||
| 66 | 50 | 5/7 | 2/4 | 4/4 | 2/3 | 2/3 | 1/3 | 2/2 | 1/1 | 1/1 | 1/1 | |
| 500 | 6/7 | nt | 4/4 | 3/3 | 3/3 | |||||||
| 68 | 50 | 4/4 | 1/2 | 1/2 | ||||||||
| 500 | nt | nt | 2/2 | nt | nt | |||||||
Includes laboratories with multiple false positives. Detection with an input volume of 50 μl was classified as data for the next 10-fold dilution compared to input with 5 μl. Input with 10 or 15 μl was classified as the same dilution compared to input with 5 μl. RFLP, restriction fragment length polymorphism; EIA, enzyme immunoassay.
DISCUSSION
We report on the development of an internationally comparable quality assurance methodology that is traceable to ISs. An accurate and internationally comparable HPV DNA detection and typing methodology is an essential component in the evaluation of HPV vaccines and in effective implementation and monitoring of HPV vaccination programs. Standardized methodology for evaluation of laboratory performance is fundamental to enable any comparison of the methodologies used in laboratories worldwide. The major tools for achieving progress toward this goal are developing international biological standards and preparing and validating proficiency panels to qualify methods. The current study has established that such international proficiency panels with units traceable to ISs can be used in global studies. We have also demonstrated that such studies provide a unique overview of the status of the HPV detection and typing methodologies that are being used globally and how well they perform in different laboratories.
Overall, it can be said that a majority of laboratories in this study had good performances of their HPV DNA-typing tests. However, some limitations were revealed.
There was a clear tendency toward systematically different limits of sensitivity for different HPV types, e.g., HPV16 and HPV18 were the types detected at the smallest amounts in most data sets (only 1 and 3 data sets, respectively, could not detect 500 IU/5 μl), whereas HPV52, HPV59, and HPV56 could not be detected at the 500-GE/5 μl concentration by 25, 19, and 18 data sets, respectively. Thus, many surveys of circulating HPV types might systematically underestimate the prevalence of HPV52, -56, and -59 compared to HPV16 and -18.
There was also a tendency toward lower sensitivity of tests when multiple HPV types were present. In the samples containing multiple HPV types, between 50% and 73% of the data sets could correctly identify the types present, but in samples with only 1 HPV type present, an average of 84% of HPV types could be identified without false-positive results. This tendency would cause a systematic underestimation of the prevalence of multiple infections and would introduce a systematic detection bias in epidemiological studies, with detectability being dependent on determinants of HPV acquisition (e.g., a given HPV type would be more difficult to detect in high-risk groups because of the higher likelihood of other HPV infections).
There were a surprisingly large number of false-positive results reported, with only 34/80 data sets being 100% specific. The proficiency panel contained only 1 entirely HPV-negative sample. The present study was designed primarily to evaluate HPV typing (rather than mere HPV detection), and we considered that in this context specificity should be measured primarily as an absence of detection of a specific HPV type, including when other HPV types were present. Thus, for each HPV type evaluated, there are at least 39 negative samples included in the panel, and 1 false-positive result thus equals >97% specificity. There was only 1 indication of a systematic mistyping (some Linear Array-based data sets reported HPV56-containing samples as positive for HPV66), but otherwise, there was no single sample that had systematic false positivity for the same type in several laboratories. These very common false positives are therefore not associated with the panel or with the assays used but rather appear to result from the laboratory environment and performance. Considering the deleterious consequences that a false-positive result may have, it appears that a substantial effort for increased specificity of testing is warranted.
On the other hand, there were some needs for improvement of the proficiency panel itself that were identified by this study. The HPV39 plasmid used in the panel was cloned into the vector at the binding site of one of the most commonly used PCR primers (PGMY). All assays using the PGMY primer system, including Linear Array and CLART, could not detect the HPV39 plasmid in the panel. As this was because of the way the plasmid was constructed, all these data sets were considered not to have been evaluated for HPV39 in this study.
The plasmid used to test for HPV68a was not full length but contained only the L1 gene. We noted that Linear Array and all other PGMY-based assays that are indeed directed against L1 could not detect the HPV68a plasmid. Comparison of the sequences of HPV68a and HPV68b (ME180 isolate) showed significant differences in the sequence corresponding to the PGMY primer binding site. As the sequence of HPV68b was published before the sequence of HPV68a, it appears that these systems are designed to detect only HPV68b (11, 14). All data sets reporting the use of primers directed to genes other than L1 or that used the PGMY primers were considered not to test for HPV68 in this study. Accordingly, only 29 data sets could be analyzed for detection of HPV68a and only 11 of the 29 laboratories (38%) could detect HPV68a. For the next WHO HPV LabNet proficiency panel, HPV39 will be recloned to change the cloning site, and full-length genomes of both HPV68a and HPV68b will also be included.
The Linear Array cannot exclude HPV52 when the sample is positive for HPV33, HPV35, or HPV58. Some laboratories have developed a type-specific PCR for HPV52 to test HPV33-, -35-, and -58-positive samples, whereas some laboratories (4/15) scored all sample with multiple infections containing HPV52 as negative for HPV52 (4, 23). This resulted in their being regarded as not proficient for HPV52 in this study. Four data sets generated using Linear Array were considered not proficient, since they reported 2 or even 3 false-positive results. HPV66 was detected as falsely positive in 7 of 15 false-positive results submitted in the 15 data sets using Linear Array; 6 of these samples contained 500 GE of HPV56 that was correctly identified. The detection of HPV66 in these samples was not reported by any other assay, indicating that the false detection of HPV66 in HPV56-positive samples is a problem that is commonly seen with the Linear Array assay.
For two commercial tests (InnoLiPA and CLART), 4 out of 6 data sets were not proficient because of too many false positives. InnoLiPa could not identify HPV52 in 5 of 6 data sets. On the other hand, HPV52 was reported in 9 samples where it was not present. The numbers of false-positive samples reported by InnoLiPA were between 3 and 5 for the 4 laboratories that were not proficient. Three laboratories using CLART reported 7, 17, and 21 false-positive results, some with more than 3 false positives in each sample. Four laboratories using CLART could not detect HPV56 and -45 in samples with multiple types. There was no consistent false positivity for any specific sample for these two assays. The false positivities for these assays appeared to be randomly distributed among the samples and were always different for the different laboratories, indicating that the problem is not related to the assay kit itself. Indeed, there were examples of several laboratories that had completely proficient results using these assays.
A major conclusion of the present study is that differences in performance were much larger between laboratories than between different types of assays. Proficiency panel testing is particularly useful to stimulate a learning process for improved performance in laboratories. Once regular feedback on proficiency testing results is implemented, improvement of performance usually follows rapidly. An example of this was the results of the PGMY-Lineblot assay that was recently set up in the HPV LabNet. Several laboratories that were using this assay for the first time had suboptimal results but became proficient in a subsequent proficiency test performed when there had been more time for practice.
The 2 samples for evaluation of the DNA extraction step before the HPV testing and typing had a surprisingly low proportion of correct results. The sample containing 2,000 cells of the cervical cancer cell line SiHa with about 1 copy of HPV16 per cell (i.e., a total of 2,000 IU of HPV16/5 μl) was detected in only about one-third of the data sets. Also, a large number of data sets (six) reported false-positive results for the sample containing an HPV-negative human cell line. This indicates that low yield in the DNA extraction step, potentially reducing sensitivity, as well as contamination in the DNA extraction step may be significant problems in the field of HPV DNA testing. Future proficiency panels will contain a larger set of samples designed to specifically evaluate the DNA extraction step before the actual HPV testing and typing.
There are additional steps in the laboratory detection process that are not evaluated by the present strategy, notably, sampling technique, handling and storage, natural variability of circulating virus strains, PCR-inhibiting substances, and naturally occurring genome modifications (e.g., integration and rearrangement). The HPV LabNet has chosen to perform quality control for these aspects of testing by launching a confirmatory testing scheme, where part of the clinical samples being tested are annually submitted to a higher-level reference laboratory for retesting (5). The alternative strategy, to include clinical samples in proficiency-testing schemes, was not chosen because of the need to have exactly reproducible panels with defined content that can be used by hundreds of laboratories over many years and since confirmatory testing schemes were considered to better reflect the actual testing being done.
It should be emphasized that the current proficiency panel study was designed to evaluate the performance of HPV testing and typing tests used in HPV vaccinology and HPV surveillance but not for evaluation of HPV tests used in cervical cancer screening (12). The demands on performance of HPV-typing assays vary depending on the purpose of the testing. In vaccinology, high sensitivity is needed for clinical vaccine trials, as failure to detect prevalent infections at entry may result in apparent vaccine failures. In contrast, the clinical HPV-associated diseases, such as high-grade cervical intraepithelial neoplasia, typically contain larger amounts of virus, and cervical-screening programs using HPV testing do not have as high demands on sensitivity (12). Guidelines for evaluations of such tests have recently been published (12).
In conclusion, we found that global HPV DNA proficiency studies are both feasible and informative. The launch of an internationally standardized methodology to analyze the specificity and sensitivity for different HPV-typing assays (as well as the performance of participating laboratories) to correctly identify the 16 HPV types that are most important in HPV surveillance and vaccinology is likely to greatly enhance the quality and comparability of studies in these fields.
Acknowledgments
Members of the WHO HPV LabNet are as follows: A. C. Bharti, Division of Molecular Oncology, Institute of Cytology and Preventive Oncology, Nodia, India; J. Dillner, Department of Medical Microbiology, Lund University, Malmo, Sweden; E. Ennaifer-Jerbi, Tunis Pasteur Institute, Tunis, Tunisia; S. Garland, Department of Microbiology and Infectious Disease, Royal Women's Hospital, Carlton, Australia; I. Kukimoto, Center for Pathogen Genomics, National Institute of Infectious Diseases, Tokyo, Japan; A. M. Picconi, National Institute of Infectious Diseases, Buenos Aires, Argentina; R. Sahli, Institut de Microbiologie, CHUV, Lausanne, Switzerland; S. Sukvirach, National Cancer Institute, Bangkok, Thailand; E. R. Unger, Centers for Disease Control and Prevention, Atlanta, GA; and A. L Williamson, Institute of Infectious Disease and Molecular Medicine, University of Cape Town, Cape Town, South Africa.
We thank Michael Pawlita of the DKFZ for acting as an external validation laboratory in evaluation of the proficiency panel. We also thank all 61 laboratories worldwide that participated in the proficiency study. We thank Ethel-Michele de Villiers (HPV6, -11, -16, and -18), Gérard Orth (HPV33, -39, -66, and -68), Attila T. Lörincz (HPV31, -35, and -56), Wayne Lancaster (HPV52), Keerti Shah (HPV45), Saul Silverstein (HPV51), and Toshihiko Matsukura (HPV58 and -59) for giving their kind permission to use the cloned HPV types for the proficiency panel.
This work was supported by the WHO via a project funded by the Bill and Melinda Gates Foundation. T.Z. is a staff member of the World Health Organization.
The authors alone are responsible for the views expressed in this publication, and they do not necessarily represent the decisions, policy, or views of the World Health Organization.
Footnotes
Published ahead of print on 15 September 2010.
REFERENCES
- 1.Bernard, H. U., S. Y. Chan, M. M. Manos, C. K. Ong, L. L. Villa, H. Delius, C. L. Peyton, H. M. Bauer, and C. M. Wheeler. 1994. Identification and assessment of known and novel human papillomaviruses by polymerase chain reaction amplification, restriction fragment length polymorphisms, nucleotide sequence, and phylogenetic algorithms. J. Infect. Dis. 170:1077-1085. [DOI] [PubMed] [Google Scholar]
- 2.Bosch, F. X., A. Lorincz, N. Munoz, C. J. Meijer, and K. V. Shah. 2002. The causal relation between human papillomavirus and cervical cancer. J. Clin. Pathol. 55:244-265. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Chouhy, D., L. B. Gil, A. L. Nocito, D. Wojdyla, L. Ornella, J. Cittadini, D. Gardiol, and A. A. Giri. 2006. Development and evaluation of a colorimetric PCR system for the detection and typing of human papillomaviruses. Int. J. Mol. Med. 18:995-1003. [PubMed] [Google Scholar]
- 4.Coutlée, F., D. Rouleau, G. Ghattas, C. Hankins, S. Vezina, P. Cote, J. Macleod, A. de Pokomandy, D. Money, S. Walmsley, H. Voyer, P. Brassard, and E. Franco. 2007. Confirmatory real-time PCR assay for human papillomavirus (HPV) type 52 infection in anogenital specimens screened for HPV infection with the linear array HPV genotyping test. J. Clin. Microbiol. 45:3821-3823. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ferguson, M., D. E. Wilkinson, and T. Zhou. 2009. WHO meeting on the standardization of HPV assays and the role of the WHO HPV Laboratory Network in supporting vaccine introduction held on 24-25 January 2008, Geneva, Switzerland. Vaccine 27:337-347. [DOI] [PubMed] [Google Scholar]
- 6.Gheit, T., S. Landi, F. Gemignani, P. J. Snijders, S. Vaccarella, S. Franceschi, F. Canzian, and M. Tommasino. 2006. Development of a sensitive and specific assay combining multiplex PCR and DNA microarray primer extension to detect high-risk mucosal human papillomavirus types. J. Clin. Microbiol. 44:2025-2031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Gravitt, P. E., C. L. Peyton, R. J. Apple, and C. M. Wheeler. 1998. Genotyping of 27 human papillomavirus types by using L1 consensus PCR products by a single-hybridization, reverse line blot detection method. J. Clin. Microbiol. 36:3020-3027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Jacobs, M. V., A. M. de Roda Husman, A. J. van den Brule, P. J. Snijders, C. J. Meijer, and J. M. Walboomers. 1995. Group-specific differentiation between high- and low-risk human papillomavirus genotypes by general primer-mediated PCR and two cocktails of oligonucleotide probes. J. Clin. Microbiol. 33:901-905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kleter, B., L. J. van Doorn, L. Schrauwen, A. Molijn, S. Sastrowijoto, J. ter Schegget, J. Lindeman, B. ter Harmsel, M. Burger, and W. Quint. 1999. Development and clinical evaluation of a highly sensitive PCR-reverse hybridization line probe assay for detection and identification of anogenital human papillomavirus. J. Clin. Microbiol. 37:2508-2517. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Klug, S. J., A. Molijn, B. Schopp, B. Holz, A. Iftner, W. Quint, J. F. P. Snijders, K. U. Petry, S. Kruger Kjaer, C. Munk, and T. Iftner. 2008. Comparison of the performance of different HPV genotyping methods for detecting genital HPV types. J. Med. Virol. 80:1264-1274. [DOI] [PubMed] [Google Scholar]
- 11.Longuet, M., S. Beaudenon, and G. Orth. 1996. Two novel genital human papillomavirus (HPV) types, HPV68 and HPV70, related to the potentially oncogenic HPV39. J. Clin. Microbiol. 34:738-744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Meijer, C. J., J. Berkhof, P. E. Castle, A. T. Hesselink, E. L. Franco, G. Ronco, M. Arbyn, F. X. Bosch, J. Cuzick, J. Dillner, D. A. Heideman, and P. J. Snijders. 2009. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. Int. J. Cancer 124:516-520. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Quint, W. G., S. R. Pagliusi, N. Lelie, E. M. de Villiers, and C. M. Wheeler. 2006. Results of the first World Health Organization international collaborative study of detection of human papillomavirus DNA. J. Clin. Microbiol. 44:571-579. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Reuter, S., H. Delius, T. Kahn, B. Hofmann, H. zur Hausen, and E. Schwarz. 1991. Characterization of a novel human papillomavirus DNA in the cervical carcinoma cell line ME180. J. Virol. 65:5564-5568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schloss, L., A. M. van Loon, P. Cinque, G. Cleator, J. M. Echevarria, K. I. Falk, P. Klapper, J. Schirm, B. F. Vestergaard, H. Niesters, T. Popow-Kraupp, W. Quint, and A. Linde. 2003. An international external quality assessment of nucleic acid amplification of herpes simplex virus. J. Clin. Virol. 28:175-185. [DOI] [PubMed] [Google Scholar]
- 16.Schmitt, M., I. G. Bravo, P. J. Snijders, L. Gissmann, M. Pawlita, and T. Waterboer. 2006. Bead-based multiplex genotyping of human papillomaviruses. J. Clin. Microbiol. 44:504-512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Schmitt, M., B. Dondog, T. Waterboer, and M. Pawlita. 2008. Homogeneous amplification of genital human alpha papillomaviruses by PCR using novel broad-spectrum GP5+ and GP6+ primers. J. Clin. Microbiol. 46:1050-1059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Schuurman, R., D. Brambilla, T. de Groot, D. Huang, S. Land, J. Bremer, I. Benders, and C. A. Boucher. 2002. Underestimation of HIV type 1 drug resistance mutations: results from the ENVA-2 genotyping proficiency program. AIDS Res. Hum. Retroviruses 18:243-248. [DOI] [PubMed] [Google Scholar]
- 19.Smith, J. S., L. Lindsay, B. Hoots, J. Keys, S. Franceschi, R. Winer, and G. M. Clifford. 2007. Human papillomavirus type distribution in invasive cervical cancer and high-grade cervical lesions: a meta-analysis update. Int. J. Cancer 121:621-632. [DOI] [PubMed] [Google Scholar]
- 20.Snijders, P. J., C. J. Hogewoning, A. T. Hesselink, J. Berkhof, F. J. Voorhorst, M. C. Bleeker, and C. J. Meijer. 2006. Determination of viral load thresholds in cervical scrapings to rule out CIN 3 in HPV 16, 18, 31 and 33-positive women with normal cytology. Int. J. Cancer 119:1102-1107. [DOI] [PubMed] [Google Scholar]
- 21.Söderlund-Strand, A., J. Carlson, and J. Dillner. 2009. Modified general primer PCR system for sensitive detection of multiple types of oncogenic human papillomavirus. J. Clin. Microbiol. 47:541-546. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Sotlar, K., D. Diemer, A. Dethleffs, Y. Hack, A. Stubner, N. Vollmer, S. Menton, M. Menton, K. Dietz, D. Wallwiener, R. Kandolf, and B. Bultmann. 2004. Detection and typing of human papillomavirus by e6 nested multiplex PCR. J. Clin. Microbiol. 42:3176-3184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Stevens, M. P., S. M. Garland, and S. N. Tabrizi. 2008. Development and validation of a real-time PCR assay specifically detecting human papillomavirus 52 using the Roche LightCycler 480 system. J. Virol. Methods 147:290-296. [DOI] [PubMed] [Google Scholar]
- 24.Valentine-Thon, E., A. M. van Loon, J. Schirm, J. Reid, P. E. Klapper, and G. M. Cleator. 2001. European proficiency testing program for molecular detection and quantitation of hepatitis B virus DNA. J. Clin. Microbiol. 39:4407-4412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.van den Brule, A. J., R. Pol, N. Fransen-Daalmeijer, L. M. Schouls, C. J. Meijer, and P. J. Snijders. 2002. GP5+/6+ PCR followed by reverse line blot analysis enables rapid and high-throughput identification of human papillomavirus genotypes. J. Clin. Microbiol. 40:779-787. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.WHO. 2005. WHO technical workshop on the role of laboratory detection of human papillomavirus in global disease prevention and control. World Health Organization. http://whqlibdoc.who.int/hq/2006/WHO_IVB_06.04_eng.pdf.