Abstract
Objective
New screening guidelines recommend that HPV-negative/ASC-US results be considered as equivalent to HPV-negative/Pap-negative results, leading to rescreening in 5 years. However, despite ample research data, the routine clinical performance of HPV testing of women with ASC-US has not been adequately documented.
Methods
We estimated 5-year risks of CIN3+ and cancer for 2 groups between 2003-2010 at Kaiser Permanente Northern California: 27,050 women aged 30-64 who underwent HPV and Pap cotesting and had an ASC-US Pap result, and 12,209 women aged 25-29 who underwent HPV triage of ASC-US.
Results
Five-year risks of CIN3+ and of cancer for women aged 30-64 testing HPV-negative/ASC-US and for 923,152 women testing Pap-negative alone were similar although statistically distinguishable (CIN3+: 0.43% vs. 0.26% (p=0.001); Cancer: 0.050% vs. 0.025% (p=0.1, respectively)). The cancer risk increase for HPV-negative/ASC-US versus Pap-negative alone was confined to women aged 60-64 (0.26% vs. 0.035%, p=0.3). Five-year risks of CIN3+ and of cancer for women with HPV-negative/ASC-US were substantially higher than those for women testing HPV-negative/Pap-negative (CIN3+: 0.43% vs. 0.08% (p<0.0001); Cancer: 0.050% vs. 0.011% (p=0.003, respectively)). For women aged 30-64 testing HPV-positive/ASC-US, 5-year risks of CIN3+ and cancer were slightly higher than for the 9,374 women with LSIL (CIN3+: 6.8 % vs. 5.2% (p=0.0007); Cancer: 0.41% vs. 0.16% (p=0.04)). Similar patterns were seen for women aged 25-29.
Conclusions
Women with HPV-negative/ASC-US had similar risk as women testing Pap-negative alone, but had higher risk than women testing HPV-negative/Pap-negative. Based on the principle of “equal management of equal risks”, our findings support equal management of women with HPV-negative/ASC-US and those with Pap-negative alone, except for exiting women from screening because cancer risks at ages 60-64 may be higher for HPV-negative/ASC-US. Our findings also support managing HPV-positive/ASC-US and LSIL similarly.
Précis
Women testing HPV-negative/ASC-US have similar risk of CIN3+ or cancer as women testing Pap-negative alone, but have higher risk than women testing HPV-negative/Pap-negative.
Keywords: Human Papillomavirus (HPV), cancer prevention, Pap, cervical intraepithelial neoplasia (CIN), Hybrid Capture 2 (HC2), ASC-US
Introduction
Based on numerous research trials, HPV testing has been established to be an effective means to triage equivocal or borderline abnormal Pap results, called Atypical Squamous Cells of Undetermined Significance (ASC-US) in the Bethesda System(1-5). Accordingly, in the United States, reflex (i.e., automatic) HPV testing often follows ASC-US interpretations. In some centers, women aged 30-64 have HPV testing for ASC-US as part of HPV/Pap cotesting. Although exact numbers are lacking, HPV testing of ASC-US likely affects about 1 million women per year in the United States alone. If the HPV test is positive, the woman is referred to colposcopy. If negative, according to the previous set of guidelines sponsored by the American Society of Colposcopy and Cervical Pathology, such women have been recommended to undergo repeat screening at 1 year, rather than at a routine, longer interval (6).
However, the most recent consensus guidelines from 25 organizations under the aegis of the American Cancer Society/American Society for Colposcopy and Cervical Pathology/American Society for Clinical Pathology (ACS/ASCCP/ASCP) (7) recommend subsequent follow-up of an HPV-negative/ASC-US result by rescreening with Pap test and HPV cotesting at 5 years, or with Pap alone at 3 years (8). Also, an HPV-negative/ASC-US result is considered as a negative cotest for purposes of exiting screening. This guideline change, in which HPV-negative/ASC-US was considered a negative cotest, was based partly on data from observational studies and clinical trials showing that the risk of CIN2 or CIN3 for women testing HPV-negative/ASCUS was very similar to that from women with negative Pap results alone (without HPV testing) (5, 6, 9-11).
Despite excellent evidence from research trials, data are still lacking on the performance of HPV triage of ASC-US in routine clinical practice, especially for cancer risks. Studies from actual clinical practice are needed to reassure clinicians about the feasibility and safety of following cervical cancer screening guidelines in routine practice (10). We examine performance, estimating the 5-year absolute risks of CIN2+, CIN3+, and cancer following HPV-positive and HPV-negative/ASC-US results using data from a retrospective cohort of 1,100,741 women aged 25-64 undergoing cervical cancer screening at Kaiser Permanente Northern California (KPNC), an integrated healthcare delivery system that has used HPV testing to triage ASC-US Pap results in women under 30 since 2001, and cotesting among women 30 and older since 2003(10). The KPNC experience serves as a large-scale “demonstration project” of HPV triage of ASC-US in routine clinical practice.
We also examine whether the effectiveness of HPV triage of ASC-US in detection of CIN2+, CIN3+, and cancer varies with age. The incidence of HPV infection peaks well before age 30, corresponding with the typical age of onset of sexual activity in the US. Consequently, many HPV infections in women under age 30 will be recently acquired infections, most of which will naturally clear in a few years without progressing, even to CIN2 (12). As a result, HPV testing in women under age 30 may be more likely to detect infections that will naturally clear, and thus HPV triage may be less efficient for women under age 30.
Methods
The design of our cohort study from KPNC has been described previously(10); in this report, we enlarged the dataset to include women age 30 and older entering cotesting between 2006-2010, and to include data HPV triage of ASC-US in women 25-29. As a result of the data expansion, we were able to examine 965,360 women aged 30-64 and 135,382 women aged 25-29 screened from 2003 to 2010. We considered as the baseline screen the first cotest or HPV triage of ASC-US recorded for the women. For women without cotests or HPV triage of ASC-US (almost all were under age 30), her first Pap-alone was her baseline screen. Biopsy and cancer information was collected on all women through December 31, 2010. The Kaiser Permanente Northern California Institutional Review Board (IRB) approved use of the data, and the National Institutes of Health Office of Human Subjects Research deemed this study exempt from IRB review.
Pap tests were performed at KPNC regional and facility labs; HPV testing was performed at the single regional lab. Conventional Pap slides were manually reviewed following processing by the BD FocalPoint Slide Profiler (BD Diagnostics, Burlington, NC, USA) primary screening and directed quality control system, in accordance with FDA-approved protocols. Starting in 2009, KPNC transitioned to liquid-based Pap testing using BD SurePath (BD Diagnostics, Burlington, NC, USA). Conventional or liquid-based Pap tests are reported according to the 2001 Bethesda System (4). Hybrid Capture 2 (HC2; Qiagen, Germantown, MD, USA) was used to test for high-risk HPV types according to manufacturer’s instructions.
The Permanente Medical Group (TPMG), which is the physician component of KPNC, develops Clinical Practice Guidelines for cervical cancer screening and management of abnormal tests in KPNC in partnership with the KP National Guideline Program, Care Management Institute, to support the clinical decisions of their providers. According to KPNC guidelines, women with HPV-positive/ASC-US, or with LSIL or worse Pap results, should undergo colposcopy. Women with HPV-negative/ASC-US should be re-tested in 1 year.
Cumulative risk of CIN2+, CIN3+, or cervical cancer alone for each co-test result was calculated as the sum of risk at the baseline test (plotted at time zero on each figure) and the incidence after baseline(13). Risk at the baseline screen was the risk of CIN2+, CIN3+, or cancer for Pap results or co-test results where women were immediately referred to colposcopy and was estimated using logistic regression, stratified by 5-year age groups 25-29, 30-34, ..., 60-64, separately for each co-test result or Pap results. We included in the logistic regression analyses the very small numbers of women testing HPV-negative/ASC-US or Pap-negative at their baseline test who underwent colposcopy. We used Weibull survival models (14) to estimate risks over time strictly after the baseline test, among women for whom CIN2+ was not found at the baseline test. Weibull models can make smoother and more accurate risk estimates than non-parametric methods analogous to Kaplan-Meier (15) and naturally handle interval-censoring of disease outcomes between screening tests. Separate Weibull models were fit for each co-test result or Pap result, with age group as a covariate. When risk was calculated for a cytology result without regard to HPV testing, we refer to those risks as “Pap-alone”.
Results
Table 1 shows the distribution of the worst histologic findings by Pap results and co-test result through 2010, for women aged 25-29, and women age 30-64. Among women with ASC-US at baseline, 293 women age 25-29 were diagnosed with CIN3 or AIS (adenocarcinoma in situ) and 7 women were diagnosed with cancer; there were 479 women age 30-64 who developed CIN3 or AIS and 36 women who developed cancer.
Table 1.
Total |
Worst histologic diagnosis during follow-up |
|||||||||
---|---|---|---|---|---|---|---|---|---|---|
Baseline Pap and HPV test result | n | <CIN1, n |
CIN1, n |
CIN2, n |
CIN3, n |
AIS, n |
Total CIN3 or AIS, n |
Squamous carcinoma, n |
Adeno- carcinoma, n |
Total cancers, n |
(A) Aged 25-29 at baseline | ||||||||||
Total | 135,382 | 129,428 | 4,037 | 1250 | 621 | 23 | 645 | 12 | 6 | 22 |
LSILa | 3,236 | 1,911 | 981 | 262 | 81 | 1 | 82 | 0 | 0 | 0 |
ASC-USa | 12,209 | 8,963 | 2,292 | 654 | 283 | 9 | 293 | 5 | 2 | 7 |
HPV-positive/ASC-US | 6,340 | 3,418 | 2,034 | 605 | 270 | 8 | 279 | 2 | 2 | 4 |
HPV-negative/ASC-US | 5,485 | 5,237 | 201 | 34 | 11 | 1 | 12 | 1 | 0 | 1 |
HPV-unknown/ASC-US | 384 | 308 | 57 | 15 | 2 | 0 | 2 | 2 | 0 | 2 |
Pap-negativea | 118,684 | 117,936 | 554 | 127 | 61 | 2 | 63 | 1 | 1 | 4 |
(B) Aged 30-64 at baseline | ||||||||||
Total | 965,360 | 942,657 | 15,357 | 4235 | 2480 | 228 | 2723 | 198 | 114 | 388 |
SILa | 9,374 | 4,845 | 3,414 | 780 | 315 | 8 | 325 | 6 | 2 | 10 |
ASC-USa | 27,050 | 21,714 | 3,894 | 927 | 448 | 28 | 479 | 16 | 11 | 36 |
HPV-positive/ASC-US | 9,901 | 5,285 | 3,314 | 844 | 404 | 25 | 432 | 11 | 9 | 26 |
HPV-negative/ASC-US | 16,326 | 15,708 | 511 | 65 | 35 | 2 | 37 | 3 | 1 | 5 |
HPV-unknown/ASC-US | 823 | 721 | 69 | 18 | 9 | 1 | 10 | 2 | 1 | 5 |
Pap-negativea | 923,152 | 913,113 | 7,259 | 1716 | 836 | 106 | 946 | 41 | 52 | 118 |
HPV-negative/Pap-negativeb | 836,803 | 831,843 | 3,990 | 675 | 224 | 22 | 247 | 21 | 16 | 48 |
Total cancers includes not only squamous cell carcinoma and adenocarcinoma but also adenosquamous carcinoma, and cervical cancer of unknown histology or histology unrelated to HPV infection. The category “Total CIN3 or AIS” also includes women with diagnoses that did not differentiate between CIN3 and AIS. “<CIN1” is included for completeness, and includes normal or metaplastic biopsy findings, no biopsy taken, and no record of colposcopy found.
Baseline Pap result alone (regardless of HPV test result)
HPV-positive/Pap-negative results are not presented here but addressed in a separate paper (1)
Figure 1 shows age-specific prevalence of HPV positivity at baseline screen by age and Pap results. For women with ASC-US, HPV positivity was similar for women aged 25-29 and 30-34 (54% vs. 56%) and then declined sharply through age 50-54 (56% vs. 24%, p<0.0001). Figure 1 shows that for LSIL, HPV positivity was similar for ages 25-29 and 30-34 (85% vs. 88%) and then declined through age 55-59 (88% vs. 69%, p<0.0001). For Pap-negative, HPV positivity declined over ages 25-29, 30-34, through ages 60-64(10% to 6.1% to 2.1%, p<0.0001).
Figure 2 shows 5-year risks of CIN2+ and CIN3+ for women aged 30-64 at baseline. For Pap results-alone, the CIN2+ and CIN3+ risks for negative Pap results, ASC-US, and LSIL were very different from each other, suggesting the existence of 3 separate risk groups. However, when we examine ASC-US by HPV status (in red), risks of CIN3+ and cancer for women aged 30-64 testing HPV-positive/ASC-US and for LSIL were similar, with slightly higher risks for HPV-positive/ASC-US than LSIL (CIN3+: 6.8 % vs. 5.2% (p=0.0007); Cancer (not in figure): 0.41% vs. 0.16% (p=0.04)).
Risks of CIN3+ and cancer for HPV-negative/ASC-US and for negative Pap results were similar although statistically distinguishable (CIN3+: 0.43% vs. 0.26% (p=0.001); Cancer (not in figure): 0.050% vs. 0.025% (p=0.1)). Not shown in Figure 2, the risks for HPV-negative/ASC-US were substantially higher than those for women testing HPV-negative/Pap-negative (CIN3+: 0.43% vs. 0.08% (p<0.0001); Cancer (not in figure): 0.050% vs. 0.011% (p=0.003)).
Figure 3 shows the 5-year risks of CIN2+, CIN3+, and cancer by age from 25 to 64. At every age, negative Pap-alone results, ASC-US, and LSIL appeared to be 3 separate risk groups, but upon splitting ASC-US by HPV status, HPV-negative/ASC-US had similar risks as negative Pap-alone results, and HPV-positive/ASC-US had similar risks as LSIL. The difference between the CIN2+ risks of HPV-positive/ASC-US and HPV-negative/ASC-US was very similar at ages 25-29 and ages 30-34 (risk difference: 18% vs. 18%).
Figure 3 also indicates that the cancer risk difference between HPV-negative/ASC-US and negative Pap-alone results was very similar until age 60, after which it increased for HPV-negative/ASC-US versus negative Pap results (ages 60-64:0.26% vs. 0.035%, p=0.3).
Discussion
Our analysis of performance data in the large KPNC study population confirms the results of the clinical trials that led to widespread clinical acceptance of the ASC-US triage strategy (2). We show that HPV-positive/ASC-US is equivalent to LSIL in predicting 5-year risk of CIN2+, CIN3+, or cancer. We also show that HPV-negative/ASC-US has CIN2+ and CIN3+ risks nearly equivalent to those of a negative Pap result alone (i.e., a cytology result that is “negative for intraepithelial lesion or malignancy (NILM)”, without additional risk stratification by HPV cotesting). However, HPV-negative/ASC-US had substantially higher CIN2+/CIN3+/Cancer risks than an HPV-negative/Pap-negative result, indicating that an ASC-US Pap may convey some risk information in the absence of detectable HPV.
According to the recent guidelines, the recommendation for next screening following negative Pap-alone results is 3 years, not the 5 years that is recommended following the ultra-low risk HPV-negative/Pap negative. Therefore, the recent ACS/ASCCP/ASCP consensus recommendation to extend rescreening interval to 5 years following HPV-negative/ASC-US is not supported by our data. Following the principle of “equal management of equal risks”, the latter warrants 3-year follow-up.
The very slightly elevated risk of cancer for HPV-negative/ASC-US compared to negative Pap alone for women aged 30-64 is concerning. However, this potential difference in cancer risk was limited to women aged 60-64. Although these cancer risk increases are not formally statistically significant, we note this tentative finding because of the historic concern about cancer risks for women with HPV-negative/ASC-US. Because women aged 60-64 with consecutive negative screens are candidates for exiting lifetime screening, our findings of potentially elevated cancer risks in women with HPV-negative/ASC-US at age 60-64 suggests that ASC-US Pap results should be further investigated before exiting.
Therefore, our findings generally support managing women with HPV-negative/ASC-US with a 3-year retesting interval, just like women with a negative Pap-alone. For women aged 60-64, however, our data suggest that HPV-negative/ASC-US findings should be investigated, and not used in place of negative Pap results to qualify a woman to exit screening.
We also noted a strong decline in HPV positivity of ASC-US by increasing age of women being tested, as observed in other settings. Because age is strongly associated with HPV prevalence in women with ASC-US Pap results, we examined whether the effectiveness of HPV triage was affected by age. The main finding was that HPV testing was predictive at all ages; the risks of CIN3+ found by HPV triage of ASC-US were quite similar for ages 25-29 and 30-34.
A major strength of this investigation was the very large number (~40,000) of ASC-US results and use of a single HPV testing method (HC2). Nonetheless, there were limitations: Because biopsy information was only collected through 2010, we had too few data points to separately estimate risks based on liquid-based Pap results. Note that a meta-analysis (16) and 2 large randomized clinical trials (17, 18) have failed to show any clinical performance advantage of liquid-based Pap tests over conventional Pap smears for detection of CIN3+. The data were derived from a study population in Northern California. Of note, KPNC cares for more than 3.2 million persons (approximately 30% of the population in 14 Northern California counties) who are broadly representative of the local and statewide population (with the exception of a slight underrepresentation of the extremes of income) (19). This is a retrospective cohort study with incomplete follow-up due to reliance on passive surveillance of women per usual clinical protocols. Although three-quarters of HPV-negative/ASC-US women returned within 1.5 years, follow-up was incomplete, as would be expected in a clinical practice, partly as a result of changes in KPNC membership.
In conclusion, in clinical practice, HPV-positive/ASC-US was as risky as LSIL Pap results, and HPV-negative/ASC-US conferred a very low CIN2+/CIN3+ risk result for all women, and a low cancer risk for women under age 60. Using the principle of “equal management of equal risks”, women with HPV-positive/ASC-US women should (like those with LSIL) be referred to colposcopy. Using the same principle, women with HPV-negative/ASC-US should (like those with a negative Pap) be re-tested at 3 years. There may be an increased risk of cancer among women age 60-64 with HPV-negative/ASC-US; before these women exit screening, their HPV-negative/ASC-US findings require careful consideration.
Acknowledgments
Role of the funding source The Intramural Research Program of the US National Institutes of Health/National Cancer Institute and Kaiser Permanente Northern California reviewed the final manuscript for publication. The Kaiser Permanente Northern California Institutional Review Board (IRB) approved use of the data, and the National Institutes of Health Office of Human Subjects Research deemed this study exempt from IRB review.
Footnotes
Conflicts of Interest: Dr. Schiffman and Dr. Gage report working with Qiagen, Inc. on an independent evaluation of non-commercial uses of CareHPV (a low-cost HPV test for low-resource regions) for which they have received research reagents and technical aid from Qiagen at no cost. They have received HPV testing for research at no cost from Roche. Dr. Castle has received compensation for serving as a member of a Data and Safety Monitoring Board for HPV vaccines for Merck. Dr. Castle has received HPV tests and testing for research at a reduced or no cost from Qiagen, Roche, MTM, and Norchip. Dr. Castle is a paid consultant for BD, GE Healthcare, and Cepheid, and has received a speaker honorarium from Roche. No other authors report any conflicts of interest.
NOTE: For the cancer panel, there were too few cancers in women with LSIL to present 5-year risk estimates.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Results of a randomized trial on the management of cytology interpretations of atypical squamous cells of undetermined significance. Am J Obstet Gynecol. 2003;188(6):1383–92. doi: 10.1067/mob.2003.457. [DOI] [PubMed] [Google Scholar]
- 2.Arbyn M, Sasieni P, Meijer CJ, Clavel C, Koliopoulos G, Dillner J. Chapter 9: Clinical applications of HPV testing: a summary of meta-analyses. Vaccine. 2006;24(Suppl 3):S3, 78–89. doi: 10.1016/j.vaccine.2006.05.117. [DOI] [PubMed] [Google Scholar]
- 3.Einstein MH, Martens MG, Garcia FA, Ferris DG, Mitchell AL, Day SP, et al. Clinical validation of the Cervista HPV HR and 16/18 genotyping tests for use in women with ASC-US cytology. Gynecol Oncol. 2010;118(2):116–22. doi: 10.1016/j.ygyno.2010.04.013. [DOI] [PubMed] [Google Scholar]
- 4.Solomon D, Davey D, Kurman R, Moriarty A, O’Connor D, Prey M, et al. The 2001 Bethesda System: terminology for reporting results of cervical cytology. JAMA. 2002;287(16):2114–9. doi: 10.1001/jama.287.16.2114. [DOI] [PubMed] [Google Scholar]
- 5.Stoler MH, Wright TC, Jr., Sharma A, Apple R, Gutekunst K, Wright TL. High-Risk Human Papillomavirus Testing in Women With ASC-US Cytology: Results From the ATHENA HPV Study. Am J Clin Pathol. 2011;135(3):468–75. doi: 10.1309/AJCPZ5JY6FCVNMOT. [DOI] [PubMed] [Google Scholar]
- 6.Wright TC, Jr., Stoler MH, Sharma A, Zhang G, Behrens C, Wright TL. Evaluation of HPV-16 and HPV-18 genotyping for the triage of women with high-risk HPV+ cytology-negative results. Am J Clin Pathol. 2011;136(4):578–86. doi: 10.1309/AJCPTUS5EXAS6DKZ. [DOI] [PubMed] [Google Scholar]
- 7.Saslow D, Solomon D, Lawson HW, Killackey M, Kulasingam SL, Cain JM, et al. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. J Low Genit Tract Dis. 2012;16(3):175–204. doi: 10.1097/LGT.0b013e31824ca9d5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Saslow D, Solomon D, Lawson HW, Killackey M, Kulasingam SL, Cain J, et al. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. CA Cancer J Clin. 2012;62(3):147–72. doi: 10.3322/caac.21139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Castle PE, Solomon D, Schiffman M, Wheeler CM. Human papillomavirus type 16 infections and 2-year absolute risk of cervical precancer in women with equivocal or mild cytologic abnormalities. J Natl Cancer Inst. 2005;97(14):1066–71. doi: 10.1093/jnci/dji186. [DOI] [PubMed] [Google Scholar]
- 10.Katki HA, Kinney WK, Fetterman B, Lorey T, Poitras NE, Cheung L, et al. Cervical cancer risk for women undergoing concurrent testing for human papillomavirus and cervical cytology: a population-based study in routine clinical practice. Lancet Oncol. 2011;12(7):663–72. doi: 10.1016/S1470-2045(11)70145-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Safaeian M, Solomon D, Wacholder S, Schiffman M, Castle P. Risk of precancer and follow-up management strategies for women with human papillomavirus-negative atypical squamous cells of undetermined significance. Obstet Gynecol. 2007;109(6):1325–31. doi: 10.1097/01.AOG.0000263461.71732.40. [DOI] [PubMed] [Google Scholar]
- 12.Schiffman M, Wentzensen N, Wacholder S, Kinney W, Gage JC, Castle PE. Human papillomavirus testing in the prevention of cervical cancer. J Natl Cancer Inst. 2011;103(5):368–83. doi: 10.1093/jnci/djq562. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Katki HA, Schiffman M, Castle PE, Fetterman B, Poitras NE, Lorey T, et al. Benchmarking CIN3+ risk as the basis for incorporating HPV and Pap cotesting into cervical screening and management guidelines. J Low Genit Tract Dis. doi: 10.1097/LGT.0b013e318285423c. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lawless JF. Statistical models and methods for lifetime data. 2nd ed Wiley-Interscience; Hoboken, N.J.: 2003. [Google Scholar]
- 15.Turnbull B. The empirical distribution function with arbitrarily grouped, censored and truncated data. J Roy Stat Soc B. 1976;38:290–5. [Google Scholar]
- 16.Arbyn M, Bergeron C, Klinkhamer P, Martin-Hirsch P, Siebers AG, Bulten J. Liquid compared with conventional cervical cytology: a systematic review and meta-analysis. Obstet Gynecol. 2008;111(1):167–77. doi: 10.1097/01.AOG.0000296488.85807.b3. [DOI] [PubMed] [Google Scholar]
- 17.Siebers AG, Klinkhamer PJ, Grefte JM, Massuger LF, Vedder JE, Beijers-Broos A, et al. Comparison of liquid-based cytology with conventional cytology for detection of cervical cancer precursors: a randomized controlled trial. JAMA. 2009;302(16):1757–64. doi: 10.1001/jama.2009.1569. [DOI] [PubMed] [Google Scholar]
- 18.Ronco G, Cuzick J, Pierotti P, Cariaggi MP, Dalla Palma P, Naldoni C, et al. Accuracy of liquid based versus conventional cytology: overall results of new technologies for cervical cancer screening: randomised controlled trial. BMJ. 2007;335(7609):28. doi: 10.1136/bmj.39196.740995.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gordon NP. Does the Adult Kaiser Permanente Membership in Northern California Compare with the Larger Community? Kaiser Permanente Division of Research; Oakland, CA: 2006. [Google Scholar]