A Multicountry Evaluation of careHPV Testing, Visual Inspection With Acetic Acid, and Papanicolaou Testing for the Detection of Cervical Cancer

Supplemental digital content is available in the text.

Abstract

Objective

This study evaluates the feasibility and performance of careHPV, a novel human papillomavirus (HPV) DNA test, when used for screening women for cervical cancer in low-resource settings.

Methods and Materials

Clinician-collected (cervical) and self-collected (vaginal) careHPV specimens, visual inspection with acetic acid (VIA), and Papanicolaou test were evaluated among 16,951 eligible women in India, Nicaragua, and Uganda. Women with positive screening results received colposcopy and histologic follow-up as indicated. The positivity of each screening method was calculated overall, by site, and age. In addition, the clinical performance of each screening test was determined for detection of cervical intraepithelial neoplasia (CIN) grade 2 (CIN2+) and CIN grade 3.

Results

Moderate or severe dysplasia or cancer (taken together as CIN2+) was diagnosed in 286 women. The positivity rate ranged between 2.4% to 19.6% for vaginal careHPV, 2.9% to 20.2% for cervical careHPV, 5.5% to 34.4% for VIA, and 2.8% to 51.8% for Papanicolaou test. Cervical careHPV was the most sensitive for CIN2+ (81.5%; 95% confidence interval [CI], 76.5–85.8) and CIN grade 3 (85.3%; 95% CI, 78.6–90.6) at all sites, followed by vaginal careHPV (69.6% and 71.3%, respectively). The sensitivity of VIA ranged from 21.9% to 73.6% and Papanicolaou test from 40.7% to 73.7%. The pooled specificities of cervical careHPV, vaginal careHPV, VIA, and Papanicolaou test were 91.6%, 90.6%, 84.2%, and 87.7%, respectively.

Conclusions

careHPV performed well in large multicountry demonstration studies conducted in resource-limited settings that have not previously been conducted this type of testing; its sensitivity using cervical samples or vaginal self-collected samples was better than VIA or Papanicolaou test. The feasibility of using careHPV in self-collected vaginal samples opens the possibility of increasing coverage and early detection in resource-constrained areas.

Globally, cervical cancer is the third leading cancer and fourth cause of cancer-related mortality in women.1 There is a high disparity for cervical cancer between higher-income and lower-income regions, with more than 85% of cervical cancer occurring in low- and middle-income countries.1 This difference is primarily due to the difficulty in implementing Papanicolaou test–based screening programs because of their complexity, need for highly trained providers for reading the samples, need for close quality control, and mediocre sensitivity of the test even in optimal conditions.2

Based on the absolute etiologic link between carcinogenic human papillomavirus (HPV) and cervical cancer, 2 new approaches for the prevention of cervical cancer have emerged, these are as follows: (1) HPV vaccination for preventing incident HPV infection in younger women3,4 and (2) carcinogenic HPV detection for screening for cervical precancer and cancer.5–11 Both vaccine and screening have demonstrated high degrees of efficacy in prevention of HPV infection or detection at a treatable stage, with maximum effectiveness when guided by an understanding of the causal model and applied in an age-appropriate manner.12 Although prophylactic HPV vaccination may be the ultimate prevention strategy, these vaccines do not treat preexisting HPV infections and precancerous conditions.13–15 Therefore, there are millions of at-risk women who will not benefit from HPV vaccination, and robust screening and management programs developed for low- and middle-income countries are needed to reduce the burden of cervical cancer. Additionally, because the HPV vaccines do not protect against all the oncogenic genotypes of the virus, screening is still required even among vaccinated cohorts.

Several new screening strategies have emerged as options for areas with limited resources. Visual inspection with acetic acid (VIA) is a method based on the use of 5% acetic acid (vinegar) that, when applied to the cervix, makes the dysplastic epithelium turn white (acetowhitening), becoming visible on evaluation by the unaided eye. The sensitivity of VIA is variable; 2 recent meta-analyses have reported sensitivity of 70% to 80% for cervical intraepithelial neoplasia (CIN) grade 2 (CIN2+) or more severe diagnoses.16,17 One large randomized clinical trial in India found a 35% reduction in cervical cancer-related mortality after a single screening by VIA,18 whereas a second trial did not find a statistically significant reduction8 in cervical cancer-related mortality compared with those randomized to the no-intervention arm of seeking standard services.

Another approach that has recently become available is a lower-cost DNA test for detection of carcinogenic genotypes of HPV.19 Recent studies have shown that HPV testing used in a screen-and-treat approach was more effective than VIA in reducing the prevalence of CIN2+,20 and when used in a more traditional program (ie, colposcopy and excisional treatment of histologically confirmed CIN2+), it was more effective in reducing cervical cancer mortality than VIA and Papanicolaou test.8 In response to the need for more robust screening tools for low- and middle-income countries, a public-private collaboration led to the development of careHPV (QIAGEN, Gaithersburg, MD), a simplified, robust, and affordable HPV test that could be used in low-resource settings under a wider range of ambient conditions. The test can be run in any room because it does not need running water or air conditioning, and the process is simple and can be completed by people with limited laboratory training. Preliminary results for careHPV were promising and compared favorably to the US Food and Drug Administration–approved Hybrid Capture 2 (hc2; QIAGEN).21

Human papillomavirus DNA testing offers the possibility of using self-collected vaginal samples for primary screening. The advantages of self-collection are that it does not require pelvic evaluation; therefore, the sample collection process could be completed without the need for a speculum or even a health center facility because the sampling can be done at the community level. A recently pooled analysis of data from 5 studies in China found that HPV testing of self-collected specimens was at least as sensitive for CIN2+ and CIN grade 3 (CIN3+) as liquid-based cytology using clinician-collected specimens,22 and another review showed that population coverage can be increased by offering self-sampling.23

Further studies were needed to demonstrate whether careHPV truly had wide applicability under a variety of settings and how it compared with the 2 standards, VIA and Papanicolaou testing, in a more realistic setting. We therefore conducted a multisite study in routine public health settings in India, Nicaragua, and Uganda using existing staff and resources to assess the clinical performance of careHPV with both clinician-collected and self-collected specimens, VIA, and Papanicolaou testing for detection of cervical precancer and cancer.

MATERIALS AND METHODS

Screening Technologies to Advance Rapid Testing–Utility and Program Planning (START-UP) study is a multisite demonstration project study conducted in India (Hyderabad and rural Uttar Pradesh), Nicaragua (Masaya Province), and Uganda (Kampala). Study protocols for each site were similar and were approved by institutional review boards in all project sites and the United States.

Efforts to promote enrollment in the study and meet screening targets (5000 women from each site) used population-based recruitment strategies that were specifically tailored to each site. Local community leaders and stakeholders were contacted in advance to gain their support in the mobilization and education of the population. Women were then invited to participate in the study using a variety of methods that differed by study site (Table 1, Supplemental Digital Content 1, http://links.lww.com/IGC/A201).

In India, women living in urban slums were invited to enroll in the study. Screenings occurred at urban health centers or mobile health clinics in Hyderabad, Andhra Pradesh, whereas in Uttar Pradesh, women were recruited from rural areas at the Dadri Tehsil of Gautam Buddha Nagar District to be screened at primary health centers and/or subcenters. In Nicaragua, women from rural and semirural areas within Masaya Province were invited to be screened at 1 of the following 3 stationary health centers: Monimbo, La Concepcion, and Tisma. In Uganda, women were selected from 5 divisions in Kampala and Wakiso district to attend screenings at Mulago Hospital, the national referral hospital.

All recruited women were assessed for eligibility. Age inclusion criteria differed slightly among sites to align and support local health practices and guidelines. In both Hyderabad and Nicaragua, the age inclusion criteria were 30 to 49 years old; in rural Uttar Pradesh, 30 to 59 years old; and in Uganda, 25 to 60 years old. Similarly, in both Nicaragua and Uganda, eligibility was restricted to women who reported having sexual intercourse in the past. However, in the India sites, having been married was used as a proxy for a history of having sexual intercourse because it was culturally more appropriate. In all sites, women were excluded if they had a history of CIN or cervical cancer, a hysterectomy that removed the cervix, and any condition that would pose a health risk to the participant or interfere with the evaluation of the study objectives such as a history of bleeding or clotting disorders, debilitating physical and/or mental illness, and inability to provide informed consent. Pregnancy was considered a temporary exclusion until 3 months postdelivery. All participants provided written consent before the screenings began. For women who were not able to read or write, the staff read the consent aloud to them and women provided a thumbprint to indicate consent in the presence of a witness.

All eligible and consented women were offered 4 screening tests. First, they were instructed (by a female study staff) how to obtain a self-collected vaginal sample for careHPV testing and were invited to provide a sample in a private room or in the presence of study staff, depending on their preference. Next, they underwent a pelvic examination using a speculum during which additional cervical samples were taken for careHPV and conventional Papanicolaou test. The careHPV specimen for both vaginal and cervical samples was collected with careHPV cervical sampler into Digene collection media (DCM). Finally, VIA was performed. Screening tests were administered per protocol, and results were acquired using standardized methodology. Before testing, careHPV specimens were stored at room temperature (15°C to 30°C) for maximum of 14 days, 2°C to 8°C for maximum of 30 days, or −20°C for maximum of 60 days.

In careHPV test results, represented as a ratio of viral load expressed in relative light units (RLU), compared with the mean RLU from a positive control set at 1 pg/mL cutoff (CO),21 the equipment had been set by the manufacturer to consider positive if the RLU/CO value was 1.0 and above. Papanicolaou tests were graded by local pathologists according to the Bethesda classification system, and any smear with atypical cells of undetermined significance or more severe changes was considered positive. Lastly, epithelia that turned white with acetic acid were classified as VIA positive following standard VIA criteria.24

The careHPV testing was done locally at each study site. The conditions for testing varied by site; in 1 site (Nicaragua), the testing was done in a small laboratory in a health center. In Uganda and Uttar Pradesh, the testing was done in offices, and in Hyderabad, the testing was done in a storage room. Air conditioning was not available at the testing facilities except for in Nicaragua. Most of the required supplies are included in the test kits; some supplies such as paper blotting towels were procured locally in each country.

Women who resulted negative on all screening tests were advised to be screened again in 3 years, and those who resulted positive on any of the screening tests were referred to colposcopy and directed biopsy of any abnormal epithelial area and/or endocervical curettage, as necessary for diagnosis. Women with moderate to severe CIN (CIN2+) were treated with cryotherapy as eligible and/or referred for further management.

To improve histologic quality control, a 10% random sample (via random number table) of negative biopsies and all CIN2+ positive histologic specimens were independently reviewed by an external pathologist from Canada who was blinded to the original diagnosis. Given that the agreement between the local and external pathologist in Hyderabad, rural Uttar Pradesh, and Nicaragua was 90.8% (κ = 0.71), 93.5% (κ = 0.79), and 75.5% (κ = 0.53), respectively, the local pathologist readings were used to assess the final diagnosis. On the other hand, the agreement for Uganda was 54.5% (κ = 0.22). As a result, all histologic slides from Uganda were requested for reevaluation by the Canadian pathologist, whose diagnosis was used as the final diagnosis. We assumed that women with negative results on all screening tests were truly negative for any precancer or cancer.

The positivity of each screening method was calculated overall by site and age. Also, the percent test positive by all 4 screening tests at each clinical site by severity of histologic diagnosis was determined. In addition, the clinical performance (sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], and test positivity) of each screening test was determined for detection of CIN2+ and CIN3+, along with the exact binomial 95% confidence interval (95% CI). Finally, the overall (4 clinical sites combined) clinical performance and 95% CI for detection of cervical CIN2+ and CIN3+ diagnosis were calculated. McNemar χ² test was used to assess differences in sensitivity and specificity for CIN2+ and CIN3+ for paired test results, and multiple test comparisons were compared using Bonferroni-adjusted McNemar χ².

All statistical analyses were conducted in Stata 12.0 (Statacorp, College Station, TX).

RESULTS

From the 4 sites in 3 countries, a total of 20,461 women were consented and screened (Fig. 1). After excluding those who did not meet the eligibility criteria (n = 58) or did not complete all screenings (n = 1063), there were 19,340 (94.5%) eligible, consenting women who completed all screenings. Of the 19,340 women, 2389 were lost to follow-up or had missing histologic results, leaving 16,951 (87.6%) women to be included in this analysis. The breakdown by country was 4658 (94.6%) of 4925 completely screened women from rural Uttar Pradesh, India, 4502 (93.8%) of 4799 from Hyderabad, India, 4645 (94.7%) of 4906 from Nicaragua, and 3146 (66.8%) of 4710 from Uganda.

FIGURE 1.

¹Age outside 25 to 60 years (n = 20), had hysterectomy (n = 24), pregnant (n = 9), previous CIN/cancer (n = 5), no sexual experience (Uganda and Nicaragua; n = 0), not married (India only; n = 0), or interfering health risk/condition (n = 0). ²Vaginal careHPV not tested within protocol timeframe (Nicaragua only; n = 461), missing careHPV vaginal (0.5 CO; n = 271), missing careHPV cervical (0.5 CO; n = 147), missing VIA (n = 75), or missing Papanicolaou test (n = 109). ³[+] indicates screen positive and [−] indicates screen negative.

The mean (SD) age of women was similar in all sites: rural Uttar Pradesh, 37.9 (7.5) years; Hyderabad, 36.3 (5.9) years; Nicaragua, 37.9 (6.1) years; and Uganda, 36.8 (8.4) years. Most of the women in rural Uttar Pradesh (97.2%) and Hyderabad (94.1%) were married (proxy measure for sexual experience), and all (100%) women in Nicaragua and Uganda reported having had sexual experience or intercourse.

The percent test positive varied greatly by screening test and clinical site, but there was a strong correlation within clinical site (ie, the percent test positive for all tests tended to cluster by clinical site; Fig. 2). In rural Uttar Pradesh, the percent test positive for vaginal HPV, cervical HPV, VIA, and Papanicolaou testing were 2.4%, 2.9%, 5.5%, and 2.8%, respectively. In Hyderabad, the percent test positive for vaginal HPV, cervical HPV, VIA, and Papanicolaou testing were 5.9%, 6.3%, 9.3%, and 3.5%, respectively. In Nicaragua, the percent test positive for vaginal HPV, cervical HPV, VIA, and Papanicolaou testing were 15.4%, 12.6%, 22.5%, and 6.4%, respectively. In Uganda, the percent test positive for vaginal HPV, cervical HPV, VIA, and Papanicolaou testing were 19.6%, 20.2%, 34.4%, and 51.8%, respectively. Notably, there was also a correlation between percent test positive and the prevalence of CIN2+.

FIGURE 2.

Percent test positive by screening test and clinical site.

The age-group specific patterns of percent test positive are shown in Figure 3. Generally, the percent test positive by VIA was much higher than that for other tests in the youngest age groups. The percent test positive by VIA declined sharply with increasing age (p_trend < 0.001 for rural Uttar Pradesh, Nicaragua, and Uganda; p_trend = 0.04 for Hyderabad) so it converged with the percent test positive for the other tests. The percent test positive for vaginal and cervical careHPV decreased with increasing age in Nicaragua (p_trend = 0.12 for both) and Uganda (p_trend < 0.001 for both) but did not change significantly with age for the two India sites. The percentage of positive Paps did not change with age except for those read in Uganda, in which the percentage of positive Paps was very high and surprisingly increased with increasing age (p_trend < 0.001).

FIGURE 3.

Percent test positive by age group.

Table 1 shows the percentage of women who resulted positive for each test by geographic site and by severity of the histologic diagnosis. As a rule, the percentage of women who resulted positive for all tests increased with increasing severity of histologically confirmed disease, with the only exception being Papanicolaou testing in Uganda, where the percentage of positive tests was greatest among those with CIN1 and then declined with increasing severity of diagnosis. Among women with negative histologic finding who had a positive screening test, VIA was the most common false-positive in rural Uttar Pradesh, Hyderabad, and Nicaragua, and Papanicolaou test had the highest false-positive rate in Uganda. Women with CIN3+ were most likely to result positive by cervical careHPV at all sites (range, 82.5% to 89.8%).

TABLE 1.

Percent test positive by each of the 4 tests at each clinical site by severity of histologic diagnosis

The clinical performance characteristics of the 4 tests are reported in Table 2 for CIN2+ and CIN3+ by geographic site. Cervical careHPV was the most sensitive and had the best NPV for CIN2+ and CIN3+ at all clinical sites. Vaginal careHPV typically had the second or third best sensitivity and NPV. Papanicolaou test performance was quite variable; Papanicolaou testing was very accurate with high sensitivity and specificity in Hyderabad, resulting in a very high PPV, whereas it was very insensitive in Nicaragua and nonspecific in Uganda. Visual inspection with acetic acid had the lowest specificity and PPV at all sites and had very poor sensitivity in rural Uttar Pradesh (21.9% for CIN2+ and 7.69% for CIN3+).

TABLE 2.

Clinical performance of the 4 screening tests for detection of CIN2+ and CIN3+

We noted an important trade off in sensitivity and false positivity (1 specificity) of VIA for CIN2+ and CIN3+ by clinical site (Figure 1, Supplemental Digital Content 2, http://links.lww.com/IGC/A200). The more sensitive VIA was read, the less specific and the greater percentage of false-positives. Interestingly, there was a correlation with the population risk of CIN2+; the higher the risk of CIN2+, the greater was the sensitivity, the lower the specificity, and the greater the false positivity.

Table 3 provides a summary of the combined, multisite clinical performance of the 4 testing modalities for detection of CIN2+ and CIN3+. Cervical careHPV had the highest sensitivity (81.5% for CIN2+ and 85.3% for CIN3+) and specificity (91.6% for CIN2+ and 91.0% for CIN3+) in this study for both end points (P < 0.01, unadjusted and Bonferroni-adjusted McNemar χ² for all comparisons) and therefore had the best PPV and NPV as well. Vaginal careHPV had the next highest sensitivity, specificity, PPV, and NPV for CIN2+ specificity (P < 0.05, unadjusted and Bonferroni-adjusted McNemar χ² vs VIA and Papanicolaou test) and consequently the next highest PPV and NPV.

TABLE 3.

Overall (4 clinical sites combined) clinical performance (sensitivity, specificity, PPV, NPV, and test positivity) for detection of CIN2+ and CIN3+

DISCUSSION

We reported on a multisite study to compare different cervical cancer screening options in low- and middle-income countries. We found that the objective tests, HPV DNA testing using clinician-collected or self-collected specimens, had better clinical performance than the subjective tests, VIA and Papanicolaou test. It is increasingly clear that there is a “menu of options” for choice of screening tests, with trade offs in clinical performance versus costs and infrastructure requirements. Although additional costing evaluations are under development to explore the cost per CIN2+ detected and treated, it is important to emphasize that “one size does not fit all,” and in countries with limited screening, the goal should be to explore new approaches to expand current screening. The choice of test has important downstream implications for patient management and care. Although one of the criticisms of HPV DNA testing is its low specificity, in our study, VIA was less specific and had lower PPV than careHPV. As a consequence, a VIA-based program will refer 50% and above more women to either colposcopy in more traditional screening programs in which diagnostic verification is required or to unnecessary treatment in a screen-and-treat program, such as the trial conducted in South Africa.25

We noted that the performance of all of the tests was site dependent, even for nonsubjective tests such as careHPV. We also observed that the test positivity (and therefore specificity) for all 4 tests tended to cluster by site rather than method. This was presumably due to differences by site in exposure to the underlying cause of cervical cancer, carcinogenic HPV infections, and seemed to be correlated with the prevalence of precancerous lesions as markers of cancer risk. A previous report found that the prevalence of carcinogenic HPV correlated with cervical cancer incidence.26

We also found that the performance of VIA differed greatly by geographic site, with a strong inverse relationship of specificity with sensitivity for VIA (ie, the more specific VIA performed, the less sensitive it was). Our results contradict a previous meta-analysis which reported that VIA accuracy was not influenced by study region or capacity of screener.12 We can only speculate on possible explanations. First, one of the challenges in assessing the true performance of VIA is that disease ascertainment is based on colposcopy and directed biopsies, which rely on the same acetowhitening phenomenon, leading to inflated sensitivity due to autocorrelation. The performance of all tests was different in 1 of the study sites (Uttar Pradesh). Because all screening tests were affected in this site, we hypothesized that there was a local factor influencing it. The study allocated additional funding and effort to review all the steps and processes in this site, but we could not find a factor for these results.

Our results also highlight the importance of having adequate local conditions at the time of implementing any cervical cancer screening program. A good example was the performance of Papanicolaou test in our Uganda site; although the project was careful to select a senior pathologist for the local reading of the Papanicolaou tests, the performance of that local expert was inadequate and the results were clearly unreliable.

We found that Papanicolaou testing at the Hyderabad site performed better than at the other sites. We conducted a post hoc analysis in which we had a second pathologist in Hyderabad reread a subset of the Papanicolaou tests (n = 994) without knowledge of the ongoing study. Restricted to the subset read by both pathologists, the second read was less sensitive (20.8% vs 81.8%) and slightly more specific (98.6% vs 97.2%) for CIN2+ than the original study reading (Table 2, Supplemental Digital Content 3, http://links.lww.com/IGC/A201). The change in performancebetween the pathologists may be the variability because of the subjective nature of the test and/or the impact of a priori knowledge of the study purpose versus routine practice. In some studies, very high sensitivity for cytology has been reported, and we wonder if it is a study artifact. However, we cannot differentiate between the 2 possible explanations, and they should be the subject of further investigation.

We acknowledge that our study was limited with respect to verification bias; women who resulted negative and had no visible lesions were not referred to colposcopy and were not biopsy examined. As a result, our results may overestimate the clinical sensitivity of the tests evaluated and the results. However, among the women who resulted positive, we highly encouraged colposcopists to take a biopsy even if only a slight abnormality was observed during colposcopy. Furthermore, both vaginal and cervical careHPV were performed on all women in conjunction with VIA and Papanicolaou test; it is very likely that most women with CIN2+ were detected.

The careHPV test is more sensitive for detecting CIN2+ and CIN3+ than VIA or Papanicolaou test. Hence, we conclude that it is possible to perform high-quality tests for HPV DNA in urban and rural areas in low-resource settings. careHPV adds another screening option for low-resource settings with the advantage of having a very high sensitivity for detecting precancer and cancer and the feasibility of performing cervical cancer screening without the need for a pelvic evaluation. It is important to consider that vaginal careHPV testing has lower sensitivity than cervical careHPV, as reported by previous studies,21 but the loss on sensitivity has to be evaluated in the context of potential for expansion of screening coverage if vaginal samples self-collected by women are used for primary screening.

Finally, it is important to emphasize that the performance of the screening test is important, but the screening test is only 1 component of the population-based program required to affect cervical cancer incidence and mortality.