Abstract
BACKGROUND:
HPV-based screen-and-treat is the recommended approach for cervical cancer screening in low resource settings, but relatively low specificity of HPV testing, particularly in women living with HIV leads to over-treatment. We evaluated whether HPV type restriction and more stringent cut-offs on Xpert HPV optimizes performance characteristics of this assay for screen-and-treat.
METHODS:
In a clinical study in Cape Town, South Africa, 586 HIV-negative and 535 HIV-positive women, aged 30–65 years, were recruited. All women had cervical samples collected for Xpert HPV and underwent colposcopy and histological sampling with consensus pathology review. Logistic regression and receiver operating characteristic curves (ROC) were used to evaluate improvements in specificity attained by modifying cycle threshold (Ct) cut-offs to define screen-positive.
RESULTS:
Sensitivity of detecting cervical intraepithelial lesions grade 2 or greater (CIN2+) at manufacturer-defined Ct cut-offs for all channels was 88.7% (95% CI: 83.1–94.3) and specificity 86.9% (95% CI: 83.4–90.4) in HIV-negative women. Sensitivity was 93.6% [95% CI: 90.0–97.3] and specificity 59.9% [95% CI: 54.1–65.7] in HIV-positive women. Ct values from channels detecting (1) HPV 16, (2) HPV 18 and/or 45, and (3) HPV 31, 33, 35, 52, and/or 58 were informative to predict CIN2+. Shifting Ct cut-offs on these three channels allowing sensitivity to decline to 85–75%, led to specificities of 91.3–95.3% in HIV-negative and 77.0–85.8% in HIV-positive women.
CONCLUSIONS:
More stringent Ct cut-offs on selected channels in Xpert HPV improve specificity with only modest losses in sensitivity making this assay an optimal choice for HPV-based screen-and-treat in settings with a high prevalence of HIV. These modifications can be made from standard output with no need for new engineering. Decision-making about performance characteristics of HPV testing can be shifted to program implementers and cutoffs selected for appropriate to resource availability and community preferences.
Introduction
Cervical cancer screening programs have been ineffective in reducing incidence and mortality of cervical cancer in low and middle-income countries (LMIC).(1) Failure of screening programs is largely due to the substantial attrition that occurs over the multi-step process required by the traditional model of screening. In addition, lack of resources, including insufficient colposcopy, cytology and pathology services, introduce delays and bottlenecks that result in inefficient and ineffective programs.(2–4)
Historically, cervical cytology has been the standard screening test and in countries with a robust healthcare infrastructure that can support this resource-intensive screening modality, the cumulative incidence of cervical cancer has been significantly reduced.(5) However, this approach has not been successfully implemented in LMIC and the World Health Organization (WHO) recommends against cytology as a screening modality for countries without pre-existing cytology-based programs.(6) Since the early 1990s, several investigators have evaluated alternative screening approaches to overcome the inefficiencies and bottlenecks of the traditional multi-step approach.(7–12) In the most efficient approach, screening and treating are done at a single visit.(13) The diagnostic step involving colposcopy and histological diagnosis is circumvented. In this approach, referred to as screen-and-treat, specificity of the screening test takes on added salience since this parameter determines the extent of over-treatment.
With all screening and diagnostic tests, there is an inevitable trade-off between sensitivity and specificity. Specificity can be improved if reductions in sensitivity can be tolerated. Assays to detect HPV DNA have high sensitivity hence their utility for primary screening in high resource settings.(14, 15) Their high sensitivity also increase their utility in LMIC where screening intervals are infrequent. For the screen-and-treat approach, HPV tests need to be run at the point-of-care in order to optimize participant retention. In addition, since a positive result flags a screening participant for immediate treatment, concerns have been raised about the relatively low specificity of HPV DNA testing, particularly in high risk groups such as HIV-positive women,(16, 17) the resources needed to treat large numbers of women and risks of morbidity associated with treatment.
Xpert HPV is a WHO pre-qualified test that can be run as a point-of-care test, making single-visit, HPV-based screen-and-treat a viable reality.(18–23) Here we report results of a clinical study in South Africa to evaluate whether Xpert HPV can be optimized by selecting only certain high risk HPV types and changing cut-offs on the assay to improve specificity without producing major reductions in sensitivity in both HIV-negative and HIV-positive women.
Methods
Study population
We conducted a clinical study of 586 HIV-negative and 535 HIV-positive women, aged 30–65 years, recruited at two sites in Cape Town, South Africa, between 16 February 2015 and 16 May 2016. At a primary care facility (Khayelitsha Site B Day Hospital, Western Cape Department of Health facility) we recruited 382 HIV-negative and 333 HIV-positive women through community out-reach aiming to be representative of healthy women seeking routine screening. At a referral colposcopy clinic (Groote Schuur Hospital [GSH], tertiary hospital of the University of Cape Town) we recruited 204 HIV-negative and 202 HIV-positive women referred to colposcopy because of abnormal cytology results obtained as part of existing screening services in the region. Other inclusion criteria were (1) no history of any anogenital cancer (2) no history of treatment for cervical dysplasia; (3) no hysterectomy; and (4) not pregnant. All women provided written informed consent. The study protocol was approved by the Institutional Review Boards of Columbia University and the University of Cape Town.
This design was chosen as the most efficient to yield a large enough number of women with cervical cancer precursor lesions. A sample size of 600 women from the screening population and 400 women from the referral population (equal numbers of HIV-positive and negative women in each group) was selected a priori based on estimates of sensitivity, specificity and CIN2+ prevalence in the four groups with the aim of attaining a precision of +/− 6% around the sensitivity estimates and +/− 5% around the specificity estimates stratified by HIV status. Enrollment proceeded more rapidly than planned and slightly higher numbers were attained. The referral population was included in order to generate sufficient numbers of women with precursor lesions required for sensitivity calculations (enrichment principle).
Clinical Protocol
All women had a pelvic examination performed by one of the study physicians during which two cervical samples were obtained, each collected using an extended tip plastic spatula with endocervical cytobrush (Medscand, Berlin, Germany). The samples were placed in separate 20ml vials of PreservCyt (Hologic, Bedford, MA). This was followed by colposcopy with cervical biopsies of any abnormal-appearing areas of the cervix. If a woman had no visible lesions on the cervix, an endocervical curettage (ECC) was performed. A loop electrosurgical excision procedure (LEEP) was performed if clinically indicated. Women recruited at the primary care facility were scheduled for follow-up 6 weeks after enrollment. Those with positive HPV test results or cervical intraepithelial neoplasia grade 1 (CIN1) on baseline histology were seen for two subsequent colposcopies to rule out missed disease. If any lesion was seen, it was biopsied or LEEP was performed. If no lesion was seen, a random biopsy was obtained from the 12 o’clock position. After initial pathology review in Cape Town for clinical management, all histology slides were shipped to Columbia University and re-reviewed. Results discordant with initial reads were reviewed by a third gynecologic pathologist as the tie-breaker for the final endpoint determination.
HPV Assay
Xpert HPV™ (Cepheid, Sunnyvale, CA) is a real-time PCR assay that detects 14 high-risk HPV types in five fluorescent channels: (1) HPV 16, (2) HPV 18, 45, (3) HPV 31, 33, 35, 52, 58, (4) HPV 51, 59, and (5) HPV 39, 56, 66, 68. Output from the assay includes a cycle threshold (Ct) value from each of these five channels. Ct values have been determined by the manufacturers to define a positive result for each channel (cut-off of 40 for the HPV16 and HPV 18, 45 channels and 38 for all other channels). With PCR assays, a high Ct value means a low viral load (many cycles needed to become positive) and a low Ct means a high viral load (fewer cycles needed to become positive). Additionally, a sample adequacy control detects whether the sample contains human DNA in a sixth channel.
The clinician-collected cervical sample was tested with Xpert HPV. A four-module GeneXpert machine (Cepheid, Sunnyvale, CA) was installed at the primary care site in Khayelitsha and a community health worker, with no prior laboratory experience, was trained in running the test. Samples collected from women recruited at the primary care site were run as soon as possible after collection and results reported to participants. Samples collected from women recruited at the colposcopy clinic were transported to the site the next day and run on the same machine. The test requires 1ml of sample and residual material was stored.
Analysis
For calculation of sensitivity, cervical intraepithelial neoplasia grade 2 or worse (CIN2+) or grade 3 or worse (CIN3+) based on adjudicated pathology were defined as true disease. Women recruited from both sites were included for calculations of sensitivity. For calculation of specificity, the absence of CIN2+ included adjudicated pathology diagnosis of within normal limits (WNL) or CIN1, and the absence of CIN3+ included WNL, CIN1, or CIN2. Only women recruited from the primary care site were included in specificity, positive predictive value (PPV) and negative predictive value (NPV) calculations to ensure estimates reflected women in a screening setting. 95% confidence intervals (CI) around sensitivity, specificity, PPV and NPV were calculated using the binomial method.
To investigate effects of changing cut-offs on the performance of Xpert, logistic regression models predicting presence or absence of CIN2+ or CIN3+ based on channel-specific reverse cycle threshold (rCt) values were run in HIV-negative and HIV-positive women separately. If the Ct values were considered positive by the assay, we defined the rCt value equal to 45 minus the observed Ct value (45 was chosen as it was the highest Ct value observed in the data). If the Ct values were considered negative by the assay, we defined the rCt value equal to 0. The rCt from each channel was considered a separate variable. The final logistic regression models included rCt values from channels that, when included together in multivariable models, were significantly associated with disease (p<0.05).
From the final logistic regression models, receiver operating characteristic (ROC) curves were generated and the area under the curve (AUC) calculated. Specificity estimates were derived from the ROC curve at sensitivity levels ranging between 85% and 65%. PPV, NPV and the screen-positive rate (SPR) were calculated applying the observed prevalence of CIN2+ or CIN3+ in the screening population and the sensitivity and specificity estimates derived from the ROC curves (See Supplementary Material). Standard errors for these parameters were calculated using the delta method. Statistical analyses were conducted using SAS version 9.4 (Cary, NC).
The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all of the data and the final responsibility to submit for publication
RESULTS
The prevalence of CIN2+ in the screening population was lower in HIV-negative (5.3%; 95% CI: 3.0–7.6) than in HIV-positive women (17.0%; 95% CI 12.9–21.0), as was the prevalence of CIN3+ of 3.2% (95% CI: 1.4–4.9) and 8.5% (95% CI: 5.5–11.5) HIV-negative and positive, respectively. Odds of CIN2+ (odds ratio [OR] = 0.304; 95% CI: 0.177–0.522) and CIN3+ (OR=0.398; 95% CI: 0.197–0.802) were lower in HIV-negative compared to HIV-positive women adjusting for age. Most HIV-positive women (79.9%) were on antiretroviral therapy and the median CD4 counts were 455 cells/mm3 (interquartile range (IQR): 300–630). Prevalence of HPV DNA was also lower in HIV-negative (16.2%; 95% CI: 12.4–19.9) than HIV-positive women (48.4%; 95% CI: 43.4–54.2) after adjusting for age (OR=0.223; 95% CI: 0.156–0.317) (Table 1).
Table 1:
Characteristics of 1121 women recruited at a primary care site (screening population) and at a colposcopy clinic (referral population) in Cape Town, South Africa, by HIV status
| Screening Population recruited at Primary Care Site | Referral Population recruited at Colposcopy Clinic | |||
|---|---|---|---|---|
| HIV-negative (n=382) | HIV-positive (n=333) | HIV-negative (n=204) | HIV-positive (n=202) | |
| Histological confirmed disease: | ||||
| Within Normal Limits | 330 (86.4) | 229 (68.8) | 80 (39.2) | 40 (19.8) |
| CIN11 | 28 (7.3) | 45 (13.5) | 14 (6.9) | 43 (21.3) |
| CIN2 | 8 (2.1) | 28 (8.4) | 23 (11.3) | 38 (18.8) |
| CIN3 | 10 (2.6) | 23 (6.9) | 71 (34.8) | 74 (36.6) |
| Squamous cell carcinoma | 2 (0.5) | 4 (1.2) | 9 (4.4) | 5 (2.5) |
| Adenocarcinoma in situ | 0 | 1 (0.3) | 1 (0.5) | 0 |
| No histology result | 4 (1.0) | 3 (0.9) | 6 (2.9) | 2 (1.0) |
| HPV Xpert result | ||||
| Any of the 5 channels positive2 | 62 (16.2) | 161 (48.4) | 151 (74.0) | 182 (90.6) |
| HPV Xpert result | ||||
| 0 of 3 selected3 channels positive | 331 | 196 | 58 | 244 |
| 1 of 3 selected channels positive | 45 (11.8) | 98 (29.4) | 119 (58.3) | 127 (63.2) |
| 2 or 3 of selected channels positive | 6 (1.6) | 39 (11.7) | 27 (13.2) | 50 (24.9) |
| Median age in years (IQR) | 44 (37 – 52) | 39 (35 – 46) | 40 (34 – 50) | 38 (34 – 44) |
| Median parity (IQR) | 3 (2 – 4) | 2 (1 – 3) | 2 (2 – 3) | 2 (1 – 3) |
| Any prior screening | 246 (64.0) | 249 (74.8) | 204 (100) | 202 (100) |
| Age at first pregnancy | ||||
| Never pregnant | 9 (2.4) | 15 (4.5) | 4 (2.0) | 6 (3.0) |
| ≤16 years | 43 (11.3) | 40 (12.0) | 26 (12.7) | 26 (12.9) |
| >16 years | 330 (86.4) | 278 (83.5) | 174 (85.3) | 170 (84.1) |
| Tobacco use | ||||
| Current | 42 (11.0) | 38 (11.4) | 60 (29.4) | 16 (7.9) |
| Former | 20 (5.2) | 16 (4.8) | 19 (9.3) | 11 (5.5) |
| Never | 320 (83.8) | 279 (83.8) | 125 (61.3) | 175 (86.6) |
| On antiretroviral therapy | ||||
| Yes | N/A | 266 (79.9) | N/A | 178 (88.1) |
| No | 67 (20.1) | 24 (11.9) | ||
Abbreviation: Cervical intraepithelial neoplasia grade 1 (CIN1), grade 2 (CIN2) or grade 3 (CIN3)
Positive for HPV 16, and/or HPV 18, 45, and/or HPV 31, 33, 35, 52, 58, and/or HPV 51, 59, and/or HPV 39, 56, 66 68.
Positive for HPV 16, and/or HPV 18, 45, and/or HPV 31, 33, 35, 52, 58.
One woman in this group had an invalid HPV result was excluded.
Sensitivity of HPV DNA testing to detect CIN2+ using all five channels and the manufacturer’s Ct cut-offs was high in HIV-negative (88.7%; 95% CI: 83.1–94.3) and HIV-positive (93.6%; 95% CI: 83.1–94.3) women. Specificities were 86.9% (95% CI: 83.4–90.4) and 59.9% (95% CI: 54.1–65.7) for HIV-negative and HIV-positive women, respectively. Restricting to the three channels detecting HPV types 16, 18, 45, 31, 33, 35, 52, 58 (manufacturer’s cut-offs), sensitivity was only modestly reduced to 87.1% (95% CI: 81.2–93.0) in HIV-negative and 90.7% (95% CI: 86.4–95.0) in HIV-positive women. Corresponding PPV, NVP and screen-positive rate (SPR) and results for CIN3+ are shown in Table 2.
Table 2:
Sensitivity, specificity, positive and negative predictive values and screen-positive rates (as percentages) in the study populations1
| Sensitivity (95% CI) | Specificity (95% CI) | Positive Predictive Value (95% CI) | Negative Predictive Value (95% CI) | Screen positive Rate (95% CI) | |
|---|---|---|---|---|---|
| CIN2+ | |||||
| HIV-negative | N=124 | N=358 | N=358 | ||
| Any of the 5 high risk channels2 | 88.7 (83.1–94.3) | 86.9 (83.4–90.4) | 23.0 (12.4–33.5) | 98.1 (96.6–99.6) | 16.1 (12.4–19.9) |
| Selected 3 channels3 | 87.1 (81.2–93.0) | 89.7 (86.5–92.8) | 27.5 (15.2–39.7) | 98.2 (96.7–99.6) | 13.5 (10.0–16.9) |
| HIV-positive | N=1724 | N=273 | N=273 | ||
| Any of the 5 high risk channels2 | 93.6 (90.0–97.3) | 59.9 (54.1–65.7) | 31.7 (24.5–38.9) | 97.0 (94.5–99.6) | 48.8 (43.4–54.2) |
| Selected 3 channels3 | 90.7 (86.4–95.0) | 67.5 (62.0–73.1) | 35.0 (27.0–43.0) | 95.9 (93.0–98.7) | 41.5 (36.2–46.8) |
| CIN3+ | |||||
| HIV-negative | N=93 | N=366 | N=366 | ||
| Any of the 5 high risk channels2 | 93.6 (88.6–98.5) | 86.3 (82.8–89.9) | 18.0 (8.4–27.7) | 99.7 (99.1–100) | 16.1 (12.4–19.9) |
| Selected 3 channels3 | 91.4 (85.7–97.1) | 89.1 (85.9–92.3) | 21.6 (10.3–32.9) | 99.7 (99.1–100) | 13.5 (10.0–16.9) |
| HIV-positive | N=106 | N=301 | N=301 | ||
| Any of the 5 high risk channels2 | 95.3 (91.3–99.3) | 55.6 (50.0–61.2) | 16.8 (11.0–22.5) | 99.4 (98.3–100) | 48.8 (43.4–54.2) |
| Selected 3 channels3 | 93.4 (88.7–98.1) | 63.3 (57.8–68.7) | 19.0 (12.4–25.5) | 99.0 (97.5–100) | 41.5 (36.2–46.8) |
By design, sensitivity is calculated combining data from the screening and referral study populations, and specificity, positive and negative predictive value and percent screen-positive from the screening population only.
Positive for HPV 16, and/or HPV 18, 45, and/or HPV 31, 33, 35, 52, 58, and/or HPV 51, 59, and/or HPV 39, 56, 66 68.
Positive for HPV 16, and/or HPV 18, 45, and/or HPV 31, 33, 35, 52, 58.
One woman with an invalid HPV result was excluded from this group.
In multivariable analysis, only rCt values for the first three channels were significantly predictive of CIN2+: adjusted odds ratio (AOR) for HPV16 channel was 1.33 (95% CI: 1.25–1.41), for HPV 18, 45 channel was 1.17 (95% CI: 1.09–1.25) and for HPV 31, 33, 35, 52, 58 channel was 1.27 (95% CI: 1.21–1.33) in HIV-negative women. For HIV-positive women, the AOR for HPV16 channel was 1.19 (95% CI: 1.14–1.25), HPV 18, 45 channel was 1.09 (95% CI: 1.04–1.13) and for HPV 31, 33, 35, 52, 58 channel was 1.17 (95% CI: 1.13–1.21).
For CIN3+ as the endpoint, results were similar for HIV-negative women. The AOR for the HPV16 channel was 1.35 (95% CI: 1.26–1.44), HPV 18, 45 channel 1.17 (95% CI: 1.08–1.27) and HPV 31, 33, 35, 52, 58 channel 1.27 (95% CI: 1.21–1.35). For HIV-positive women, only the HPV16 channel (AOR=1.16 95% CI: 1.11–1.21) and HPV 31, 33, 35, 52, 58 channel (AOR=1.16, 95% CI: 1.12–1.20) were significant predictors of CIN3+.
In HIV-negative women, AUC for CIN3+ as the endpoint was slightly higher (0.928) than for CIN2+ as the endpoint (0.907). For HIV-positive women, AUC was the same regardless of endpoint (0.862) and was slightly lower than in HIV-negative women. Specificity was more variable and lower in HIV-positive compared to HIV-negative women (Figure 1).
Figure 1:
Receiver Operating Characteristic (ROC) Curves generated from multivariable logistic regression models predicting cervical intraepithelial neoplasia grade 2 or worse (CIN2+) and cervical intraepithelial neoplasia grade 3 or worse (CIN3+) from reverse cycle threshold (Ct) values from the three channels detecting HPV 16, HPV 18 and/or 45 and HPV 31, 33, 35, 52, and/or 58 on the Xpert HPV assay stratified by HIV status
Sensitivity was allowed to range between 65–85% and the corresponding specificity from the ROC curves from the multivariable logistic regression are shown in Table 3. E.g. if sensitivity for CIN2+ is allowed to decline to 80%, applying more stringent cut-offs to the three channels detecting HPV types 16, 18, 45, 31, 33, 35, 52, 58 leads to specificity of 94.1% in HIV-negative women. At this cut-off, the PPV is 43.1%, NPV 98.8% and 9.8% of the screening population would be defined as screen-positive. For HIV-positive women, more stringent Ct values allowing sensitivity for CIN2+ to decline to 80% leads to specificity of 83.2%, PPV of 49.4%, NPV of 95.3% and screen-positive rate of 27.5%. Parameters at other sensitivity levels and results for CIN3+ are displayed in Table 3.
Table 3:
Sensitivity, specificity positive and negative predictive values, and screen-positive rates by HIV status utilizing different cycle threshold (Ct) cut-offs from the three channels on HPV Xpert detecting HPV 16, 18, 45, 31, 33, 35, 52, 58
| Sensitivity | Specificity | Positive Predictive Value (95% CI) | Negative Predictive Value (95% CI) | Screen positive (95% CI) | |
|---|---|---|---|---|---|
| CIN2+ | |||||
| HIV-negative | |||||
| 65 | 96.4 | 50.0 (42.3–57.7) | 98.0 (96.5–99.5) | 6.9 (4.5–9.2) | |
| 70 | 96.4 | 51.9 (33.3–59.5) | 98.3 (97.0–99.7) | 7.1 (4.7–9.5) | |
| 75 | 95.3 | 47.2 (40.2–54.2) | 98.6 (97.4–99.8) | 8.4 (5.7–11.1) | |
| 80 | 94.1 | 43.1 (36.7–49.5) | 98.8 (97.6–100) | 9.8 (6.9–12.8) | |
| 85 | 91.3 | 35.4 (30.0–40.8) | 99.1 (98.1–100) | 12.7 (9.3–16.1) | |
| HIV-positive | |||||
| 65 | 88.3 | 53.3 (46.6–60.1) | 92.5 (89.3–95.7) | 20.7 (16.5–25.0) | |
| 70 | 87.2 | 52.9 (46.4–59.4) | 93.4 (90.4–96.4) | 22.5 (18.0–27.0) | |
| 75 | 85.8 | 52.0 (45.8–58.2) | 94.4 (91.5–97.3) | 24.5 (19.8–29.2) | |
| 80 | 83.2 | 49.4 (43.5–55.3) | 95.3 (92.6–98.0) | 27.5 (22.5–32.5) | |
| 85 | 77.0 | 43.1 (37.8–48.4) | 96.2 (93.7–98.7) | 33.5 (28.1–39.0) | |
| CIN3+ | |||||
| HIV-negative | |||||
| 65 | 96.7 | 39.6 (31.8–47.4) | 98.8 (97.7–99.9) | 5.3 (3.1–7.4) | |
| 70 | 95.9 | 36.2 (29.7–43.2) | 99.0 (98.0–100) | 6.2 (3.8–8.5) | |
| 75 | 95.1 | 33.7 (27.3–40.1) | 99.2 (98.3–100) | 7.2 (4.6–9.6) | |
| 80 | 93.4 | 29.0 (23.4–34.6) | 99.4 (98.8–100) | 8.9 (6.1–11.8) | |
| 85 | 92.3 | 28.4 (23.1–33.7) | 99.5 (98.8–100) | 9.6 (7.1–13.2) | |
| HIV-positive | |||||
| 65 | 88.4 | 34.2 (27.7–40.8) | 96.5 (94.3–98.7) | 16.1 (12.2–20.1) | |
| 70 | 87.1 | 33.7 (27.5–40.0) | 97.0 (95.0–99.0) | 17.8 (13.6–21.9) | |
| 75 | 86.1 | 33.5 (27.5–39.5) | 97.4 (95.5–99.3) | 19.1 (14.8–23.4) | |
| 80 | 84.8 | 32.8 (27.1–38.5) | 97.9 (96.2–99.6) | 20.7 (16.2–25.2) | |
| 85 | 74.8 | 24.0 (19.7–28.3) | 98.3 (96.6–99.9) | 30.3 (25.1–35.5) |
Logistic regression models predict disease based on the joint predictive capability of the Ct values on three channels. For women who are positive on more than one channel, the Ct value defining a screen-positive result depends on the Ct values in other channels and would need to be calculated for each woman individually based on the observed Ct values across the three channels. For illustrative purposes, for women positive on only one channel, we back-calculated the Ct threshold required to attain sensitivity between 65–85% for detection of CIN2+. E.g., for HIV-negative women Ct values on the HPV 16 channel need to be <35.9 to be defined as positive for sensitivity of 80% (specificity 94.1%) or <37.8 for sensitivity of 85% (specificity 91.3%) (Figure 2).
Figure 2:
Illustration of cycle threshold (Ct) cut-points to define screen-positive to detect cervical intraepithelial neoplasia grade 2 or worse (CIN2+) at given sensitivity levels assuming that women are positive on only one of the three channels among HIV-negative women (dashed line) and HIV-positive women (solid line).
Discussion
We conducted a large and rigorous clinical study in Cape Town, South Africa, to evaluate a novel approach to adapting an existing approved HPV test to make it more suitable for screen-and-treat. Both type-restriction and changing cut-offs to define a positive result led to substantial improvements in specificity, PPV and screen-positive rates while producing only minor reductions in sensitivity. Benefits were particularly note-worthy for HIV-positive women who have high rates of HPV infection.
HPV assays have been designed to detect the 14 high-risk HPV genotypes associated with cervical cancer and its precursor lesions. During assay development, cut-off values are identified that allow tests to achieve sensitivity and specificity at pre-defined benchmarks.(24–26) However, performance characteristics appropriate for an HPV assay used for primary screening in the US and Europe are different to those desired for a test utilized for screen-and-treat in LMIC. With the screen-and-treat approach, some clinicians believe that HPV results should serve as the basis for determining whether women get treatment whereas others prefer that an additional triage step such a VIA be performed prior to treatment. In large part this controversy is driven by the relatively low specificity of currently available HPV assays.
We found that restricting Xpert HPV to 8 HPV genotypes (HPV 16, 18, 45, 31, 33, 35, 52, 58) resulted in a negligible impact on sensitivity, but led to improvements in specificity, in both HIV-negative and HIV-positive women. For HIV-negative women, the improvement in specificity from 86.9% to 89.7% obtained by type restriction alone makes Xpert HPV suitable for screen-and-treat without any further modifications. However, for HIV-positive women, although type-restriction improved performance characteristics, further improvements would be desirable.
Our study demonstrated that making more stringent cut-offs on the three channels on Xpert HPV that detect HPV 16, 18, 45, 31, 33, 35, 52, 58 was an effective way to achieve an optimal balance between sensitivity and specificity. Since Xpert HPV is a PCR assay, we can use the Ct value as a rough approximation of the quantity of HPV DNA present or “viral load.” For HIV-negative women, if we allow sensitivity for CIN2+ to decline to 80%, specificity is 94.1%. In our study population, this means that 9.8% of women would screen positive with a PPV for CIN2+ of 43.1%. These are highly attractive parameters for an effective screen-and-treat program.
It is for HIV-positive women that the benefits of shifting to more stringent cut-offs are most notable. For HIV-positive women, we observed that if we allow sensitivity for CIN2+ to decline to 80%, specificity improves to 83.2%. At this sensitivity/specificity balance, 27.5% of women would now be screen-positive with a PPV of 49.4%. For the test as originally configured, 48.8% of HIV-positive women would be screen-positive and the PPV would be 31.7%. This is a considerable improvement and makes screen-and-treat a good choice for this population.
An advantage of Xpert HPV is that the Ct values of each channel are part of standard output. Thus, it is not necessary to undertake any new engineering of the test to change the cut-offs. A calculator at the output level of the test can be utilized to classify women. This calculator would be programmed with the logistic regression parameters and would input the Ct values from an individual and then output the classification as screen-positive or screen-negative. The features of this assay allow decision-making about performance characteristics of the screening test to be shifted to program implementers. To our knowledge, at the current time, Xpert HPV is the only PCR-based and field-friendly assay that has been clinically validated. More cheap and validated assays with point-of-care applicability would be welcome. Those with a quantifiable signal output and separation of HPV types would have the additional advantage of flexibility that could be utilized in the way we propose here.
The lack of availability until now of a robust, point-of-care HPV assay is the major reason why several screen-and-treat programs have had to rely on visual inspection with acetic acid (VIA) as the primary screening test. VIA is appealing for many reasons, not the least of which is its low cost. Unfortunately, performance characteristics of VIA as a screening test are poor. Clinical trials in South Africa and India have demonstrated substantially better outcomes with HPV-based screen-and-treat.(7–9) Although costs of HPV assays are still relatively high, rigorous cost-effectiveness analyses have confirmed the affordability of HPV-based screen-and-treat given the savings accrued by simplification of the screening cascade and reduced number of visits.(13, 27) The impetus and infrastructure built to support VIA-based screen-and-treat is ideally positioned to now incorporate HPV testing.
In selecting optimal cut-offs on the assay, program implementers could weigh resource availability and community preferences. For example, if there are insufficient clinicians to treat the 27.5% of HIV-positive women who would require treatment if sensitivity is set at 80%, cut-offs could be shifted to a lower sensitivity resulting in a smaller percentage of women in need of treatment. Some may question the wisdom of reducing sensitivity to 80%, or even lower, because of resource limitations. Current guidance generally advises that a candidate HPV test should have a sensitivity for CIN2+ of not less than 90% of the sensitivity of Hybrid Capture 2 (HC2) in women 30 years and older.(28) At the time these recommendations were made the pooled sensitivity of HC2 was 97.9%.(28) More recent pooled analyses have slightly lower estimates.(14) However, it is important to recognize that cytology has a much lower sensitivity than HPV testing. E.g., the U.S.-based ATHENA trial found the sensitivity of liquid-based cytology for CIN3+ to be only 53·3%.(29) Despite its much lower sensitivity, cytology is still considered an acceptable screening test in several countries.(6) In addition, since our approach excludes those with lower HPV viral loads, the compromises in sensitivity may be less than reported here. Prospective studies are needed to evaluate this. Preferences of the clinical community could also be taken into account. E.g., for settings where over-treatment is less tolerated, cut-offs that ensure high PPV could be chosen. In contrast, where there are few alternative treatment options and women are likely to be infrequently screened, higher sensitivity could be selected and more over-treatment tolerated.
Our study has several limitations. For reasons of efficiency, we recruited women from both a screening population and a referral population to attain sufficient numbers with the desired clinical endpoints. As such, this case mix may not be representative of disease detected in a screening population. Recruiting only from a screening population is prohibitively expensive. Our approach of enriching the study population from a referral population in order to attain the number of cases needed for sensitivity calculations has also been applied in the VALGENT HPV test validation protocol.(26) In addition, our endpoint definition was rigorous with only confirmed cases of CIN2+ or CIN3+ on histology included. Our study population was recruited in a single geographic area and generalizability to other geographic regions is needed.
Screen-and-treat has many advantages in our setting, where the majority of women has limited access to health care, most health care services are curative rather than preventative and human and financial resources are limited. A major advantage is improved coverage of the target population and reduction in attrition. Other advantages include better test performance of molecular tests compared to cytology, the ease of applying the test and ease of access for women living in impoverished circumstances as the activity takes place in clinics in the community. The lack of the need for laboratory-based services, coordination between clinics and laboratories, or reliable transport between them are further advantages that make the HPV-based screen-and-treat particularly advantageous for LMIC. Our approach allows screen-and-treat programs to adjust to resource constraints, disease prevalence rates, and clinical values and preferences. Implementation studies are urgently needed.
Supplementary Material
Acknowledgements
The study was supported in part by the National Cancer Institute UH2/3 CA189908. We would like to thank Drs. Paul Pearlman, Rao Divi, Tony Dickherber, and John Jessup of NCI, Dr. Gwynn Stevens, Ms. Dipti Lallubhai and Ms. Tessa Visser of Cepheid–South Africa, Ms. Jill Birkmeier and the Cepheid Clinical Affairs and Data Management and Analytics team for study record capture, database construction, and cleaning, and the facility managers and senior leadership of the Department of Health, Western Cape Provincial Government for support with the study.
Funding:
Supported by the National Cancer Institute UH2/3 CA189908.
Footnotes
Conflicts
Kuhn, Denny
Report grants from NIH
Saidu, Boa, Tergas, Moodley, Tsai
Have no conflicts to declare.
Persing, Campbell
Declare that they receive salary, benefits and equity from Danaher Corp. Cepheid is a wholly owned entity of Danaher Corporation.
Wright
Declares that he serves as a consultant in clinical trial design and as an expert pathologist for HPV vaccine and/or diagnostic trials for Becton, Dickinson and Company, BD Life Sciences – Diagnostic Systems, Roche, and Inovio Pharmaceuticals and as a speaker for Roche and Becton, Dickinson and Company, BD Life Sciences – Diagnostic Systems.
References:
- 1.Denny L, de Sanjose S, Mutebi M, Anderson BO, Kim J, Jeronimo J, et al. Interventions to close the divide for women with breast and cervical cancer between low-income and middle-income countries and high-income countries. Lancet (London, England). 2017;389(10071):861–70. [DOI] [PubMed] [Google Scholar]
- 2.Denny L, Quinn M, Sankaranarayanan R. Chapter 8: Screening for cervical cancer in developing countries. Vaccine. 2006;24 Suppl 3:S3/71–7. [DOI] [PubMed] [Google Scholar]
- 3.Goldhaber-Fiebert JD, Denny LE, De Souza M, Wright TC Jr., Kuhn L, Goldie SJ. The costs of reducing loss to follow-up in South African cervical cancer screening. Cost effectiveness and resource allocation : C/E. 2005;3:11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Denny L, De Sousa M, Kuhn L, Pollack A, Wright TC Jr. Cervical cancer prevention - a paradigm shift? Gynecologic oncology. 2005;99(3 Suppl 1):S12. [DOI] [PubMed] [Google Scholar]
- 5.Laara E, Day NE, Hakama M. Trends in mortality from cervical cancer in the Nordic countries: association with organised screening programmes. Lancet (London, England). 1987;1(8544):1247–9. [DOI] [PubMed] [Google Scholar]
- 6.WHO Comprehensive Cervical Cancer Control: A guide to essential practice. 2014;2nd Edition. [PubMed] [Google Scholar]
- 7.Denny L, Kuhn L, De Souza M, Pollack AE, Dupree W, Wright TC Jr. Screen-and-treat approaches for cervical cancer prevention in low-resource settings: a randomized controlled trial. JAMA. 2005;294(17):2173–81. [DOI] [PubMed] [Google Scholar]
- 8.Denny L, Kuhn L, Hu CC, Tsai WY, Wright TC Jr. Human papillomavirus-based cervical cancer prevention: long-term results of a randomized screening trial. J Natl Cancer Inst. 2010;102(20):1557–67. [DOI] [PubMed] [Google Scholar]
- 9.Sankaranarayanan R, Nene BM, Shastri SS, Jayant K, Muwonge R, Budukh AM, et al. HPV screening for cervical cancer in rural India. The New England journal of medicine. 2009;360(14):1385–94. [DOI] [PubMed] [Google Scholar]
- 10.Paul P, Winkler JL, Bartolini RM, Penny ME, Huong TT, Nga le T, et al. Screen-and-treat approach to cervical cancer prevention using visual inspection with acetic acid and cryotherapy: experiences, perceptions, and beliefs from demonstration projects in Peru, Uganda, and Vietnam. The oncologist. 2013;18 Suppl:6–12. [DOI] [PubMed] [Google Scholar]
- 11.Parham GP, Mwanahamuntu MH, Kapambwe S, Muwonge R, Bateman AC, Blevins M, et al. Population-level scale-up of cervical cancer prevention services in a low-resource setting: development, implementation, and evaluation of the cervical cancer prevention program in Zambia. PloS one. 2015;10(4):e0122169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Campbell C, Kafwafwa S, Brown H, Walker G, Madetsa B, Deeny M, et al. Use of thermo-coagulation as an alternative treatment modality in a ‘screen-and-treat’ programme of cervical screening in rural Malawi. International journal of cancer. 2016;139(4):908–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Goldie SJ, Kuhn L, Denny L, Pollack A, Wright TC. Policy analysis of cervical cancer screening strategies in low-resource settings: clinical benefits and cost-effectiveness. JAMA. 2001;285(24):3107–15. [DOI] [PubMed] [Google Scholar]
- 14.Arbyn M, Ronco G, Anttila A, Meijer CJ, Poljak M, Ogilvie G, et al. Evidence regarding human papillomavirus testing in secondary prevention of cervical cancer. Vaccine. 2012;30 Suppl 5:F88–99. [DOI] [PubMed] [Google Scholar]
- 15.Ronco G, Dillner J, Elfstrom KM, Tunesi S, Snijders PJ, Arbyn M, et al. Efficacy of HPV-based screening for prevention of invasive cervical cancer: follow-up of four European randomised controlled trials. Lancet (London, England). 2014;383(9916):524–32. [DOI] [PubMed] [Google Scholar]
- 16.Heard I Prevention of cervical cancer in women with HIV. Current opinion in HIV and AIDS. 2009;4(1):68–73. [DOI] [PubMed] [Google Scholar]
- 17.Mapanga W, Girdler-Brown B, Feresu SA, Chipato T, Singh E. Prevention of cervical cancer in HIV-seropositive women from developing countries through cervical cancer screening: a systematic review. Systematic reviews. 2018;7(1):198. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Akbari A, Vanden Broeck D, Benoy I, Padalko E, Bogers J, Arbyn M. Validation of intra- and inter-laboratory reproducibility of the Xpert HPV assay according to the international guidelines for cervical cancer screening. Virology journal. 2018;15(1):166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Cubie HA, Morton D, Kawonga E, Mautanga M, Mwenitete I, Teakle N, et al. HPV prevalence in women attending cervical screening in rural Malawi using the cartridge-based Xpert((R)) HPV assay. Journal of clinical virology : the official publication of the Pan American Society for Clinical Virology. 2017;87:1–4. [DOI] [PubMed] [Google Scholar]
- 20.Mbulawa ZZA, Wilkin TJ, Goeieman B, Swarts A, Williams S, Levin S, et al. Xpert human papillomavirus test is a promising cervical cancer screening test for HIV-seropositive women. Papillomavirus research (Amsterdam, Netherlands). 2016;2:56–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Toliman PJ, Kaldor JM, Badman SG, Gabuzzi J, Silim S, Kumbia A, et al. Performance of clinical screening algorithms comprising point-of-care HPV-DNA testing using self-collected vaginal specimens, and visual inspection of the cervix with acetic acid, for the detection of underlying high-grade squamous intraepithelial lesions in Papua New Guinea. Papillomavirus research (Amsterdam, Netherlands). 2018;6:70–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Rabaan AA, Alfaraj SA, Alkhalifah MA. Comparison of the Cepheid Xpert HPV test and the HC2 High-Risk HPV DNA Test for detection of high-risk HPV infection in cervical smear samples in SurePath preservative fluid. Journal of medical microbiology. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Einstein MH, Smith KM, Davis TE, Schmeler KM, Ferris DG, Savage AH, et al. Clinical evaluation of the cartridge-based GeneXpert human papillomavirus assay in women referred for colposcopy. Journal of clinical microbiology. 2014;52(6):2089–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Meijer CJ, Berkhof H, Heideman DA, Hesselink AT, Snijders PJ. Validation of high-risk HPV tests for primary cervical screening. Journal of clinical virology : the official publication of the Pan American Society for Clinical Virology. 2009;46 Suppl 3:S1–4. [DOI] [PubMed] [Google Scholar]
- 25.Arbyn M, Snijders PJ, Meijer CJ, Berkhof J, Cuschieri K, Kocjan BJ, et al. Which high-risk HPV assays fulfil criteria for use in primary cervical cancer screening? Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases. 2015;21(9):817–26. [DOI] [PubMed] [Google Scholar]
- 26.Arbyn M, Depuydt C, Benoy I, Bogers J, Cuschieri K, Schmitt M, et al. VALGENT: A protocol for clinical validation of human papillomavirus assays. Journal of clinical virology : the official publication of the Pan American Society for Clinical Virology. 2016;76 Suppl 1:S14–s21. [DOI] [PubMed] [Google Scholar]
- 27.Campos NG, Lince-Deroche N, Chibwesha CJ, Firnhaber C, Smith JS, Michelow P, et al. Cost-Effectiveness of Cervical Cancer Screening in Women Living With HIV in South Africa: A Mathematical Modeling Study. Journal of acquired immune deficiency syndromes (1999). 2018;79(2):195–205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Meijer CJ, Berkhof J, Castle PE, Hesselink AT, Franco EL, Ronco G, et al. Guidelines for human papillomavirus DNA test requirements for primary cervical cancer screening in women 30 years and older. International journal of cancer. 2009;124(3):516–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Castle PE, Stoler MH, Wright TC Jr., Sharma A, Wright TL, Behrens CM. Performance of carcinogenic human papillomavirus (HPV) testing and HPV16 or HPV18 genotyping for cervical cancer screening of women aged 25 years and older: a subanalysis of the ATHENA study. The Lancet Oncology. 2011;12(9):880–90. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.