Abstract
Background:
Of women with cervical cancer (CC) and HIV, 85% live in sub-Saharan Africa (SSA), where 21% of all CC cases are attributable to HIV infection. We aimed to generate internationally acceptable facility-based indicators to monitor and guide scale-up of CC prevention and care services offered on- or off-site by HIV clinics.
Methods:
We reviewed the literature and extracted relevant indicators, grouping them into domains along the CC control continuum. From February 2021 to March 2022, we conducted a three-round, online Delphi process to reach consensus on indicators. We invited 106 experts to participate. Through an anonymous, iterative process, participants adapted the indicators to their context (Round 1), then rated them for five criteria on a 5-point Likert-type scale (Rounds 2 and 3), then ranked their importance (Round 3).
Results:
We reviewed 39 policies from 21 African countries and seven from international organizations; 72 experts from 15 SSA countries or international organizations participated in our Delphi process. Response rates were 34% in Round 1, 40% in Round 2, and 44% in Round 3. Experts reached consensus for 17 indicators in the following domains: primary prevention (HPV prevention, n=2); secondary prevention (screening, triage, treatment of precancerous lesions, n=11); tertiary prevention (cervical cancer diagnosis and care, n=2); long-term impact of the program and linkage to HIV service (n=2).
Conclusion:
We recommend that HIV clinics that offer CC control services in SSA implement the 17 indicators stepwise and adapt them to context to improve monitoring along the CC control cascade.
Keywords: Women living with HIV, Acquired Immunodeficiency Syndrome, Early Detection of Cancer, Cervical Cancer, Consensus, Sub-Saharan Africa
Introduction
Cervical cancer (CC) is the most common cancer among women living with HIV, who are at high risk of persistent Human Papillomavirus (HPV) infection and six times more likely to develop CC than the general population.1,2 HIV infection contributes to 21% of all CC diagnoses among women in Africa, accounting for 85% of the global tally of women diagnosed with CC attributed to HIV.1,3 To achieve the World Health Organization (WHO)’s goal of eliminating CC, countries in sub-Saharan Africa (SSA) must scale-up access to primary, secondary, and tertiary prevention measures, especially for girls and women living with HIV.4–6 To improve CC control programs, clinicians, researchers, and policymakers need high-quality routine health facility data,7,8 which can be collected by monitoring each step of the path that people take through the health system. To create a monitoring plan for cancer control, each sequential step through a complex health system must be quantified within the framework of a cascade9 and indicators must be specified for each step.10,11 Cascades are widely used conceptual models that support monitoring, assess engagement and identify gaps in services.9,12,13 Several studies have taken this approach to evaluating the performance of CC control programs for women living with HIV in SSA,14–20 but they did not use standardized indicators, so it is difficult to compare their findings.14–20 Indicators that consider HIV status are often omitted from cancer control policies, even in countries with high HIV burden,21 where they are most necessary.22 Most cancer control polices in these countries advice leveraging existing infrastructure and integrating CC prevention and care services into existing HIV programs to facilitate access to and scale-up of these services and eventually significantly reduce CC incidence and mortality.23–27 But today, data on access to and uptake of services for women attending HIV clinics in SSA are limited or rare, even though electronic data systems are widely available.21,22,28
We urgently need standardized indicators for each step in the CC prevention and care cascade to measure and compare access to and quality of the services offered to girls and women living with HIV, so we used a Delphi process to bring experts to consent on facility-based indicators for monitoring, managing and scaling up the CC prevention and care cascade through which girls and women attending HIV clinics in SSA progress.
Methods
Study settings
We collaborated with the International epidemiology Databases to Evaluate AIDS consortium (IeDEA, https://www.iedea.org/), a network that collects and analyzes data from routine care of more than 2.2 million people living with HIV globally. In SSA, IeDEA is present in 22 countries across four regions (Central, East, Southern, and West Africa) and comprises 240 HIV treatment and care sites in both urban and rural areas, operating mostly at the primary or secondary care level.29 The study received an ethics waiver from the Cantonal Ethics Committee of Bern (BASEC-Nr: Req-2020-00748).
Literature review
Three researchers (MD, KT, SLA) reviewed the literature to identify relevant indicators for monitoring CC control programs. We first reviewed the recent WHO toolkit, Improving Data for Decision Making in Global Cervical Cancer Programmes (IDCCP), which describes indicators and best monitoring practices,30 and the International Cancer Control Partnership database.31 Next, we included the most recent national cancer control policies, strategic plans, and where available, national plans for controlling non-communicable diseases in SSA countries. We explored national health ministry websites and online web tools and contacted experts in the field to identify relevant unpublished literature. We included documents published between 2010 and 2020 in English and French. Two researchers (MD, SLA) independently extracted relevant indicators and the definitions of numerators and denominators when they were available. These researchers compared results, deduplicated, and grouped similar indicators. When they disagreed, they consulted a third investigator (KT) to arrive at consensus. From our list of extracted indicators, we deliberately pre-selected those that could be quantified with data collected at HIV clinics during routine care. We did not limit the number of indicators, but we excluded indicators that would require facilities to conduct surveys or patients to fill out satisfaction questionnaires, e.g., qualitative indicators that measure CC awareness or quality of care, patient experience and satisfaction.
The Expert Panel (EP)
Based on pre-defined selection criteria (File S1), we recruited experts in CC or HIV/AIDS prevention and care in SSA through the IeDEA network. We also invited participants of the 2019 workshop “CC Prevention and Care Cascade in Women Living with HIV in SSA,” hosted by the 3rd IeDEA All Africa meeting. Expert Panel (EP) members were asked to volunteer their participation in the Delphi process and to attend our online meetings. We aimed for equal geographic and gender distribution of EP members.
Delphi process
We conducted a three-round online Delphi process (Figure S1), following recommendations from guidelines and reviews.32–36 The Delphi process is a structured method for gathering and distilling the collective knowledge and opinions of a group of topic experts. We developed and piloted a Delphi questionnaire in English and French, which included (1) informed consent, (2) study description and instructions, and (3) general and demographic questions. The questionnaire also included (4) the indicators we had identified, pre-selected, and then adapted or revised with the EP members during the process, along with any remaining open questions. Rating and ranking instructions (5) were also provided. We emailed EP members and asked them to use the QualtricsXM survey platform to participate anonymously in the online Delphi process.
The first Delphi round questionnaire included a preliminary list of the 30 pre-selected indicators in tabular format,37 listing title and definition, purpose and rationale, measurement method , data collection methodology and frequency, data disaggregation, guidelines for interpreting and using data, and relevant additional information. The questionnaire included multiple choice questions about additional items for indicators, e.g., definition of the population, appropriate levels of disaggregation, age ranges, and time periods. We used the responses to modify indicators in subsequent rounds, based on majority rule. Then, we grouped the indicators into the six domains that match the steps of the CC control continuum (Figure 1). In the second Delphi round, we presented these revised indicators to our experts, along with summaries of the first-round comments. We asked EP members if they agreed with the updates or thought they needed further discussion. We also told them that, once they reached consensus on indicators (high or very high rating by at least 70% of respondents, see below for details), we would implement the variables needed to calculate those indicators into the IeDEA Data Exchange Standard (IeDEA DES). Experts were told to rate the revised indicators on a 5-point Likert-type scale (1–very low, 5–very high) for five rating criteria: relevance; feasibility; comparability; reliability; and understandability (File S2, Table S3. Rating Criteria). We drew our selection of the type of Likert scale, and our definitions and the number of rating criteria from the literature and made final decisions within our team through the voting process. Between the second and third Delphi rounds, we organized four satellite sessions and an online stakeholder meeting. At the satellite sessions, we discussed definitions of indicators and data elements, key populations, age ranges, time periods, rating results, comments we had selected from previous rounds, and domains. The EP members shared and discussed their concerns and ideas, and proposed solutions. At the final stakeholder meeting, we presented and discussed successful regional models of CC management and data collection and future activities. Professional moderators guided all sessions, and we employed interpreters to ensure that language was not a barrier to joining the discussions. In the third Delphi round, we shared a summary of comments from previous rounds and minutes of our meetings (File S3). We asked the EP to re-rate indicators based on our five rating criteria. We presented again 30 indicators, even though some did not reached consensus in the second round, because we discussed and adjusted indicators based on feedback we received during satellite sessions. The EP then ranked the importance of each indicators, stratified by the six domains. Throughout the process, participants could comment in open-ended question fields. Two researchers (MD, AZ) could access the database containing the responses; feedback could not be linked back to individuals. In each Delphi round, we sent weekly reminders to participants who had not yet submitted their answers.
Figure 1.
The Cervical Cancer Control Continuum at Facility Level: the overview of domains, core, optional and 1st ranked indicators per each domain that reached consensus in Round 3. Consensus is reached if the indicator had a high level of agreement (more than 70% of respondents rated an indicator as 4 and 5 points on Likert scale) in three or more criteria. Within each domain, the core and optional indicators are ordered based on their rating results, with the highest-rated indicator placed at the top. Core indicators are indicators that reached a high level of agreement in all five criteria, and optional indicators are those with a high level of agreement in three or four criteria. The indicator ranked as the most important in each domain is presented as 1st ranked indicator
Data analysis
We used descriptive statistics to report characteristics of EP members and participation, response and completion rates; these equations are detailed in Table S1. Rating and ranking results are presented by level of agreement and consensus, ranking score, and total rank; descriptions of rating and ranking calculations are provided in File S4. We defined consensus as the median score above our predefined threshold and a high level of agreement (File S2, Definition of consensus),38,39 defined as an indicator rated 4 (high) or 5 (very high) points on the Likert scale for at least three of five criteria (relevance, feasibility, comparability, reliability and understandability) by 70% of respondents. We provided an illustrated overview and comprehensive tables for indicators that reached consensus in Round 3, basing our presentation on international recommendations. Tables include title, definition, calculation, purpose and rationale, data source, frequency, disaggregation, and guidelines. We used thematic analysis to interpret qualitative data from open-ended questions (File S5).40,41
Results
Literature review
We identified and reviewed 46 documents (39 in English, seven in French): 39 policies from 21 African countries and seven from international organizations; and two web tools for cancer-related data analysis (https://canscreen5.iarc.fr/ and https://nordscreen.org/) (Table S2). In total, we extracted and reviewed 509 indicators; of these 52 were extracted from the WHO IDCCP Toolkit.30 Two researchers deduplicated and then grouped the extracted indicators based on similarity. We then proposed 30 indicators to the EP.
Characteristics of Expert Panel (EP) members
We emailed 106 experts (85 in Round 1, 84 in Round 2, and 101 in Round 3) and invited them to participate. In the second round, one participant opted out. In the third round, we invited additional experts who had expressed interest in joining at the stakeholder meeting. In total, 72 individuals participated in at least one round (46 in Round 1, 40 in Round 2, and 55 in Round 3). Fifteen African countries were represented in the EP (Figure 2) and it was gender balanced (52% women). Most members were researchers (56%) and clinicians (31%); 68% were affiliated with the IeDEA consortium, and about half (48%) worked in Southern Africa (Table S3). Most participants self-reported they had either less than 5 years (31%) of experience or 10-20 years (34%) of experience in CC prevention and care and 10-20 years (39%) in HIV/AIDS care and treatment. A third of participants reported additional experience in other areas of research or health care (Table S4).
Figure 2.
Representative countries in the Expert Panel in all three Delphi rounds (total participants, n=65).
Delphi rounds
The response rate (number of participants who completed the survey / number of emailed participants) was 34% in Round 1, 40% in Round 2, and 44% in Round 3 (See Table S3 for completion rates and participation rates). The definitions of key population were guided by WHO recommendations on CC screening and treatment for women living with HIV,42 informed by participants’ answers in the first and second round, and discussed and agreed upon during satellite sessions: “Women living with HIV/AIDS who are enrolled in care and had at least one HIV clinic visit during the period of interest” and who are “25-49 years” of age; and “Girls living with HIV enrolled in care with at least one HIV clinic visit during the period of interest” and who are “9-14 years” of age. Where applicable, we incorporated these definitions for all indicators in the final rating and ranking session.
In the second and third round, EP members rated the 30 proposed indicators; consensus (at least 70% agreement in 3 or more criteria) was reached on 13 indicators in Round 2 (Figure S2) and 17 indicators in Round 3 (Figure 3). The 17 indicators that reached consensus in Round 3 covered all domains of the CC prevention and care continuum: primary prevention (HPV prevention, n=2); secondary prevention (screening, n=8; triage, n=6; treatment of precancerous lesions, n=4); tertiary prevention (CC diagnosis and care), n=5; and long-term impact of the program and linkage to HIV services, n=5. These are comprehensively described in File S6. In the domain primary prevention (HPV prevention), both of the proposed indicators reached consensus. In the secondary prevention domain, six of eight screening indicators reached consensus; half of triage (3/6) indicators and treatment of precancerous lesions indicators (2/4) reached consensus. In the domains tertiary prevention (CC diagnosis and care) and long-term program impact and linkage to HIV services, two of five proposed indicators reached consensus.
Figure 3.
List of indicators that reached consensus in Round 3. Consensus was reached if more than 70% of participants rated the indicator as 4 (High) or 5 (Very high) points on the Likert scale in 3 or more criteria
Five indicators obtained a high level of agreement (>70% of participants) in all five criteria and we labeled these as core indicators. We labeled the other 12 indicators as optional. Of the five core indicators, four belonged to the domain secondary prevention (screening): Cervical Screening Rate; Number of Women Screened for Cervical Pre-cancer; Screening Test Positivity Rate; and Screening Test Positivity Rate for First Time Screened Women. One belonged to the domain secondary prevention (treatment of precancerous lesions): Treatment Rate of Precancerous Lesions (Figure 3). The same indicators, except Screening Test Positivity Rate for First Time Screened Women, reached consensus for all five criteria in Round 2. Cervical Cancer Incidence Rate reached consensus for all five criteria in Round 2, but not Round 3. More than 70% of EP members rated the relevance of 16 indicators in Round 2 and 17 indicators in Round 3 as 4 (high) or 5 (very high). In Round 3, all indicators that reached consensus had been rated 4 or 5 for comparability and understandability. In Round 2, only 13 indicators were rated 4 or 5 for comparability, and 14 indicators were rated 4 or 5 for understandability (Figure 3). Ratings on feasibility and reliability were lower; only six indicators in Rounds 2 and 3 were rated 4 or 5 for feasibility and reliability. Between Rounds 2 and 3, the greatest change in the level of agreement was for Triage Examination Positivity Rate: feasibility increased by 27% (from 35% to 62%) and understandability by 29% (from 62% to 91%). Of the 13 indicators that failed to reach consensus in Round 3, ten were rated 4 or 5 for relevance by more than 70% of participants; none was rated 4 or 5 for feasibility, comparability, or reliability (Figure S2).
Our analysis of the qualitative data we collected in all three rounds revealed that the topic of most concern was improving the definitions of indicators (e.g., age ranges). Several participants thought it could be difficult to collect the data that informed the indicators during routine care and to disaggregate that information, especially in resource-limited settings and settings where cervical screening services are offered off-site. We integrated these concerns in Round 2, when we drafted the agenda for the satellite meetings. For example at the satellite sessions, we discussed the recent update to WHO screening and treatment guidelines for CC, in which WHO newly recommended that women living with HIV should take an HPV DNA primary test and then a triage test if they were found to be HPV positive.42 Members presented their ideas and suggestions for overcoming challenges to implementing these guidelines, e.g. the feasibility of collecting the data (File S3).
Table 1 and Figure 1 present the 17 indicators that reached consensus in Round 3, ranked by importance and stratified by domain. The highest ranked indicators in each domain were HPV Vaccination Rate in primary prevention, Number of Women Screened for Cervical Pre-cancer in secondary prevention (screening), Received Triage Examination Rate in secondary prevention (triage), Treatment Rate of Precancerous Lesions in secondary prevention (treatment of precancerous lesions), Suspected Cervical Cancer Cases Rate in tertiary prevention (CC diagnosis and care), and Cervical Cancer Incidence Rate in long-term program impact and linkage to HIV service.
Table 1.
Ranking of indicators that reached consensus per domains in Round 3 by importance
| Rank§ (score) | Indicator’s title and definition |
|---|---|
| Domain: Primary Prevention – HPV prevention | |
|
| |
| 1 (85) |
HPV Vaccination Rate HPV vaccinated “girls living with HIV enrolled in care with at least one HIV clinic visit during the period of interest” aged 9-14 years |
| 2 (50) |
High Risk HPV Incidence Rate Newly diagnosed high-risk HPV cases among “girls and women living with HIV/AIDS enrolled in care with at least one HIV clinic visit during the period of interest” in a specific age range in a 12-month period |
|
| |
| Domain: Secondary Prevention – Screening | |
|
| |
| 1 (312) |
Number of Women Screened for Cervical Pre-cancer† Number of screened “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” |
| 2 (304) |
Cervical Screening Rate‡ Screened “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” |
| 3 (237) |
Screening Test Positivity Rate for the Primary Screening Test Screened “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” who received a positive primary screening test result in a 6-month period |
| 4 (156) |
Received Screening Test Results “Women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” who received their screening test results in a 6-month period |
| 5 (113) |
Screening Test Positivity Rate for the Primary Screening Test for First Time Screened Women The first time screened “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” who received a positive primary screening test result in a 12-month period |
| 6 (75) |
Rescreened after a previous Negative Result, within Recommended Screening Interval “Women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” who were rescreened (after a previous negative result) within the recommended screening interval |
|
| |
| Domain: Secondary Prevention – Triage | |
|
| |
| 1 (215) |
Received Triage Examination Rate Screen-positive “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” who received a triage examination in a 12-month period |
| 2 (185) |
Triage Examination Positivity Rate Screen-positive “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” with a positive triage examination result in in a 12-month period |
| 3 (116) |
Triage Examination Provision Rate Screen-positive “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” who attended the triage visit and received a triage examination in a 12-month period |
|
| |
| Domain: Secondary Prevention – Treatment of precancerous lesions | |
|
| |
| 1 (176) |
Treatment Rate of Precancerous Lesions Screen-positive “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” who have received treatment in a 6-month period |
| 2 (111) |
Precancerous Lesions Post-Treatment Follow-Up Rate “Women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” treated for precancerous lesions who return for a post-treatment follow-up screening test in a 12-month period |
|
| |
| Domain: Tertiary Prevention – CC diagnosis and care | |
|
| |
| 1 (197) |
Suspected Cervical Cancer Cases Rate Screened “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” with suspected cervical cancer in a 12-month period |
| 2 (86) |
Confirmed Cervical Cancers Rate Screen-positive “women living with HIV/AIDS 25-49 years old enrolled in care with at least one HIV clinic visit during the period of interest” diagnosed with invasive cervical cancer in a 12-month period |
|
| |
| Domain: Long-term program Impact and Linkage of HIV Services | |
|
| |
| 1 (200) |
Age-Specific Cervical Cancer Incidence Rate New invasive cervical cancer cases diagnosed in “women living with HIV/AIDS enrolled in care with at least one HIV clinic visit during the period of interest” in a specific age group or range in a 12-month period |
| 2 (112) |
HIV Testing and Counseling Service Provision Rate Women with previously unknown HIV status who received testing and counseling service for HIV at their cervical screening visit, and now know their HIV status in a 12-month period |
Rank position and ranking score (RS) per each domain. To determine the RS, we first calculated frequency (how many respondents placed an indicator as 1st, 2nd, 3rd etc., within each domain). We multiplied frequency by the weight of the ranked position: first place was highest and last place lowest: RS = 1W1+x2W2+x3W3+x4W4… where x is the frequency (response count) for the indicator choice, and W is the weight of the ranked position). Then we ordered RS from highest to lowest and assigned the ranks: 1 for the first highest RS within domain, 2 for the second highest RS etc.; The supplementary information, File S4 – Quantitative analysis (rating and ranking) provides step by step instructions how ranking was performed.
this is an absolute number;
this is a proportion.
Discussion
We worked with international experts to come to consensus on facility-based indicators for managing and scaling up CC prevention and care services offered to girls and women living with HIV, who receive care at HIV clinics across SSA. The group reached consensus (at least 70% agreement in 3 or more criteria) on 17 indicators in the domains of primary prevention (HPV prevention, n=2), secondary prevention (screening, triage, treatment of precancerous lesions, n=11), tertiary prevention (cervical cancer diagnosis and care, n=2), long-term impact of the program and its linkage to HIV services (n=2). Five indicators from the domains secondary prevention (screening and treatment of precancerous lesions) garnered at least 70% agreement for all criteria (relevance, feasibility, comparability, reliability and understandability) the experts used to rate them.
We took a comprehensive methodological approach that comprised a rigorous EP selection process and iterative online Delphi rounds in which discussions were guided and participants presented structured feedback. Questionnaires contained detailed instructions in two languages. We assembled an EP of participants from a variety of professional backgrounds and levels of experience, to increase the likelihood our results would be generalizable and applicable across contexts. We were limited by several factors, including low response rates (34-45%) in all rounds. In our study, a long questionnaire may have reduced our response rate, especially in Round 1; the first round questionnaire was the longest and most complex, containing items to help participants adapt the indicators. Finally, due the COVID-19 pandemic, we replaced our planned face-to-face events with online discussions which may have reduced the EP members’ motivation to participate.
Some reviews found that three-round Delphi processes reported response rates between 45% and 93%,43 but less than a third (31%) of included studies had reported response rates for all rounds.39 Differences in reported response rates can be also explained by different denominators used to calculate them (e.g., number of emailed participants, participants who agreed to participate, or participants who completed the survey in the previous round). To improve the response rates in our study, we used online management survey software to design and administrate user-friendly survey to maintain participants’ motivation and to send weekly reminders to non-resondents.35
The WHO IDCCP Toolkit30 and previously published studies that evaluated CC control services for women living with HIV in SSA focused primarily on the secondary prevention portion of the cascade (screening, treatment of precancerous lesions and follow up). Our study identified core and optional indicators across the CC control continuum, from primary prevention to long-term impact and linkage of services. In general, core indicators results in better data and better use of data to improve programs.13,30 Optional indicators add insight into program performance and outcome and capture aspects of patient care in more detail.30 We discussed some of our optional indicators at the satellite meetings, especially those related to WHO’s updated CC screening and treatment recommendations. These discussions highlighted the importance of triage in screening women living with HIV, which may be why two indicators from the domain triage reached consensus in Round 3 instead of Round 2. But both these indicators were still rated low on feasibility and reliability, perhaps because most cervical screening programs in SSA still rely on VIA-based “screen and treat” strategies and have not yet implemented HPV-testing followed by a triage test.22 Though EP members agreed that all optional indicators were highly relevant, comparable, and understandable (high level of agreement in these criteria), at satellite meetings they expressed their concern that it was not feasible to collect the necessary data; this concern was reflected in their ratings. EP members also recognized that it would be useful to disaggregate indicators to identify existing differences in service access and quality within subpopulations,13 but were concerned that it would make data collection, management, and aggregation more complex.
In resource-limited settings, we recommend prioritizing the core indicators that garnered the highest level of agreement for feasibility and reliability. Facilities with mature programs, robust data systems, available resources or needs to monitor specific priorities may consider to include optional indicators. Nevertheless, to perform a comprehensive cascade analysis it is needed to consider all domains of CC control and include both core and optional indicators. In future, researchers and program managers should weigh the benefits of collecting data to inform these indicators against their capacity to collect high-quality data and manage it. Our next step will be to define a minimum data set and variables needed to inform the core and optional indicators to facilitate data collection at HIV facilities offering CC control services. We will implement the variables within the IeDEA DES so we can analyze, interpret, and disseminate CC data and support efforts44 to track the progress of the WHO CC Elimination Strategy,4 with a focus on girls and women living with HIV. International research collaborations, e.g., IeDEA, could increase local capacity to collect and analyze patient-level facility-based data through partnered research activities and help facilities and programs overcome infrastructure or capacity limitations.26 These activities require dedicated resources because each step of the CC prevention and care cascade requires comprehensive assessment. Because many countries in SSA are investing in cost-effective efforts to improve access to and to manage CC screening and treatment services for women living with HIV, we have reason to believe that assessing some indicators might soon become more feasible.45 We should support these efforts by improving monitoring along with data collection and management.
Conclusions
We recommend implementing the 17 indicators (File S6) we identified into routine data collection at HIV clinics and facilities in SSA that offer CC prevention and care services, and this has the potential to significantly increase the quality of data collection and reporting. Programs and facilities can use these core and optional indicators to improve monitoring and evaluation in a variety of contexts, so they can improve cervical cancer control services for women living with HIV.
Supplementary Material
File S1: Inclusion criteria for the Expert Panel members
File S2: Delphi methodology
File S3: Meeting minutes
File S4: Quantitative analysis (ranking and rating)
File S5: Qualitative analysis
File S7: Acknowledgments and members of the International epidemiology Databases to Evaluate AIDS (Central, East, Southern, and West Africa)
File S6: The final list of indicators and relevant information
Table S1. Survey rates calculations
Table S2. List of documents reviewed and number of extracted indicators
Table S3. Self-reported characteristics of the Expert Panel members per round and in total
Table S4. Other experiences reported by the Expert Panel Members in rounds 2 and 3
Figure S1. Three-round online Delphi process
Figure S2. List of indicators that did not reach consensus
Acknowledgments
We thank the Expert Panel members for their participation and contribution, IeDEA Cancer Working Group Members (Michael Silverberg, Sally Coburn, and Jessica L Castilho), and IeDEA Executive Committee (Annette H Sohn) for their feedback on the manuscript. Additional appreciation goes to Kali Tal for her editorial assistance, Alena Zwahlen for her technical assistance, Ellen Brazier and Nicolas Bonadies for their input on Delphi methodology, and Mathurin Fatou, Nicolas Loizeau, and Brady Hooley for their revision on translation of surveys. We express our sincere gratitude to the interpreters, Giselle Efron and Elisabeth Perelló-Santandreu (AVL, Switzerland), and facilitators, Corinne Sprecher (Begegnungsreich, Switzerland) and Nadia Von Holzen (Learning Moments, Switzerland) who provided exceptional support during the Stakeholder Meeting 2019 and Satellite Sessions.
Funding
This research was supported by: the Swiss National Science Foundation (SNSF), under funding scheme r4d (Swiss Programme for Research on Global Issues for Development), grant number 177319; the U.S. National Institutes of Health’s National Institute of Allergy and Infectious Diseases, the Eunice Kennedy Shriver National Institute of Child Health and Human Development, the National Cancer Institute, the National Institute of Mental Health, the National Institute on Drug Abuse, the National Heart, Lung, and Blood Institute, the National Institute on Alcohol Abuse and Alcoholism, the National Institute of Diabetes and Digestive and Kidney Diseases, and the Fogarty International Center (Central Africa, U01AI096299; East Africa, U01AI069911; Southern Africa, U01AI069924; West Africa, U01AI069919). Informatics resources are supported by the Harmonist project, R24AI24872. This work is solely the responsibility of the authors and may not represent the official views of any of the institutions mentioned above. Complete investigator lists and regional acknowledgments are in the Supporting Information (File S7). Three authors (MD, SLA, and TD) received the SSPH+ Global PhD Fellowship Program in Public Health Sciences, funded by the European Union’s Horizon 2020 research and innovation program under Marie Skłodowska-Curie grant agreement No 801076.
Conflicts of Interest and Source of Funding:
None of the authors have a known conflict of interest. This research was supported by the Swiss National Science Foundation (SNSF) and the U.S. National Institutes of Health (NIH). Three authors (SLA, MD, and TD) received the SSPH+ Global PhD Fellowship Program in Public Health Sciences funded by the European Union’s Horizon 2020 research and innovation program under Marie Skłodowska-Curie grant agreement No 801076. One author (SLA) received the Swiss Government Excellence Scholarship, No 2019.0741.
The list of abbreviations
- AIDS
Acquired Immune Deficiency Syndrome
- CC
Cervical Cancer
- DNA
Deoxyribonucleic acid
- EP
Expert Panel
- HIV
Human Immunodeficiency Virus
- HPV
Human Papillomavirus
- IDCCP
Improving Data for Decision-making in Global Cervical Cancer Programs
- IeDEA
The International epidemiology Databases to Evaluate AIDS
- IeDEA DES
The IeDEA Data Exchange Standard
- PEPFAR
United States President’s Emergency Plan for AIDS Relief
- SSA
Sub-Saharan Africa
- VIA
Visual Inspection with Acetic Acid
- WHO
World Health Organization
- WLHIV
Women living with HIV
Footnotes
Some of the data were presented at the World Cancer Congress 2022, Oct 18-20, 2022, Geneva, Switzerland.
Competing interests
The authors have no relevant financial or non-financial competing interests to report.
Disclaimer
Some of the data were presented at the World Cancer Congress 2022, Oct 18-20, 2022, Geneva, Switzerland. This publication was a requirement for MD’s PhD degree.
References
- 1.Stelzle D, Tanaka LF, Lee KK, et al. Estimates of the global burden of cervical cancer associated with HIV. Lancet Glob Health. 2021;9(2):e161–e169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Liu G, Sharma M, Tan N, et al. HIV-positive women have higher risk of human papilloma virus infection, precancerous lesions, and cervical cancer. AIDS. 2018;32(6):795–808. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ibrahim Khalil A, Mpunga T, Wei F, et al. Age-specific burden of cervical cancer associated with HIV: A global analysis with a focus on sub-Saharan Africa. Int J Cancer. 2022;150(5):761–772. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.World Health Organization. Global strategy to accelerate the elimination of cervical cancer as a public health problem. Geneva: World Health Organization; 2020. [Google Scholar]
- 5.Rao DW, Bayer CJ, Liu G, et al. Modelling cervical cancer elimination using single-visit screening and treatment strategies in the context of high HIV prevalence: estimates for KwaZulu-Natal, South Africa. Journal of the International AIDS Society. 2022;25(10):e26021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Boily M-C, Barnabas RV, Rönn MM, et al. Estimating the effect of HIV on cervical cancer elimination in South Africa: Comparative modelling of the impact of vaccination and screening. eClinicalMedicine. 2022;54:101754. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.World Health Organization, World Health Organization. Reproductive Health, World Health Organization. Chronic Diseases, Health Promotion. Comprehensive cervical cancer control: a guide to essential practice. World Health Organization; 2006. [Google Scholar]
- 8.Drummond JL, Were MC, Arrossi S, Wools-Kaloustian K. Cervical cancer data and data systems in limited-resource settings: Challenges and opportunities. Int J Gynaecol Obstet. 2017;138 Suppl 1:33–40. [DOI] [PubMed] [Google Scholar]
- 9.Wagner AD, Gimbel S, Ásbjörnsdóttir KH, et al. Cascade Analysis: An Adaptable Implementation Strategy Across HIV and Non-HIV Delivery Platforms. J Acquir Immune Defic Syndr. 2019;82 Suppl 3(Suppl 3):S322–s331. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.World Health Organization. WHO framework for strengthening and scaling-up of services for the management of invasive cervical cancer. Geneva: World Health Organization; 2020. [Google Scholar]
- 11.National Cancer Institute Division of Cancer Control and Population Sciences. Cancer Control Continuum. National Cancer Institute, 2020. Sept. Available at: https://cancercontrol.cancer.gov/about-dccps/about-cc/cancer-control-continuum. [Google Scholar]
- 12.World Health Organization. Regional Office for the Western Pacific. Metrics for monitoring the cascade of HIV testing, care and treatment services in Asia and the Pacific. Manila: WHO Regional Office for the Western Pacific; 2014. [Google Scholar]
- 13.World Health Organization. Consolidated HIV strategic information guidelines: driving impact through programme monitoring and management. Geneva: World Health Organization; 2020. [Google Scholar]
- 14.Rohner E, Mulongo M, Pasipamire T, et al. Mapping the cervical cancer screening cascade among women living with HIV in Johannesburg, South Africa(a). Int J Gynaecol Obstet. 2021;152(1):53–59. [DOI] [PubMed] [Google Scholar]
- 15.Mabachi NM, Wexler C, Acharya H, et al. Piloting a systems level intervention to improve cervical cancer screening, treatment and follow up in Kenya. Front Med (Lausanne). 2022;9:930462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Taghavi K, Mandiriri A, Shamu T, et al. Cervical Cancer Screening Cascade for women living with HIV: A cohort study from Zimbabwe. PLOS Global Public Health. 2022;2(2):e0000156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Boni S, Tchounga B, Comoe K, et al. Assessment of the scale-up of cervical cancer screening in Abidjan stratified by HIV status. International Journal of Gynecology & Obstetrics. 2019;147(2):246–251. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Quedraogo Y, Furlane G, Fruhauf T, et al. Expanding the Single-Visit Approach for Cervical Cancer Prevention: Successes and Lessons From Burkina Faso. Glob Health Sci Pract. 2018;6(2):288–298. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Campbell C, Kafwafwa S, Brown H, et al. Use of thermo-coagulation as an alternative treatment modality in a ‘screen-and-treat’ programme of cervical screening in rural Malawi. Int J Cancer. 2016;139(4):908–915. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Horo A, Jaquet A, Ekouevi DK, et al. Cervical cancer screening by visual inspection in Côte d’Ivoire, operational and clinical aspects according to HIV status. BMC Public Health. 2012;12:237. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Asangbeh-Kerman SL, Davidović M, Taghavi K, et al. Cervical cancer prevention in countries with the highest HIV prevalence: a review of policies. BMC Public Health. 2022;22(1):1530. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Asangbeh SL, Davidović M, Taghavi K, et al. Cervical cancer prevention and care in HIV clinics across sub-Saharan Africa: results of a facility-based survey. Chimbetete C, editor. AfricArXiv [Preprint], 2023. Jul 13. [DOI] [PMC free article] [PubMed]
- 23.Huchko MJ, Maloba M, Nakalembe M, Cohen CR. The time has come to make cervical cancer prevention an essential part of comprehensive sexual and reproductive health services for HIV‐positive women in low‐income countries. J Int AIDS Soc. 2015;18:20282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Kintu A, Sando D, Okello S, et al. Integrating care for non-communicable diseases into routine HIV services: key considerations for policy design in sub-Saharan Africa. J Int AIDS Soc. 2020;23(S1):e25508. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Hyle EP, Naidoo K, Su AE, El-Sadr WM, Freedberg KA. HIV, tuberculosis, and noncommunicable diseases: what is known about the costs, effects, and cost-effectiveness of integrated care? J Acquir Immune Defic Syndr. 2014;67 Suppl 1(0 1):S87–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Adebamowo CA, Casper C, Bhatia K, et al. Challenges in the detection, prevention, and treatment of HIV-associated malignancies in low- and middle-income countries in Africa. J Acquir Immune Defic Syndr. 2014;67 Suppl 1(0 1):S17–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Sigfrid L, Murphy G, Haldane V, et al. Integrating cervical cancer with HIV healthcare services: A systematic review. PLoS One. 2017;12(7):e0181156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Coleman JS, Cespedes MS, Cu-Uvin S, et al. An Insight Into Cervical Cancer Screening and Treatment Capacity in Sub Saharan Africa. J Low Genit Tract Dis. 2016;20(1):31–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Chammartin F, Dao Ostinelli CH, Anastos K, et al. International epidemiology databases to evaluate AIDS (IeDEA) in sub-Saharan Africa, 2012-2019. BMJ Open. 2020;10(5):e035246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.World Health Organization. Improving data for decision-making: a toolkit for cervical cancer prevention and control programmes Geneva: World Health Organization; 2018. [Google Scholar]
- 31.International Cancer Control Partnership. ICCP Portal. International Agency for Research on Cancer. Available at: https://www.iccp-portal.org/.
- 32.Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Practical assessment, research, and evaluation. 2007;12(1):10. [Google Scholar]
- 33.Daniel B, Nicolas F, Heiko A, Sascha LS, Vera MS. Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements. MethodsX. 2021;8:101401. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Jones J, Hunter D. Qualitative Research: Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. 2017(1477-030X (Electronic)). [DOI] [PubMed] [Google Scholar]
- 36.von der Gracht HA. Consensus measurement in Delphi studies: Review and implications for future quality assurance. Technological Forecasting and Social Change. 2012;79(8):1525–1536. [Google Scholar]
- 37.UNAIDS HD. An Introduction to Indicators. UNAIDS Monitoring and Evaluation Fundamentals. Geneva: UNAIDS; 2010. [Google Scholar]
- 38.Diamond IR, Grant RC, Feldman BM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–409. [DOI] [PubMed] [Google Scholar]
- 39.Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Maguire M, Delahunt B. Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars. All Ireland Journal of Teaching & Learning in Higher Education. 2017. Oct 31;9(3). [Google Scholar]
- 41.Byrne D A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Quality & Quantity. 2022;56(3):1391–1412. [Google Scholar]
- 42.World Health Organization. WHO guideline for screening and treatment of cervical pre-cancer lesions for cervical cancer prevention. Geneva: World Health Organization; 2021. [PubMed] [Google Scholar]
- 43.Gargon E, Crew R, Burnside G, Williamson PR. Higher number of items associated with significantly lower response rates in COS Delphi surveys. J Clin Epidemiol. 2019;108:110–120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Basu PLE, Carvalho AL, Sauvaget C, Muwonge R, Herrero R, Sankaranarayanan R Cancer Screening in Five Continents. International Agency for Research on Cancer, 2019. Available at: https://canscreen5.iarc.fr. [Google Scholar]
- 45.Campos NG, Lince-Deroche N, Chibwesha CJ, et al. Cost-Effectiveness of Cervical Cancer Screening in Women Living With HIV in South Africa: A Mathematical Modeling Study. J Acquir Immune Defic Syndr. 2018;79(2):195–205. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
File S1: Inclusion criteria for the Expert Panel members
File S2: Delphi methodology
File S3: Meeting minutes
File S4: Quantitative analysis (ranking and rating)
File S5: Qualitative analysis
File S7: Acknowledgments and members of the International epidemiology Databases to Evaluate AIDS (Central, East, Southern, and West Africa)
File S6: The final list of indicators and relevant information
Table S1. Survey rates calculations
Table S2. List of documents reviewed and number of extracted indicators
Table S3. Self-reported characteristics of the Expert Panel members per round and in total
Table S4. Other experiences reported by the Expert Panel Members in rounds 2 and 3
Figure S1. Three-round online Delphi process
Figure S2. List of indicators that did not reach consensus