How do cancer screening guidelines trade off benefits versus harms and burdens of screening? A systematic survey

Abstract

Objectives

Cancer screening guidelines differ in their recommendations for or against screening. To be able to provide explicit recommendations, guidelines need to specify thresholds for the magnitude of benefits of screening, given its harms and burdens. We evaluated how current cancer screening guidelines address the relative importance of benefits versus harms and burdens of screening.

Data source

We searched the Guidelines International Network, International Guideline Library, ECRI Institute and Medline. Two pairs of reviewers independently performed guideline selection and data abstraction.

Eligibility criteria

We included all cancer screening guidelines published in English between January 2014 and April 2019.

Results

Of 68 eligible guidelines, 25 included a statement regarding the trade-off between screening benefits versus harms and burdens (14 guidelines), or a statement of direction of the net effect (defined as benefits minus harms or burdens) (13 guidelines). None of these 25 guidelines defined how large a screening benefit should be to recommend screening, given its harms and burdens. 11 guidelines performed an economic evaluation of screening. Of these, six identified a key benefit outcome; two specified a cost-effectiveness threshold for recommending a screening option. Eight guidelines commented on people’s values and preferences regarding the trade-off between benefits versus harms and burdens.

Conclusions

Current cancer screening guidelines fail to specify the values and preferences underlying their recommendations. No guidelines provide a threshold at which they believe the benefits of screening outweigh its harms and burdens.

PROSPERO registration number

CRD42019138590.

Strengths and limitations of this study.

• This systematic survey adheres to the state-of-the-art systematic summary methodology.

• The report states explicit definitions of key concepts, and detailed criteria for making judgements for setting a threshold of a key beneficial outcome, that ensures reproducible and accurate inferences.

• The findings may be limited by under-reporting of methods in eligible guidelines.

• The findings may only be applicable to guidelines published in English.

Introduction

Cancer is the second leading cause of death globally, with an estimated 9.6 million deaths in 2018.¹ To reduce the mortality and burden of cancer, cancer screening programmes and associated guidelines for screening practice have been established.^{2 3} However, cancer screening recommendations implemented in countries with similar levels of healthcare spending differ.⁴ Even guideline panels using similarly rigorous methods for guideline development reach different conclusions.^{5 6} The differences are likely to stem from variation in how guideline panels value benefits, harms and burdens of screening, and what evidence they consider.⁴ People’s perspective on benefits-harm trade-offs might vary according to the importance individuals place on possible benefits versus harms and burdens of screening.⁷

Ideally, guidelines should entail explicit assumptions regarding values and preferences that underlie their judgements of the trade-off between screening benefits versus its harms and burdens.⁸ However, weighing these desirable and undesirable consequences, and clearly articulating the underlying values and preferences while simultaneously considering multiple outcomes, can be challenging. Researchers have developed frameworks for incorporating patients’ preferences regarding benefits and harms while simultaneously considered multiple outcomes into the evaluation of health interventions.^{9 10} Due to the complexity of the process, these approaches have not seen wide use.

Cancer screening guidelines have a potential advantage over guidelines in other areas of medicine because they often have one or two key benefit outcomes: reduction in cancer mortality or cancer incidence. Therefore, the central question guideline panels face may be framed as: given the harms and burdens of screening, what magnitude of its key benefits (effect on cancer incidence and/or mortality) would people require to undergo screening? In other words, what is the threshold above which people would undergo screening and below which they would not? Some may argue that the values and preferences regarding such threshold might vary among target population. However, guideline panel could always identify the distribution of individuals’ values and preferences and find the magnitude of benefit that majority of target population would require. By establishing such a threshold, a panel makes transparent, through a quantitative trade-off between benefits and harms or burdens, their assessment of the values and preferences of the target population.

We evaluated how current cancer screening guidelines address the relative importance of benefits versus harms and burdens. In particular, we evaluated:

1. The proportion of cancer screening guidelines that established a threshold for key benefit outcomes (reduction in cancer incidence or mortality) given the evidence of harms and burdens, above which people would undergo screening and below which they would not.

2. The proportion of cancer screening guidelines that qualitatively traded off the benefits versus the harms and burdens of screening.

3. The proportion of cancer screening guidelines that commented on the values and preferences of target population regarding the trade-off between the benefits versus the harms and burdens of screening.

Methods

Design overview

We conducted a systematic survey of cancer screening guidelines published between 1 January 2014 and 30 April 2019 using the standard methodology for systematic surveys.¹¹

As defined by the WHO, screening is the presumptive identification of unrecognised disease in an apparently healthy, asymptomatic population by means of tests, examinations or other procedures that can be applied rapidly and easily.¹² For this systematic survey, we defined cancer screening guideline as guidelines addressing recommendations of screening technologies for early detection or prevention of cancer, including guidelines that address the entire spectrum of screening, diagnosis and management. Ideally, guideline panels consider benefits, harms or burdens to make recommendations. Guideline panels may, if cost is in their scope, also consider cost-effectiveness to generate a cost-effectiveness ratio.¹³

For each identified screening guideline, we evaluated if the guideline included information about (1) an applied threshold for the key benefit outcome (ie, reduction in cancer mortality and/or incidence) required to balance harms or burdens of screening, and (2) an applied threshold for a beneficial cost-effectiveness ratio between benefits and costs of screening. We defined the threshold for a key benefit outcome as the threshold above which typical members of the target population would undergo screening and below which they would decline. We defined the threshold of cost-effectiveness as the magnitude of cost per unit of incremental benefit deemed acceptable to recommend a screening test.

Literature search

An experienced librarian developed a search strategy for the Guidelines International Network-International Guideline Library, ECRI Institute and Medline (online supplemental appendix 1).

The inclusion criteria were cancer screening guidelines published in full text in English between January 2014 and April 2019 including new, updated or adapted guidelines, or consolidated guidelines. We defined consolidated guidelines as those that aggregate existing guidance addressing a disease or condition, and provide recommendations that have been evaluated and found to be up to date; such guidelines may contain new recommendations.¹⁴ We had no restriction with respect to the cancer type.

Two pairs of trained reviewers (LZ and FKN as a pair; LY and YW as another pair) independently performed the title and abstract screening of each citation for potential eligibility and full-text screening in duplicate. The reviewers abstracted data of eligible guidelines using a standardised, pilot-tested electronic form, attempted to resolve discrepancies by discussion and, if disagreement persisted, by discussion with arbitrators (LMH, RAS, GHG).

Data abstraction

We extracted information on year, institution, and country of publication, type of cancer and screening tests, target population and methods for grading quality of evidence or strength of recommendations.

Because a cancer screening guideline might address more than one key clinical question (eg, whether to recommend a particular screening test, or one test over another), we assumed that a guideline panel would apply the same approach for trading off the benefits and harms in each clinical question. Therefore, we abstracted data for one clinical question for each eligible guideline, with the following hierarchy of clinical questions: (1) recommendation for or against cancer screening; (2) a recommendation of one screening test over another(s); (3) a recommendation of a particular starting age for screening over other(s).

We established specific criteria to evaluate whether and how cancer screening guidelines defined a threshold for a key benefit outcome in the trade-off between benefits versus harms and burdens, and in cost-effectiveness evaluation (table 1). We also evaluated whether the guidelines qualitatively commented on the trade-off between benefits versus harms and burdens (eg, a statement of benefits over harms and burdens, or a statement of the magnitude of net effect) or commented on the target population’s values and preferences regarding the trade-off between benefits versus harms and burdens (ie, the relative importance of outcomes or health states of interest related to cancer screening) (online supplemental appendix 2).

Table 1.

Criteria for setting a threshold for the key benefit of cancer screening in the trade-off between benefits versus harms and burdens and in cost-effectiveness evaluation in cancer screening guidelines*

Questions	Criteria or examples for judging ‘yes’
Key steps for setting a threshold for the key benefit of screening given harms and burdens
1. Does the guideline specify cancer mortality and/or incidence as the only key benefit of screening?	Specifying by a statement that cancer mortality and/or incidence is the ‘key’, ‘crucial’, ‘most important’ (or using synonyms) benefit. Specifying by another statement reflecting that cancer mortality and/or incidence is the only/main benefit considered by the guideline panel. Specifying by grading the importance of outcomes.
2. Does the guideline have an explicit statement of key harms and burdens of screening?	Specifying by a statement that certain outcomes are ‘key’, ‘crucial’ or ‘most important’ (or using synonyms) harms or burdens considered by the guideline panel. Specifying by a statement reflecting that certain outcomes are the main harms and burdens considered by the guideline panel. Specifying by grading the importance of outcomes.
3. Does the guideline specify the magnitude of effect of the key harms and burdens of screening?	Yes, specifies in absolute term or relative term. No, not specified.
4. Does the guideline specify the magnitude of effect of the key benefit (ie, reduction in cancer mortality or cancer incidence) that would be required for recommending screening, given the evidence of key harms and burdens?	That is, specifying a threshold for the key benefit that would be required for recommending screening (vs no screening) or a particular screening option (vs other options), given the evidence of harms and burdens.
Key steps for setting a threshold for the key benefit of screening in cost-effectiveness evaluation
1. Does the guideline consider cost-effectiveness evaluation?	The guideline panel performed cost-effectiveness evaluation. The guideline panel identified cost-effectiveness evidence performed by other researchers.
2. Does the guideline identify the key benefit of screening in cost-effectiveness evaluation?	Specifying by a statement of ‘key’, ‘crucial’ or ‘most important outcome’ (or using synonyms). Specifying by other statement reflecting that a certain outcome is the only benefit considered by the guideline panel in cost-effectiveness evaluation. Specifying by grading the importance of outcomes.
3. Does the guideline specify the measurement of the cost-effectiveness ratio for the key benefit?	For example, the incremental cost-effectiveness ratio (ICER).
4. Does the guideline set a cost-effectiveness threshold that would be required for recommending screening (vs no screening) or a particular screening option (vs other options)?	For example, a threshold for ICER that would be required for recommending screening.

*Guidelines that performed each of the key steps meet the criteria of setting a threshold for the key benefit of screening in the trade-off between benefits versus harms and burdens or in cost-effectiveness evaluation.

Analyses

We conducted descriptive analyses for all variables. We summarised categorical variables with frequencies and percentages. Using univariable and multivariable logistic regression analyses, we explored the association between characteristics of guidelines and (1) performing key steps of setting thresholds for a key benefit outcome (table 1), (2) considering the values and preferences of the target population regarding trade-off between benefits and harms or burdens, (3) qualitatively commenting on the trade-off between benefits and harms. We required at least 10 events per category of variables for conducting a regression analysis. We specified a priori the following hierarchy of independent variables (ie, guideline characteristics): (1) region of guideline development (North America vs other regions), (2) use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE) versus no GRADE. We presented associations using ORs and associated 95% CIs. For all analyses, we used R V.3.5.2.¹⁵ P<0.05 provided the threshold for statistical significance.

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination of our research.

Results

Characteristics of included guidelines

Our search resulted in 741 records, and 68 guidelines proved eligible for analyses (figure 1). Of these, 64 (94%) were new or updated guidelines, the remaining were adapted or consolidated guidelines; 42 (62%) were from North America, and 31 (45%) in breast or colorectal cancer (table 2). online supplemental appendix 3 presents the characteristics of the eligible guidelines.

Figure 1.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram representing the systematic literature search. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097

Table 2.

Characteristics of cancer screening guidelines

Guideline characteristics	Total (n=68) n	%
Type of guideline
De novo developed guideline or updated guideline	64	94.1
Adapted guideline	2	2.9
Consolidated guideline	2	2.9
Country of publication by region
North America	42	61.8
Europe and Central Asia	15	22.1
East Asia and Pacific	6	8.8
Latin America and the Caribbean	3	4.4
Middle East and North Africa	2	2.9
Country of publication by income*
High-income countries	63	92.6
Upper middle-income countries	5	7.4
Clinical area
Breast	16	23.5
Colorectal	15	22.1
Cervical	8	11.8
Prostate	8	11.8
Lung	7	10.3
Ovarian	3	4.4
Thyroid	2	2.9
Others†	9	13.2
Method for rating the certainty of evidence or strength of recommendations
GRADE	15	22.1
US Prevention Services Task Force (USPSTF) level of evidence	13	19.1
Modified GRADE	6	8.8
National Comprehensive Cancer Network (NCCN) Categories of Evidence and Consensus	3	4.4
Other methods	16	23.5
No grading for certainty of evidence and strength of recommendation	12	17.6
Not reported	3	4.4

*The category of country income is from World Bank Country and Lending Groups.

†Other clinical areas include anal, endometrial, hepatocellular carcinoma, renal, oral, pancreatic, skin—melanoma, bladder and Kaposi sarcoma (one guideline in each clinical area).

GRADE, Grading of Recommendations, Assessment, Development and Evaluations.

Setting threshold in trade-off between benefits and harms, burdens or in cost-effectiveness evaluation

None of the guidelines defined a threshold for the key benefit that would be required to recommend screening (or for a particular screening test or screening starting age), given screening harms and burdens. Twenty-two (32%) guidelines specified cancer mortality and/or cancer incidence as their key benefit outcome(s) (table 3). Among the 22 guidelines, 17 also specified key harms and burdens including diagnostic procedure-related harms (eg, invasive procedure, radiation exposure, infection), false-positive results, overdiagnosis, overtreatment, cost, life disruption and anxiety. Among the 17 guidelines that specified the key benefits and harms or burdens, nine specified the magnitude of the key harms and burdens, and 11 presented the magnitude of the key benefit in a quantitative way (eg, a small, moderate or large benefit).

Table 3.

Key steps for setting a threshold for the key benefit in cancer screening guidelines

Guideline characteristics	Total (n=68) n	%
Setting a threshold for the key benefit given harms and burdens
Specification of cancer mortality and/or incidence as the only key benefit	22	32.4
Specification of key harms and burdens	33	48.5
Specification of the magnitude of effect of the key harms and/or burdens	26	38.2
Specification of a threshold for key benefit, given the evidence of key harms and burdens	0	0.0
Setting a threshold for the key benefit in cost-effectiveness evaluation
Performance of an economic evaluation or identification of economic evidence	11	16.2
Identification of key benefit for economic evaluation/evidence	6	8.8
Specification of measurement of cost-effectiveness ratio for the key benefit	6	8.8
Specification of a cost-effectiveness threshold that would be required to recommend screening	2	2.9

Of the 68 guidelines, 25 (37%) included a qualitative statement of trade-off between benefits versus harms and burdens (14; 20%), or a statement of direction of net effect (defined as benefits minus harms or burdens) (13; 19%) (two of which stated both). However, none of the 25 guidelines described a quantitative approach for establishing a trade-off between benefits versus harms and burdens or a net effect of screening.

Eleven (16%) guidelines performed economic evaluation or identified economic evidence, among which six identified a key benefit outcome, and two specified a cost-effectiveness threshold required for recommending screening (table 3). Both of these two guidelines used the incremental cost-effectiveness ratio as the variable, and quantified the threshold at $100 000 per quality-adjusted life year gained.

Values and preferences

Eight (12%) guidelines commented on people’s values and preferences regarding the trade-off between benefits versus harms and burdens of screening (table 4). Of these, two reported important uncertainty or variability in people’s values and preferences (table 4). Because no guideline set thresholds for key benefit outcomes, and few considered the values and preferences regarding trade-off between benefits versus harms and burdens, the regression analysis was minimally informative (online supplemental appendix 4).

Table 4.

Considering values and preference in cancer screening guidelines

Guideline characteristics	Total (n=68) n*	%
Explicit comment on people’s values and preferences regarding the trade-off between benefits and harms/burdens of cancer screening	8	11.8
By commenting on values and preferences regarding trade-off between benefits and harms/burdens	8	11.8
By commenting on experience in shared decision-making	2	2.9
Source of information for values and preferences	8	11.8
Systematic review(s) of studies of values and preferences conducted by team(s) other than the guideline developers	4	5.9
Systematic review(s) of studies of values and preferences conducted by the guideline developers	1	1.5
Individual study(ies) of values and preferences identified by the guideline developers	1	1.5
Not reported	2	2.9
Important uncertainty or variability about people’s values and preference regarding the trade-off of benefits and harms/burdens
Yes	2	2.9
No	3	4.4
Unclear	3	4.4

*Some guidelines meet more than one category and are counted several times.

Discussion

An individual’s decision to participate in cancer screening is based on personal judgement of the magnitude of benefit versus harms and burdens. Our study reveals that current cancer screening guidelines lack transparency in trading off benefits and harms or burdens, hence lack transparency in the rationale for their recommendations. Fewer than half provided quantitative estimates of the harms and burdens associated with screening, and many did not quantify key benefits. Very few guidelines explicitly addressed the issue of people’s values and preferences, and none explicitly specified a threshold of key benefit given the magnitude of the harms and burdens. Few guidelines addressed economic issues. Of those that did, only two offered an explicit cost-effectiveness ratio that would justify screening.

Our study has several strengths. We used robust systematic survey methodology including explicit and reproducible broad eligibility criteria without restrictions on the type of cancers, thus ensuring generalisability; sensitive search strategies; and standardised forms for guideline screening and data abstraction. We developed explicit definitions of key concepts and detailed criteria for making judgements regarding the setting of a threshold of a key beneficial outcome. These definitions and criteria ensured reproducible and accurate inferences.

Our study also has limitations. Our evaluation was based on the reporting in guidelines. The findings may be vulnerable to under-reporting or selective reporting of guideline methods. To minimise this vulnerability, we included all published supporting documents for data abstraction. We included only guidelines published in full text in English. Generalisation to guidelines published in non-English languages is therefore limited. Because the primary aim of this study was to evaluate whether current cancer screening guidelines quantitatively trade off the benefits and harms or burdens of screening, we included only guidelines published in the last 5 years, and we did not address the qualitative approach the guideline panels used in the trade-off.

Commentators have frequently called for a requirement for a more systematic incorporation of patients’ preferences in guidelines,^16–19 in particular when treatment is burdensome, benefits are limited or uncertain and harms may impact quality of life.²⁰ Challenges for articulating values and preferences regarding the trade-off between benefits versus harms and burdens lie in how to simultaneously consider multiple outcomes and, given the time and resource limitation in most of the guideline development, how to efficiently address the values and preferences of the target population. These challenges, however, could also be considered as opportunities to develop new and better methods. A recent BMJ Rapid Recommendation addressing colorectal cancer screening has described the first application of a method to elicit, based on the evidence of key harms and burdens, panel members’ view on the target population’s threshold of magnitude of key benefit for undergoing screening. This Rapid Recommendation has shown the feasibility of the method being applied to guide a formal recommendation.^{21 22}

Because values and preferences regarding the threshold benefit required will certainly vary among individuals in the target population, no threshold will be right for all patients (and thus the appropriateness of weak recommendations for virtually all screening tests).²³ The goal for a guideline panel (which can only make recommendations for typical patients) is to identify the distribution of individuals’ values and preferences and find the magnitude of benefit the majority of the target population would require. By establishing such a threshold for the key benefit, a panel makes their assessment of the values and preferences of the target population transparent. The recommendations that follow from the threshold will, ideally, represent the starting point for a shared decision-making discussion between patients and clinicians.

Conclusions

Cancer screening guidelines, relative to many other areas, have the advantage of being able to identify one or two key benefit outcomes. This greatly facilitates an explicit specification of thresholds for the magnitude of benefit required to justify screening given the harms and burdens. Unfortunately, current guidelines do not use this approach.