International variation in adherence to referral guidelines for suspected cancer: a secondary analysis of survey data

Abstract

Background

Variation in cancer survival persists between comparable nations and appears to be due, in part, to primary care practitioners (PCPs) having different thresholds for acting definitively in response to cancer-related symptoms.

Aim

To explore whether cancer guidelines, and adherence to them, differ between jurisdictions and impacts on PCPs’ propensity to take definitive action on cancer-related symptoms.

Design and setting

A secondary analysis of survey data from six countries (10 jurisdictions) participating in the International Cancer Benchmarking Partnership.

Method

PCPs’ responses to five clinical vignettes presenting symptoms and signs of lung (n = 2), colorectal (n = 2), and ovarian cancer (n = 1) were compared with investigation and referral recommendations in cancer guidelines.

Results

Nine jurisdictions had guidelines covering the two colorectal vignettes. For the lung vignettes, although eight jurisdictions had guidelines for the first, the second was covered by a Swedish guideline alone. Only the UK and Denmark had an ovarian cancer guideline. Survey responses of 2795 PCPs (crude response rate: 12%) were analysed. Guideline adherence ranged from 20–82%. UK adherence was lower than other jurisdictions for the lung vignette covered by the guidance (47% versus 58%; P <0.01) but similar (45% versus 46%) or higher (67% versus 38%; P <0.01) for the two colorectal vignettes. PCPs took definitive action least often when a guideline recommended a non-definitive action or made no recommendation. UK PCPs adhered to recommendations for definitive action less than their counterparts (P <0.01). There wasno association between jurisdictional guideline adherence and 1-year survival.

Conclusion

Cancer guideline content is variable between similarly developed nations and poor guideline adherence does not explain differential survival. Guidelines that fail to cover high-risk presentations or that recommend non-definitive action may reduce definitive diagnostic action.

INTRODUCTION

The known differences in the proportion of patients diagnosed with cancer at an early stage (when it is still localised) across international jurisdictions suggest that investigating how cancer-associated symptoms are managed by primary care practitioners may be important in understanding and addressing international variation in cancer survival.^1–8 Many countries have guidelines to advise primary care practitioners on the symptoms and signs of potential oncological significance that will require investigation or referral.⁹ However, these guidelines lack a robust primary care evidence base^10,11 and commentators caution that they are ‘guidelines not tramlines’.¹²

The five-day reduction in diagnostic delay observed in the UK since the introduction of the 2005 National Institute for Health and Care Excellence (NICE) guidelines for managing suspected cancer suggests that guidelines are important, but substantial variation persists in the median diagnostic interval between cancer types (26– 156 days).¹³ It is possible, therefore, that the presence of guidelines, the threshold risk for cancer at which the guideline recommends investigation or referral, and the adherence by clinicians to the guideline between countries, may contribute to the differences in cancer survival.^13,14

The International Cancer Benchmarking Partnership (ICBP) seeks to understand how and why cancer survival varies between countries. Eleven jurisdictions in eight countries with universal access to health care via primary care practitioners participate: Australia (New South Wales and Victoria); Canada (British Columbia, Manitoba, and Ontario); Denmark; Norway; Sweden; and the UK (England, Northern Ireland, and Wales). Module three of the partnership (ICBP3) focuses on primary care and conducted a survey investigating systems and practice around the management of symptoms of cancer.¹⁵ It included multistage vignettes involving patients presenting with symptoms suggestive of cancer. The full results are published elsewhere; a positive correlation was found in each jurisdiction between the primary care practitioner’s willingness to act definitively at an earlier stage of the scenarios and 1-year (and conditional 5-year) survival rates.¹⁶

The researchers of this study wanted to explore international variations in cancer-specific diagnostic guidance adherence by primary care practitioners. As such, the aim of this study was to:

• delineate primary care practitioner cancer-specific diagnostic guidance in jurisdictions participating in the ICBP;

• use the survey data to assess guideline adherence;

• specifically look at performance in the UK compared with other jurisdictions; and

• assess whether adherence correlated with cancer survival.

How this fits in

Primary care cancer guidelines exist internationally to prompt the investigation and referral of patients who are symptomatic, but guideline content is variable. Primary care practitioner adherence to current guidelines does not explain differences in cancer survival, but guidelines that fail to cover high-risk presentations or recommend non-definitive action may, by contributing to a lack of definitive diagnostic action, be causing false reassurance and diagnostic delay.

METHOD

Data from British Columbia were excluded from this analysis as the ICBP3 survey in that locality was organised differently; as such, it was not possible to combine responses with other jurisdictions.

Survey

The survey was conducted between May 2012 and July 2013. It contained five vignettes:

• two for lung cancer;

• two for colorectal cancer; and

• one for ovarian cancer.¹⁵

Each vignette described a patient presenting to primary care and the sequential presentation of cancer-related symptoms in up to three stages. Primary care practitioners were asked to choose which action they would take at each stage. If they proposed definitive diagnostic action to confirm or rule out cancer at any stage, no further response was required. A definitive action was defined as any secondary care referral or a test undertaken in primary care with a high probability of diagnosing underlying cancer.¹⁵ Definitive tests were:

• chest X-ray or computed tomography (CT) thorax for lung cancer;

• CT scan of abdomen or colonoscopy for colorectal cancer; and

• abdominal ultrasound, transvaginal ultrasound, or abdominal-pelvic CT scan for ovarian cancer.

UK responders were asked additional questions about their use of, or attitudes towards, cancer guidelines.

Guideline identification

Guidelines for the primary care investigation and referral of patients with suspected lung, colorectal, and ovarian cancer in use at the time of the survey were identified (details available from the authors on request). The recommended actions corresponding to each stage of the vignettes were extracted by the lead author and verified by two others. Any disagreements were resolved by discussion. The same NICE guidance (CG27) was used in all included UK jurisdictions.

Calculating adherence

The actions recommended by the jurisdictional guideline were compared with the action proposed by the survey responders. If more than one action was recommended by the guidance (for example, colonoscopy or referral) choice of either by the primary care practitioner was classified as adherence. If the guideline was ambiguous about timing, a ‘persistent symptom’ was interpreted as meaning one that was present for >3 weeks and ‘recent onset’ was defined as being within 6 months. In the context of the second lung cancer vignette, an upper respiratory tract infection was accepted as a potential explanation for the presence of cough.

The number of primary care practitioners who chose to take a definitive action was assessed, irrespective of whether this action was recommended by the guideline. Adherence was only measured for stage one of each vignette (details available from the authors on request) as definitive action ended the vignette for that responder; consequently there was a significant attrition in the number of responders after stage one.¹⁵

Analysis

To identify factors that may contribute to the relatively low survival rates observed in the UK, pooled estimates for adherence and definitive action were compared for UK jurisdictions and their non-UK counterparts. The estimate for each jurisdiction was weighted by the inverse of the variance to allow for variation in sample sizes. Statistical significance was assessed using the Z-test.

The association between 1-year survival for each of the vignettes with applicable guidelines was investigated using scatterplots; simple linear regression and the R² statistic were used to explore the strength of association. Routinely collected jurisdictional survival data for 2007 were used for the analyses.¹ Multivariate analysis was not feasible as these are population-level data and a limited number of data points were available.

An a priori sensitivity analysis was performed; Denmark was excluded as its health system had undergone significant redevelopment since the publication of the 2007 survival estimates.¹⁷ All analyses were conducted in Microsoft Excel (2010) and Stata (version 12).

The UK survey responders’ additional responses regarding guideline adherence were analysed descriptively.

RESULTS

In total, 2795 primary care practitioners from the 10 jurisdictions (six countries) included in this analysis completed the survey (crude response rate, 12%).¹⁶

Lung cancer

Eight of the 10 jurisdictions had a lung cancer guideline: seven gave recommendations applicable to the first vignette but only one, from Sweden, covered the second. Most guidelines defined 3 weeks as the interval after which action was recommended for an unexplained cough, although the Danish guideline quoted 4 weeks and the Swedish guideline recommended a ‘low threshold’ for action.

For vignette 1, primary care practitioner guideline adherence ranged from 42% to 83%, with lower adherence in UK (England, Northern Ireland, and Wales) compared with non-UK jurisdictions (47% versus 58%, P <0.01). Estimates of adherence are similar to estimates of definitive action: in all cases a chest X-ray was recommended (a definitive action) and this is what primary care practitioners almost invariably proposed.

Although the risk of lung cancer was four times higher in the second vignette (with a positive predictive value of 3.6% versus 0.9%) the lack of applicable guidelines may have contributed to the much lower rate of proposed definitive action (Table 1).

Table 1.

Lung cancer vignettes: guideline adherence and definitive action

	UK			Europe			Canada		Australia
	Wales	Northern Ireland	England	Denmark	Norway	Sweden	Manitoba	Ontario	New South Wales	Victoria
	1-year lung cancer survival, %	28.5	30.6	29.7	34.9	39.5	43.6	42.7	43.4	42.9	42.7
Vignette one summary: 68-year-old female ex-smoker, persistent cough for 3 weeks following URTI with some sputum, examination unremarkable (PPV 0.9%)
Definitive action taken by clinician, %	42	50	51	81	82	83	69	51	52	48
% (95% CI)	47 (42 to 53)			67 (64 to 70)a
Action recommended by guideline	CXR	CXR	CXR	NDG	No guideline	CXR	No guideline	CXR	CXR	CXR
Adherence to guideline, %	42	50	51	–	–	82	–	50	51	46
% (95% CI)	47 (42 to 53)			58 (54 to 61)a
Vignette two summary: 62-year-old male smoker with COPD presents with 1-week history of URTI with increased sputum production, crepitation’s left base (PPV 3.6%)
Definitive action taken by clinician, %	4	0	8	23	17	24	53	40	30	31
% (95% CI)	5 (1 to 9)			31 (27 to 34)a
Action recommended by guideline	NDG	NDG	NDG	NDG	No guideline	CXR	No guideline	NDG	NDG	NDG
Adherence to guideline, %	–	–	–	–	–	24	–	–	–	–

^aP<0.01 for difference between UK and non-UK jurisdictions.

COPD = chronic obstructive pulmonary disease. CXR = chest X-ray. NDG = not defined in guideline. PPV = positive predictive value. URTI = upper respiratory tract infection.

Colorectal cancer

Nine jurisdictions had guidelines covering the two colorectal cancer vignettes but they recommended different actions (Table 2). Three guidelines covering five jurisdictions advised a full blood count as the first step in both vignettes, while the remaining guidelines recommended colonoscopy or specialist referral. Two guidelines recommending a full blood count (Ontario and the UK) also include formal diagnostic algorithms based on specified time intervals and symptom trajectories. In contrast, the guidelines from countries recommending immediate definitive action (Australia and Sweden) were more narrative, describing symptoms and indications for investigation but encouraging primary care practitioners to make their own clinical assessment of cancer risk.

Table 2.

Colorectal cancer vignettes: guideline adherence and definitive action

	UK			Europe			Canada		Australia
	Wales	Northern Ireland	England	Denmark	Norway	Sweden	Manitoba	Ontario	New South Wales	Victoria
1-year colorectal cancer survival, %	73.6	76.2	74.7	77.7	82.4	83.8	82.1	83.9	84.7	85.1
Vignette 3 summary: 43-year-old female, diarrhoea predominant IBS with an increase in colicky, abdominal pain during the day, unresponsive to antispasmodic, no change in bowel habit or weight loss, 5-day regular (not heavy) periods (PPV 0.7%)
Definitive action taken by clinician, %	7	15	7	44	31	38	44	25	42	33
% (95% CI)	8 (5 to 11)			34 (31 to 37)a
Action recommended by guideline	FBC	FBC	FBC	Referral	FBC	NDG	No guideline	FBC	C or R	C or R
Adherence to guideline, %	38	50	51	33	64	–	–	54	39	29
% (95% CI)	45 (40 to 51)			46 (42 to 49)b
Vignette 4 summary: 68-year-old male, loose stools for 4 weeks, normal examination, no significant history (PPV 0.9%)
Definitive action taken by clinician, %	17	16	15	37	21	23	30	25	27	33
% (95% CI)	16 (11 to 22)			27 (23 to 31)a
Action recommended by guideline	FBC	FBC	FBC	Referral	FBC	SC	No guideline	FBC	C or R	C or R
Adherence to guideline, %	64	68	68	20	74	20	–	57	27	33
% (95% CI)	67 (60 to 74)			38 (34 to 43)a

^aP<0.05 for difference between UK and non-UK jurisdictions.

^bP = 0.73 for difference between UK and non-UK jurisdictions.

C or R = colonoscopy or referral. FBC = full blood count, complete blood count, or blood haemoglobin. IBS = irritable bowel syndrome. NDG = not defined in guideline. PPV = positive predictive value. SC = specialist colonoscopy.

Primary care practitioner adherence to colorectal cancer guidelines varied from 20% to 74% across jurisdictions: it was higher when ordering a full blood count (38– 74%) than when arranging a colonoscopy or referral (20–39%). Adherence was no different between the UK and non-UK jurisdictions for vignette 3 (45% versus 46%, P = 0.73), but was higher for vignette 4 (67% versus 38%, P <0.01). In accordance with the guidelines, definitive diagnostic action was taken substantially less frequently by UK than non-UK primary care practitioners in response to both vignettes (8% versus 34% for vignette 3, P <0.01; 16% versus 27% for vignette 4, P <0.01).

Ovarian cancer

The UK and Denmark (four jurisdictions) were the only ones that had guidelines for a diagnosis of ovarian cancer. No recommendations relevant to the vignette scenario were included, so guideline adherence cannot explain the differential survival for ovarian cancer. However, as with lung and colorectal cancer, there was a substantial difference between UK and non-UK primary care practitioners in the propensity to take definitive diagnostic action (38% versus 61%, P <0.01) (Table 3).

Table 3.

Ovarian cancer vignettes: guideline adherence and definitive action

	UK			Europe		Canada			Australia
	Wales	Northern Ireland	England	Denmark	Norway	Sweden	Manitoba	Ontario	New South Wales	Victoria
1-year ovarian cancer survival, %	60.5	63.9	65.4	70.6	75.2	Not available	68	75.9	73.3	73.7
Vignette 5 summary: 53-year-old female, LMP 6 months ago, R colicky lower abdominal pain for 3 weeks, no change in bowel habit (PPV 0.3%)
Definitive action taken by clinician, %	39	38	37	42	44	53	59	73	62	65
% (95% CI)	38 (32 to 34)			61 (58 to 64)a
Action recommended by guideline	NDG	NDG	NDG	NDG	No guideline	No guideline	No guideline	No guideline	No guideline	No guideline

^aP<0.05 for difference between UK and non-UK jurisdictions.

LMP = last menstrual period. NDG = not defined in guideline. PPV = positive predictive value.

Scatterplots and sensitivity analysis

Scatterplots showed no association between guideline adherence and jurisdictional 1-year survival (R² = <0.01 to 0.26; details available from the authors on request). Excluding Denmark from our analyses (to reduce the influence of health system redevelopment) made no clinically important or statistically significant difference to either the pooled results shown in Tables 1–3, or to the reported lack of association between guideline adherence and 1-year survival.

Adherence to UK guidelines

One-quarter of the 639 UK primary care practitioners surveyed (28%, n = 180) reported referring to cancer guidelines ‘usually’ or ‘always’, whereas 8% (n = 51) said they rarely or never acted outside of them. Half thought the removal of guidelines would result in no change in referrals (51%, n = 325). Nine out of 10 UK primary care practitioners stated they would ignore the guidance if they thought a patient had cancer but their symptom profile did not fit the urgent referral criteria (Figure 1); 20% (n = 128) of these said they would ‘record the patient’s history and symptoms in a way that allows them to fit the guidelines for urgent referral’.

Figure 1.

UK primary care practitioners’ response to the question ‘What would you do if you suspect cancer and yet the patient does not reflect guideline referral criteria?’

DISCUSSION

Summary

Guidelines were available in most ICBP jurisdictions for the primary care assessment of lung and colorectal cancer symptoms. Advice applicable to the lung cancer vignettes was identical where defined, but for colorectal cancer the recommended actions varied. Only Denmark and the UK jurisdictions had guidelines for ovarian cancer. Guideline adherence ranged from 20% to 82% across vignettes.

The findings presented here demonstrate that there are several differences between jurisdictions:

• some do not have guidelines for every clinical scenario covered by this study;

• the threshold for definitive action varies; and

• a significant anomaly exists in the lung cancer guidelines — investigation was recommended at a low positive predictive value (PPV) in one vignette but not in another with a higher PPV.

UK adherence was significantly lower than non-UK jurisdictions when a definitive action was recommended, and significantly higher (or equivocal) for a non-definitive recommendation. Differences in the emphasis placed on how strictly clinicians are expected to follow guidelines may explain the variation in adherence: the UK’s NICE guidance sets out clear symptoms on which to base specific actions on, and the Canadian guidelines are more algorithm-based. The Swedish and Norwegian guidelines are less directive than the UK and Canadian guidelines, leaving more to clinical judgement. Differences in approach partly represent how well equipped a healthcare system is to carry out definitive tests, with pathways written with these constraints in mind.¹⁸

One in five UK primary care practitioners admitted to reporting a symptom profile that fit the referral criteria if they were concerned about the possibility of cancer regardless of the presenting symptom profile. This perhaps reflects primary care practitioners’ recognition of the limitations of uniform disease-specific referral guidance and a willingness to bend the rules.¹⁹ It does not indicate whether UK primary care practitioners are bending the rules for the ‘right’ patients or whether this is also the case in other jurisdictions.

Non-UK primary care practitioners were more likely to act definitively outside of guidelines than UK primary care practitioners: Sweden has some of the highest 1-year survival rates, perhaps explained by its relatively high rate of definitive action despite one of the lowest guideline adherence rates. Conversely, England had the highest rates of adherence, a low rate of definitive action, and low survival rates.

Despite these observations, the findings demonstrated no relationship between guideline adherence and cancer survival. This is unsurprising as the effect of guidelines on survival is multifaceted, determined by the threshold for referral or investigation, adherence, and the actions taken by primary care practitioners for cases that fall outside the guidelines. Variables additional to the primary care interval that were not taken into account in this analysis also influence survival outcomes; these include the patient and system intervals, patient factors such as comorbidity, and treatment received.^3,7,8

Strengths and limitations

To the authors’ knowledge, this is the first study to compare the clinical decisions of primary care practitioners in relation to primary care cancer guidelines and 1-year survival. Data from a recent, large validated international survey of primary care practitioners developed by primary care cancer experts were used.¹⁵ The main limitations of this analysis are:

• sampling methods between jurisdictions were variable;¹⁶

• the response rate was low, ranging from 5.5% (England) to 45.6% (Manitoba);¹⁶

• responders were not totally representative of the primary care practitioners in all jurisdictions;¹⁶

• there is a risk that the UK researchers involved in this analysis have incorrectly interpreted each jurisdiction’s guidelines;

• primary care practitioners were not required to access guidelines when completing the survey, although this could closely resemble clinical practice as one-quarter of UK GPs said they ‘usually’ referred to guidelines.

Comparison with existing literature

Guideline adherence was lower for the UK primary care practitioners in the current study compared with the 18 775 urgent referrals audited across 516 Scottish GP practices, for which adherence ranged from 75–94% depending on cancer type.²⁰ However, the Scottish analysis only included referred cases and so adherence to non-definitive and definitive actions other than referral were not included.²⁰

The findings presented here support the ICBP3¹⁶ by suggesting that cancer guidelines may have a modifying effect on primary care practitioner desire to refer or undertake a definitive investigation. Adherence to guidance that will not confirm a suspected diagnosis (for example, full blood count) may inhibit definitive action (for example, colonoscopy), and false reassurance may be given, preventing definitive action when a guideline does not cover the clinical scenario in question.

This survey supports the findings of Cook and colleagues, who interviewed 39 primary care staff in six practices; their results suggested uniform guidelines fail to wholly resolve risk management in primary care. GPs ‘fudge’, ‘embellish’, or ‘bend the rules’ to ensure assessment of patients for whom they have concerns but who did not meet the rigid referral criteria.²¹ The findings presented here also suggest some primary care practitioners share the view of patients, who would prefer to be investigated for suspected cancer at a much lower risk threshold than is currently used,²² a preference taken into account in the 2015 NICE guidelines update.¹⁸

Implications for research and practice

The evidence base underpinning primary care guidelines should be reviewed to prioritise research in primary care cancer diagnosis and to determine whether a standardised clinical guideline should be developed for all jurisdictions, with explicit thresholds for referral or investigation. Interpreting the guidelines to assess primary care practitioner adherence also emphasised the potential difficulty in interpreting three commonly used, if not precisely defined, terms:

• ‘persistent’ symptoms;

• ‘unexplained’ symptoms; and

• ‘recent onset’.

Primary care practitioner guidelines for cancer work-up should take care to use language that does not restrict action in cases where risk factors exist or patients have symptoms that are less well defined.

Research into efficient pathways for definitive investigation of those patients who do not meet current referral criteria but who primary care practitioners perceive might have cancer appears particularly important.²³ Further work should investigate whether strict adherence to available guidance reduces the primary care interval when compared with clinical judgement alone, and which strategies could be implemented to overcome the mechanisms leading to missed opportunities for earlier cancer diagnosis, for example, by drawing on expertise in human factors science and informatics.^24,25

In the UK, primary care practitioner guideline uptake should be studied during the rollout of the 2015 NICE guideline¹⁸ to identify which factors determine adherence and non-adherence, and which techniques are used to ‘safety net’ when guideline criteria are not satisfied.

Considerable variation exists in primary care cancer guideline content, and adherence to current guidelines is not, at present, a major factor in the difference in cancer survival rates between countries. The effect of referral guidelines for suspected cancer on survival is determined by the recommended threshold for referral or investigation, the primary care practitioner’s adherence to guidelines, and the actions taken by primary care practitioners for cases that fall outside of the guidelines. However, guidelines that fail to cover high-risk presentations or those that recommend non-definitive action may be causing false reassurance and diagnostic delay, contributing to a lack of definitive diagnostic action by primary care practitioners.

Funding

This work was supported by Canadian Partnership Against Cancer; Cancer Care Manitoba; Cancer Care Ontario; Cancer Council Victoria; Cancer Institute New South Wales; Danish Health and Medicines Authority; Danish Cancer Society; Department of Health, England; Department of Health, Victoria; Northern Ireland Cancer Registry; the Public Health Agency, Northern Ireland; Norwegian Directorate of Health; South Wales Cancer Network; Swedish Association for Local Authorities and Regions; Tenovus; British Columbia Cancer Agency; Public Health Wales and the Welsh Government. Brian Nicholson was funded by Cancer Research UK and the National Institute for Health Research (NIHR) School for Primary Care Research. Bethany Shinkins is an NIHR-funded research fellow and Richard Neal receives funding from Public Health Wales and Betsi Cadwaladr University Health Board. The views expressed are those of the author(s) and not necessarily those of the funders.

Ethical approval

The original ICBP3 survey received ethical approval from the NRES Committee South Central — Berkshire. This secondary analysis did not require further ethical approval.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

Willie Hamilton was the clinical lead for the current revision of the National Institute for Health and Care Excellence’s 2015 guidance (NG12) on the topic of selecting patients for cancer investigation. His contribution to this article is in a personal capacity and is not to be interpreted as representing the view of the Guideline Development Group, or of NICE itself. All other authors declare they have no competing interests.

Discuss this article

Contribute and read comments about this article: bjgp.org/letters