Decision support tools to improve cancer diagnostic decision making in primary care: a systematic review

Abstract

Background

The diagnosis of cancer in primary care is complex and challenging. Electronic clinical decision support tools (eCDSTs) have been proposed as an approach to improve GP decision making, but no systematic review has examined their role in cancer diagnosis.

Aim

To investigate whether eCDSTs improve diagnostic decision making for cancer in primary care and to determine which elements influence successful implementation.

Design and setting

A systematic review of relevant studies conducted worldwide and published in English between 1 January 1998 and 31 December 2018.

Method

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials were searched, and a consultation of reference lists and citation tracking was carried out. Exclusion criteria included the absence of eCDSTs used in asymptomatic populations, and studies that did not involve support delivered to the GP. The most relevant Joanna Briggs Institute Critical Appraisal Checklists were applied according to study design of the included paper.

Results

Of the nine studies included, three showed improvements in decision making for cancer diagnosis, three demonstrated positive effects on secondary clinical or health service outcomes such as prescribing, quality of referrals, or cost-effectiveness, and one study found a reduction in time to cancer diagnosis. Barriers to implementation included trust, the compatibility of eCDST recommendations with the GP’s role as a gatekeeper, and impact on workflow.

Conclusion

eCDSTs have the capacity to improve decision making for a cancer diagnosis, but the optimal mode of delivery remains unclear. Although such tools could assist GPs in the future, further well-designed trials of all eCDSTs are needed to determine their cost-effectiveness and the most appropriate implementation methods.

INTRODUCTION

A timely diagnosis of cancer is critical, as delays are associated with poorer patient outcomes and survival rates.^1,2 GPs play a key role in early cancer diagnosis, with 75–85% of cases first presenting symptomatically in primary care.^3–5

The primary care interval describes the time from first symptomatic presentation to the GP, through to referral to a specialist.⁶ The length of this interval varies, with many patients presenting to their GP three or more times before referral.⁷ Consequently, interventions that assist GPs’ clinical decision making have the potential to improve the timeliness of cancer diagnosis and improve cancer outcomes.

Electronic clinical decision support tools (eCDSTs) are electronic systems that assist clinical decision making.⁸ Patient-specific information is entered into the eCDST by the GP or can be automatically populated from the patient’s electronic health record. Using validated algorithms, the eCDST produces recommendations, prompts, or alerts for the GP to consider. eCDSTs can be actively used during a GP consultation or may be designed to continuously mine data in the background.

The development of eCDSTs has been driven by the complex nature of a cancer diagnosis. Often, patients present to the GP with non-specific symptoms that have a low diagnostic value.⁹ Algorithms have been designed to apply epidemiological data on combinations of symptoms and test results, and prompt consideration of a cancer diagnosis based on cancer risk thresholds.^10,11 eCDSTs have been proposed as a solution for cancers that are more challenging to diagnose in primary care because of their variable symptomatic presentation and limited specific features.¹²

eCDSTs have been shown to improve both practitioner performance¹¹ and diagnostic accuracy in simulated patients for a range of conditions, such as dementia, osteoporosis, and HIV.¹³ The effects of eCDSTs on referral behaviours have been summarised in a previous systematic review,¹⁴ but it did not investigate cancer diagnosis specifically; consequently, the role of eCDSTs in cancer diagnosis has not been adequately addressed.

This systematic review aimed to summarise existing evidence on the effects of eCDSTs on decision making for cancer diagnosis in primary care, and determine factors that influence their successful implementation.

How this fits in

Electronic clinical decision support tools (eCDSTs) improve practitioner performance and patient care, but their role in cancer diagnosis has not been adequately addressed. This review outlines the effectiveness of eCDSTs for cancer diagnosis and factors affecting their implementation. Decision support tools have been proposed as an approach to reduce delays in diagnosis, particularly for cancer with non specific symptom signatures. To the authors’ knowledge, this is the first systematic review of available publications to inform eCDST implementation in primary care for the diagnosis of cancer.

METHOD

A mixed-methods narrative review was conducted. The review was registered on PROSPERO (registration ID: CRD42018107219) and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria were followed.¹⁵

Search strategy

Electronic searches were run across three databases: MEDLINE, EMBASE (Ovid), and the Cochrane Central Register of Controlled Trials (CENTRAL). The search strategy (available from the authors on request) included MeSH headings and word variations for three terms: ‘general practitioner’, ‘cancer’, and ‘electronic decision support’. All studies from 1 January 1998 until 31 December 2018 were included. Titles and abstracts were screened independently by two reviewers and any disagreements were resolved with a third researcher. To identify studies not found via the electronic searches, reference lists were manually checked, citation tracking was performed, and experts in the field were contacted. The corresponding authors from all the included studies were contacted via email to identify further studies or unpublished research.

Inclusion and exclusion criteria

Studies investigating an eCDST designed to aid decision making for a potential cancer diagnosis were selected. For inclusion in the review, the study had to report on a:

• cancer diagnosis;

• cancer referral; or

• cancer investigation.

Healthcare utilisation and cost, practitioner performance, and other educational outcomes were also included in the study. As the mode and delivery of eCDSTs vary, studies using any form of electronic support that included algorithm-based prompts or recommendations were eligible. Tools that applied risk markers for prevalent undiagnosed cancer were included, as were qualitative studies if they evaluated barriers and facilitators to implementing eCDSTs for cancer diagnoses in primary care.

Exclusion criteria for qualitative and quantitative studies included:

• decision support used for cancer screening in asymptomatic populations, including tools that incorporated risk factors to predict future incident risk of cancer;

• studies that did not involve decision support designed for use in primary care;

• articles not in English;

• unpublished work;

• editorials; and

• academic theses.

Assessment of bias

Several Joanna Briggs Institute (JBI) Critical Appraisal Checklists were used, depending on the study design of the articles included in the systematic review, to assess the risk of bias of included studies.¹⁶ The authors used the following JBI Critical Appraisal Checklists:

• Checklist for Quasi-Experimental Studies (non-randomised experimental studies);

• Checklist for Qualitative Research;

• Checklist for Diagnostic Test Accuracy Studies; and

• Checklist for Randomized Controlled Trials.

Studies with a percentage score of >80% were considered to have a low risk of bias; those with a percentage score of 60–80% were considered to have a moderate risk of bias.

Study design

Data were extracted and analysed separately from included studies, before all results were combined into an extensive narrative synthesis. Segregated methodology was used to synthesise the evidence while maintaining the standard distinction between quantitative and qualitative research, in line with recommendations.^17,18

Data extraction and synthesis

Data extraction was performed, and cross-checked by the two reviewers who screened the articles. For quantitative studies, data extraction was based on an adapted version of the Cochrane Effective Practice and Organisation of Care data-collection checklist.¹⁹ For qualitative studies, data extraction was performed by a reviewer who screened the original articles, together with a third author, guided by the approach of Noblit and Hare.²⁰ This involved identifying the major themes from primary papers, determining how they are related, and building on themes to interpret overarching theories and understandings.²⁰

Categorisation of extracted themes was based on the normalisation process theory (NPT) framework,²¹ a theory used to describe factors and actions that promote or impede the embedding of new technologies into an existing practice. NPT uses four constructs to explain the processes that affect the integration and adoption of new technologies:

• coherence;

• collective action;

• cognitive participation; and

• reflexive monitoring.^22,23

Using these constructs, the barriers and facilitators identified in the qualitative studies were mapped onto the NPT framework to explain the results.²⁰

RESULTS

In total, 1065 titles were identified and 66 full-text papers reviewed for eligibility (Figure 1). Twelve articles, reporting on nine individual studies, fulfilled the selection criteria: eight quantitative,^24–31 three qualitative,^32–34 and one mixed methods.³⁵ Characteristics of the included studies are summarised in Table 1.

Figure 1.

PRISMA flow diagram of literature search. eCDST = clinical decision support tool. PRISMA = Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Table 1.

Characteristics of included studies

Reference	Country	Study type	Cancer type	GPs, n	Patients, n
Gerbert et al (2000)24	US	Before and after, pilot study	Non-melanoma skin cancer	20	30 images of skin lesions (no contact with patients)
Jiwa et al (2006)35,a	UK	Mixed methods: cluster RCT and qualitative	Colorectal	180 (8 qualitative)	514
Kidney et al (2015)25,b Kidney et al (2017)34	UK	Mixed methods: cross-sectional and qualitative	Colorectal	20 practices (number of GPs not given) Qualitative: 18 GPs (+12 practice managers)	809
Logan et al (2002)26	UK	Cluster RCT	Colorectal/gastric/oesophageal and other cancers (not specified)	89 practices (number of GPs not given)	431
Murphy et al (2015)27,b Meyer et al (2016)30	US	Cluster RCT	Colorectal, prostate and lung	72	733
Walter et al (2012)28,b Wilson et al (2013)31	UK	RCT	Melanoma	28	1297
Winkelmann et al (2015)29	US	Diagnostic accuracy	Melanoma	34	12 images of skin lesions (no contact with patients)
Dikomitis et al (2015)32	UK	Qualitative	All	23	n/a
Chiang et al (2015)33	Australia	Qualitative simulated patient study	All	15	n/a

^aMixed-methods publication, contributing to both quantitative and qualitative synthesis.

^bTrial described in two publications. RCT = randomised controlled trial.

The design of each eCDST and key results are summarised in Table 2. Outcomes included:

• appropriateness of care (n = 5);

• diagnostic accuracy (n = 1);

• time to diagnosis (n = 1);

• cost-effectiveness (n = 1);

• process measures (n = 1); and

• qualitative (n = 4).

Table 2.

Quantitative study descriptions and results

Reference	Description of decision support	Primary outcome	Results	Secondary outcomes	Results	Risk of bias scorea
Gerbert et al (2000)24	The eCDST comprised a clinical information form, decision tree, and support features. An algorithm embedded in the software uses clinical information entered by the GP to provide triage recommendations through a decision tree	Appropriateness of decision to triage skin lesion	Intervention: 86.7% Control: 63.3%	Appropriateness of triage decision for cancerous lesions	Intervention: 96.4% Control: 77.9%	High risk
				Appropriateness of triage decision for non-cancerous lesions	Intervention: 83.7% Control: 65.6%
				Mean change in physician performance (range = −15 to 15)	3.15, P<0.001
Jiwa et al (2006)35	An interactive pro forma requested information, through drop-down menus, for 15 clinical signs and symptoms identified as being significant in diagnosing CRC. Once the clinical data were entered in the pro forma, the interactive software offered the GP guidance on which cases needed urgent referral based on UK Department of Health guidelines. A referral letter was then automatically produced, seeking an appropriate appointment at a hospital clinic	Appropriateness of referral	Intervention: 14.2% Control: 19.4% RR 0.73 (95% CI = 0.46 to 1.15), P= 0.18	Assessment score of letters	Intervention: 2.4 Control: 2.1 Mean difference 0.3 (95% CI = 0.17 to 0.42), P<0.0001	Low risk
				Percentage of GPs who used the eCDST in intervention arm	18.1%
Kidney et al (2015)25	Clinical management software was modified to incorporate an algorithm that identifies patients who meet NICE (2005) urgent referral criteria for suspected CRC. The algorithm flagged up patients aged 60–79 years, who had diarrhoea or rectal bleeding for >6 months, or increased haemoglobin accompanied by iron deficiency anaemia. Patients without a previous diagnosis of CRC, whose records indicated that they met urgent referral criteria up to 2 years before the date of the search, were flagged. GPs reviewed the records of patients who were flagged and decided on further clinical management	Patients flagged and needing further review	34%	Percentage of CRC diagnosis in patients who were flagged	1.2%	Moderate risk
Logan et al (2002)26	Laboratory computers were programmed to print a decision prompt based on blood indices on the FBC report received by GPs. The intervention prompt stated ‘consistent with iron deficiency-? cause. Suggest treat with ferrous sulphate, 200 mg tds [three times a day] for 4 months, but check response in 3–4 weeks. Simultaneously investigate cause. Consider barium enema to exclude colorectal problems.’	Appropriateness of referral	Intervention: 45% Control: 49% OR 0.88 (95% CI = 0.60 to 1.29), P = 0.52	Oral iron prescribed	OR 2.19 (95% CI = 1.27 to 3.77), P = 0.005	Low risk
				Adequate dose of iron	OR 1.96 (95% CI = 1.24 to 3.10), P= 0.004
				Adequate course of iron	OR 1.26 (95% CI = 0.77 P= 0.36
				FBC repeated within 6 weeks	OR 0.85 (95% CI = 0.57 to 1.27), P= 0.43
				Normal haemoglobin within 1 year	OR 1.16 (95% CI = 0.74 to 1.80), P= 0.52
Murphy et al (2015)27 Meyer et al (2016)30	Electronic triggers were applied to electronic health record data repositories twice a month over a 15-month period to identify records of patients with potential delays in diagnostic evaluation of CRC, prostate cancer, or lung cancer. Electronic triggers identified ‘red-flag’ symptoms as: a positive FOBT, elevated PSA, iron deficiency anaemia, or hematochezia (blood in the stool). All trigger-positive records were initially considered to be high risk for delayed diagnostic evaluation. The records were then manually checked by study clinicians to determine whether delayed diagnostic evaluation had occurred. The patient’s GP was then contacted, including information about patient’s red flags. The information above was communicated to GPs in three escalating steps: first, secure emails were sent; if the GP did not follow up within 1 week, up to three telephone calls were made to either the GP or their nurse; if no one could be reached, clinic directors were informed	Median time to diagnostic evaluation27	Intervention versus control: Colorectal: 104 versus 200 days, P= 0.001 Prostate: 144 versus 192 days, P= 0.001 Lung: 65 versus 93 days, P= 0.59	Diagnostic evaluation at 7 months	Intervention: 73.4% Control: 52.2% RR 1.41 (95% CI = 1.25 to 1.58), P<0.001	Low risk
		Use of secure emails (cumulative response rate)30	11.1%	Response rates (by role)	GP: 67.9% Nurse: 69.7% P= 0.82
		Telephone calls (cumulative response rate)30	72.1%
		Contacting clinic directors (cumulative response rate)30	73.4%
Walter et al (2012)28	MoleMate is a computerised diagnostic tool that utilises spectrophotometric intracutaneous analysis (SIAscopy) integrated with a primary care scoring algorithm. Clinicians used the MoleMate system to assist their assessment and management of the suspicious lesion, deciding whether to refer patients through the fast-track skin cancer pathway or manage them in primary care.	Appropriateness of referral28	Intervention: 56.8% Control: 64.5% Percentage difference: −8.1% (95%CI = −18.0 to 1.8), P= 0.12	Appropriate management of skin lesions in primary care	Percentage difference 0.5 (95% CI = −0.6 to 2.0)	Low risk
Wilson et al (2013)31	The results of the RCT were used to estimate the expected long-term cost and health gain of the MoleMate system versus best practice	Incremental cost-effectiveness ratio31	£1896/QALY	Sensitivity	2.8 (95% CI = −1.8 to 7.4)
				Specificity	−6.2 (95% CI = −9.9 to −2.6)
Winkelmann et al (2015)29	MSDSLA analysed pigmented skin lesions and generates a ‘classifier score’. Participants were first asked if they would biopsy the lesion based on clinical images, then asked again after observing high-resolution dermoscopy images, and once more when subsequently shown MSDSLA probability information	Diagnostic accuracy	Intervention: 73% Control: 54% P<0.0001	Sensitivity	Intervention: 95% Control: 66% P<0.0001	High risk
				Specificity	Intervention: 55% Control: 46% P<0.0001

^aRisk of bias score calculated according to Joanna Briggs Institute Critical Appraisal Checklists.¹⁶ CI = confidence interval. CRC = colorectal cancer. eCDST = clinical decision support tool. FBC = full blood count. FOBT = faecal occult blood test. MSDSLA = multispectral digital skin lesion analysis. NICE = National Institute for Health and Care Excellence. OR = odds ratio. PSA = prostate-specific-antigen. QALY = quality-adjusted life year. RCT = randomised controlled trial. RR = relative risk.

Appropriateness of referral was defined by the proportion of patients referred who were diagnosed with cancer. The approach to implement an eCDST within GP workflow varied, but the tools were designed to be used in real time, during consultation, or applied outside of the consultation to flag up potential cases of cancer.

Quantitative synthesis of the included studies was not possible because of significant methodological and clinical heterogeneity. Results for the risk of bias assessment for each included study are given in Table 2. In summary, four quantitative studies had a low risk of bias, including the quantitative component of the mixed-methods study,^26–28,35 one had a moderate risk,²⁵ and in two the risk of bias was high.^24,29 Of the qualitative studies, three had a low risk of bias^32–34 and one a high risk, including the qualitative component of the mixed-methods paper.³⁵ High risk of bias did not influence the inclusion of articles in the review.

Quantitative

eCDSTs used during GP consultation

Three studies^26,28,35 examined eCDSTs that were designed to be used during the GP consultation with patients. Jiwa et al³⁵ assessed whether an electronic referral pro forma used when patients present with bowel symptoms improved the appropriateness of referral for colorectal cancer (CRC). Control practices received an educational outreach visit by a local colorectal surgeon. The pro forma had no impact on the appropriateness of referral; however, it did improve the information and quality of the referral in comparison with the standard referral used by the control group.

Logan et al²⁶ used a computer-generated prompt that recommended further investigations to rule out CRC when full blood-count results indicated iron deficiency anaemia. Control practices received laboratory results as per ‘usual care’. The prompts had no effect on the appropriateness of referral or investigation for CRC but, instead, led to increased prescriptions and adequate dose of iron.

Walter et al²⁸ assessed MoleMate, a diagnostic tool for melanoma, which incorporates a scoring algorithm with spectrophotometric intracutaneous analysis (also known as SIAscopy) of pigmented skin lesions. MoleMate did not improve the appropriateness of referral, due in part to the high sensitivity and relatively low specificity set for the eCDST.²⁸ Despite this, a health economic analysis found that, in UK practice, MoleMate was likely to be cost-effective compared with current best practice with an incremental cost-effectiveness ratio of £1896 per quality-adjusted life year gained.³¹

eCDSTs used outside the GP consultation

Two studies assessed eCDSTs designed to be applied outside of the consultation, identifying patients at increased risk of an undiagnosed cancer. Murphy et al²⁷ applied electronic triggers to identify ‘red-flag’ symptoms in patients who had presented to their GP in the previous 90 days, without documented follow-up. There was a statistically significant reduction in time to diagnostic evaluation for CRC and prostate cancer in the intervention arm.

Kidney et al²⁵ evaluated an eCDST that searched the patient’s electronic medical record and created a list of patients at increased risk of an undiagnosed CRC based on National Institute for Health and Care Excellence guidelines for urgent referrals. A third of all patients flagged by the algorithm were judged to need further review by their GP; 1.2% were subsequently diagnosed with CRC.

Clinical images

Two studies^24,29 — both of low quality — that tested eCDSTs to support GPs’ assessment of clinical images of melanoma and non-melanoma skin cancer were identified. Both studies showed an improvement in decision making for cancerous and non-cancerous lesions.

Qualitative

Three studies showed improvements in decision making related to cancer diagnosis,^24,25,29 one showed reduced time to diagnosis,²⁷ and three demonstrated positive effects on secondary clinical or health service outcomes such as prescribing,²⁶ quality of referrals,³⁵ or cost-effectiveness.³¹ Jiwa et al³⁵ and Kidney et al³⁴ conducted a qualitative sub-study within their quantitative evaluation of the eCDST, both involving semi-structured interviews with GPs. Chiang et al³³ and Dikomitis et al³² conducted exploratory qualitative studies of GPs’ experiences using eCDSTs in practice, one of which used simulated consultations. Both studies used NPT as a framework. The themes and constructs extracted from each qualitative study are outlined in Table 3.

Table 3.

Qualitative results: normalisation process theory

	Coherence	Cognitive participation	Collective action	Reflexive monitoring	Risk of biasa
Dikomitis et al (2015)32	The eCDST raised awareness of potential cancer symptoms and alerted GPs to possible risks Outputs such as positive predictive values were valued less than other components of the eCDST	Participants questioned the robustness of the research and expressed a desire to understand the research behind risk calculations Acceptability and useability would have been enhanced had the training been more comprehensive	Incompatibility with other clinical systems, not fitting in with usual workflow Uptake of the eCDST was dependent on time pressures in consultations Perceived extra pressure on secondary care Prompt overload On-screen prompts took attention away from the patient	Not addressed	Low risk
Chiang et al (2015)33	GPs could make sense of the eCDST and identify its benefits eCDST alerted GPs to the potential presence of cancer Provided a tool to reassure patients at low risk Interpretation of how eCDST should be used differed between GPs	Low prevalence of cancer diagnoses in primary care meant that GPs would rarely consider using the eCDST ‘Cancer suspicion was unnaturally heightened’ GP characteristics such as age and experience impacted uptake of the eCDST A disagreement between the eCDST suggestion and the GPs’ clinical assessment led to mistrust of the risk model	Variation in GP use, interpretation of the history and symptoms entered Potential to raise anxiety and lose control of the consultation. ‘Prompt fatigue’ GPs felt the eCDST should be integrated into general practice clinical software eCDST was not considered compatible with current work practices	No monitoring or resources to adapt or evaluate the intervention	Low risk
Jiwa et al (2006)35	Not addressed	Lack of confidence in the eCDST	The eCDST created additional tasks to the referral process Variation in GPs’ views about what constitutes an urgent referral	Not addressed	High risk
Kidney et al (2017)34	GPs were more likely to refer if the recommendation fit with current guidelines	Personal judgement was used to override patients who were flagged It was clear GPs would refer if they were suspicious of cancer, but not all symptoms flagged up were viewed as suspicious by GPs	Consideration of the effect on patient anxiety resulted in not referring those flagged by the algorithm Resource pressure and being told to reduce unnecessary referrals (competing demands)	Not addressed	Low risk
Summary	In three of the four studies, GPs described the eCDSTs as valuable, suggesting the role of these tools in assisting with diagnosis has a place in primary care. They reported that the eCDSTs raised their awareness of symptoms indicative of cancer. In contrast, GPs had different interpretations of how and when the eCDSTs should be used, with some using it with patients who were asymptomatic	Clinician buy-in was identified as one of the limiting factors to successful implementation; disconnection and mistrust of the eCDST occurred when the recommendations conflicted with the GP’s intuition or clinical judgement. Buy-in was stronger if recommendations were in line with clinical knowledge and aligned with the GP’s judgement	Compatibility of the eCDST in current practice and — at an organisational level — was a common barrier across all four studies. The barriers occurred at the interpersonal, clinical, and health-system levels. Introducing the tool disrupted the consultation, with GPs reporting feeling a loss of control. At the clinical level, additional tasks and time pressures impacted clinical flow. At a health-system level, the prompts for referral placed extra pressure on secondary care	There was limited evidence about how GPs monitored and adapted the new interventions over time. No study reported any system in place to monitor and adapt the eCDST to ensure sustainability over time. This important construct is not being considered at the point of implementation

^aRisk of bias score calculated according to Joanna Briggs Institute Critical Appraisal Checklists.¹⁶ eCDST = clinical decision support tool.

Overarching themes

Three core constructs were identified in the synthesis:

• trust;

• the GP’s role as a gatekeeper; and

• the impact on workflow.

Mistrust of the eCDST was driven by the disagreement between the tool’s recommendations and the GP’s assessment, ambiguity of underlying guidelines embedded within the eCDST, and a desire to understand the evidence that underpinned the clinical recommendation.^32,33,35

The GP’s role as a gatekeeper was identified as a barrier due to conflicting referral thresholds between the eCDST and the GP, with GPs concerned about potential over-referral of patients at low risk.^32,34,35 Finally, for eCDSTs designed to be used during consultation, there were challenges due to disruption of the usual workflow and the generation of additional tasks in an already-busy appointment.^32–34

DISCUSSION

Summary

This systematic review evaluated the efficacy of eCDSTs used for cancer diagnosis in primary care, and describes factors that influence effective implementation. Three studies showed improvements in decision making related to cancer diagnosis,^24,25,29 one showed reduced time to diagnosis,²⁷ and three demonstrated positive effects on secondary clinical or health service outcomes such as prescribing²⁶, quality of referrals,³⁵ or cost-effectiveness.³¹ Key qualitative findings related to issues of trust in the tool, the impact on a GP’s role as gatekeeper, and potential negative effects on GP workflow.

eCDSTs that were used outside of GP consultations appeared to be more effective than tools used in real-time during consultation; they seemed to have the ability to detect patients at an increased risk of an undiagnosed cancer, leading to improvements in clinical assessment and time to diagnostic assessment. However, the implementation issues shifted from a disruption of the GP workflow during consultation to the ability to successfully convey the results of the eCDST to GPs outside of the consultation and ensure they acted on the information.^25,27 Communicating this information to GPs did not always lead to follow-up of the patient.

Strengths and limitations

To the authors’ knowledge, this is the first review to evaluate the efficacy of eCDSTs used for cancer diagnosis in primary care and examine factors influencing their effective implementation. Rigorously conducted, it provides a summary of available findings to inform eCDST implementation in primary care for the diagnosis of cancer.

However, there are some limitations. This review is limited by the small number of included studies and large-scale randomised controlled trials: eCDSTs are relatively under-utilised for cancer diagnosis. Further, none of the included studies looked at outcomes such as survival rates and only one evaluated time to diagnosis;²⁷ this highlights the challenges of conducting trials of diagnostic interventions for relatively rare conditions in primary care. Much larger implementation trials with long-term follow-up of cancer diagnoses, stage, and survival are required to determine the magnitude or effect of eCDSTs on cancer outcomes.

Comparison with existing literature

As with this work, a 2011 systematic review by Mansell et al³⁶ did not identify any studies that examined a delay in referral of cancer as a primary outcome; all 22 included studies used a proxy measurement, such as GP knowledge or quality of referrals.

Mistrust of the eCDST was driven by the disagreement between the tool’s suggestion and the GP’s assessment, ambiguity of guidelines, and a desire to understand the underlying research underpinning the clinical recommendation.^32,33 GPs reported that the eCDST compromised their autonomy, with the eCDST recommendation being perceived as ‘the final word’ rather than support at the time of decision making.³³ This is consistent with recent evidence from the GUIDES implementation guidelines for eCDSTs.³⁷ These guidelines comprise a checklist of factors that were found by patient and healthcare users to influence the effectiveness of eCDST implementation. The GUIDES checklist highlighted that the most important factor for successful implementation is ‘trustworthy evidence-based information’.³⁷

As gatekeepers, there is much pressure on GPs to balance the use of limited and costly referrals for tests against potentially missing a cancer diagnosis.³⁸ There are conflicting thresholds when comparing an eCDST’s output with the GP’s ability to refer everyone who was recommended.^32–34 The International Cancer Benchmarking Partnership, a collaboration between six countries, identified that a stronger gatekeeper role, and different cancer-risk thresholds for referral, were associated with poorer cancer survival.³⁹ Concerns about resource constraints and unwillingness to refer differed by country, but was found to play a large role in decision making in Australia and the UK.³⁹

The usability and acceptability of eCDSTs was dependent on several competing issues, such as disruption of workflow, prompt fatigue, and time. There is a growing recognition in the literature that the technology being developed must seamlessly integrate into the current work practices of those using eCDSTs.³⁷ The eCDST’s functionality and how it affects workflow could be mitigated using a consistent feedback loop between GPs and tool designers.⁴⁰ There were no practices in place to monitor and adapt the eCDSTs for use in consultation, and no opportunity for the GPs to critically appraise how the tool affects workflow.

Implications for research and practice

The diagnostic algorithms in the eCDSTs included in this review were of a limited nature, but diagnostic and clinical utility could increase with more sophisticated algorithms that combine a larger number of factors such as symptoms, abnormal test results, and patterns over time. With the advances in artificial intelligence (AI) and machine learning in clinical practice, future developments will likely drive the next wave of eCDSTs. The use of AI has had promising preliminary results in areas such as visual image analysis in dermatology⁴¹ and radiology;⁴² however, further research using large primary care datasets is required before it can be known whether this approach will have utility to improve diagnoses of symptomatic cancers in general practice.

The available evidence in this review suggests that eCDSTs have the capacity to improve decision making for a cancer diagnosis, but the optimal mode of delivery remains unclear. Given the complex nature of a cancer diagnosis, the advancement and sustainability of eCDSTs in primary care relies on a continuous loop of practitioner feedback and refinement. The findings of the review presented here indicate that improvements in their design and implementation are needed to ensure they can be embedded in normal general practice workflows and alter professional decision making as intended. Strategies for effective communication need to be better explored.

Funding

This work was supported by the Cancer Australia Primary Care Cancer Collaborative Clinical Trials Group (PC4). Jon D Emery is supported by a National Health and Medical Research Council Practitioner Fellowship and is a member of the senior faculty of the multi-institutional CanTest Collaborative, which is funded by Cancer Research UK (C8640/A23385).

Ethical approval

Ethical approval was not required for this study.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article: bjgp.org.uk/letters