Clinical decision support for therapeutic decision-making in cancer: A systematic review

Abstract

Cancer management, including supportive care, is complex and requires availability and synthesis of published and patient-specific data to make appropriate therapeutic decisions. Clinical decision support (CDS) may be an effective implementation strategy to support complex decision making although it is unclear whether it improves process outcomes, patient outcomes or both in cancer settings. We therefore conducted a systematic review to identify CDS that have been used to support therapeutic decision making in clinical cancer settings. Outcomes of interest included the effect of CDS on the process, such as clinician’s decision making and effect on patient outcomes. Ten studies met inclusion criteria, with variability in the study design, setting, and intervention. Of the nine studies that measured process outcomes, five demonstrated significant improvement; and of the six that measured patient outcomes, four demonstrated significant improvement. All included studies utilized CDS that were informed by clinical practice guidelines. In conclusion, CDS to guide cancer therapeutic decision making is an understudied but promising area. Further research is needed.

1. Background

Forty percent of the US population will be diagnosed with cancer during their lifetime, and although the treatment and prognosis depends on the type and stage of cancer, the majority of cancer patients will undergo intensive treatment either for curative or palliative intent [1]. Further, provision of supportive care is important to maximize quality of life during treatment and to minimize late effects of therapy. Although treatment guidelines have been developed for many cancers, decision making for cancer management is complicated because many factors influence the best treatment plan for an individual patient. First, the era of genomic testing and precision medicine has introduced a more complex treatment landscape due to the heterogeneity of cancer as a disease and provided insight into individual susceptibility to toxicities and late effects [2]. Second, factors such as patient age, patient functional or clinical status, patient or provider preferences and values or patient insurance status may influence the prescribed treatment plan [3,4]. Finally, our ability to predict, diagnose, and treat both cancer- and therapy-related toxicities has improved greatly, and this sometimes allows for multiple options for an individual’s treatment plan. For each clinical decision, providers are expected to have the most updated information to inform shared decision making with a patient and their family [5,6].

In addition to the complexities of clinically managing cancer, the burden of cancer continues to increase, primarily due to an aging population [7]. The increasing burden of cancer and its costs impact upon clinical demands and overall care delivery [8]. Health information technology (HIT) has frequently been cited as a main driver to any proposed solution. HIT is a key contributor to harnessing “big data” and allowing the healthcare system to learn from every patient, better predict the best treatment options and ultimately, deliver high quality care [9]. With exponentially-increasing volumes of data becoming available through electronic health records (EHR) and technology available to process and translate these data into usable predictive algorithms, a rapid learning health system for cancer care seems more tangible than ever [10].

Clinical decision support (CDS) is a HIT tool that processes patient-specific information through a previously-determined algorithm and provides clinicians with a data-driven recommendation to support clinician decision making at the point of care [11]. Three concepts, or pillars, are required to give CDS the best chance of success in a clinical setting: a strong evidence and knowledge base, clinician adoption, and consistently-updated information [12]. CDS tools or systems were first developed prior to the widespread use of technology in healthcare, and were usually paper-based and knowledge-driven rather than data driven. Although some early CDS systems demonstrated improvement in care delivery, commonly-cited barriers to CDS implementation included poor integration into clinical workflows and inability to constantly update the knowledge base in the CDS tool [13–15]. Electronic and automated CDS were subsequently developed to overcome these barriers, and implementation of CDS has greater success if it is interoperable and integrated with the EHR [16]. With suboptimal or lack of interoperability with the EHR, a CDS may not be perceived as useful and accepted into a busy clinical workflow. Ever-increasing HIT capabilities, however, allows novel approaches to address these barriers, and CDS is now considered a highly-valued component of improving care delivery across the healthcare system [11,17]. Specifically in cancer, CDS has showed promise in cancer screening, prevention, diagnostic, and surgical or radiation oncology settings [18–23].

The benefits of CDS in settings where cancer is clinically managed, specifically to provide decision support for disease-directed therapy or supportive care management, however, are not well known. It is unclear whether CDS in these settings improves process outcomes, such as provider adherence to CDS recommendations, patient outcomes, such as reduction of symptoms or satisfaction with care, or both. Cancer-specific CDS in therapeutic settings should process the current evidence and provide relevant patient-specific knowledge and decision support in an understandable and usable format at the point of clinical care. If integrated into a healthcare setting effectively, CDS may help to support appropriate, evidence-based care and ultimately improve patient outcomes in cancer management. The purpose of this systematic review is to: 1) describe clinical decision support systems that have been used in clinical cancer settings to guide therapeutic decision making, including supportive care management, and 2) measure the effect of CDS on care delivery process and patient outcomes.

2. Methods

2.1. Search strategy

Four databases, PubMed, EmBase, OVID Medline and Institute of Electrical and Electronics Engineers (IEEE) Xplore, were searched to identify studies where electronic or automated CDS was tested to guide cancer therapeutic decision making, including supportive care management (see Appendix 1 for search strategy). The search strategy was developed in consultation with an informationist at the primary author’s academic institution following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [24]. This study was registered in Prospero (https://www.crd.york.ac.uk/PROSPERO/#recordDetails) and the eligibility criteria for included publications was defined a priori. Our search included studies from database inception through November 2018; we also conducted an updated search in April 2019 to identify any recent publications. For this study, we defined CDS broadly, and searched for terms including: CDS system, decision support system, decision aids and expert systems in our search terms. These terms were based off of prior publications [11,12,25]. Inclusion criteria were 1) an electronic-based CDS; 2) studied prospectively in an oncology setting after the diagnosis of cancer had already been made; 3) delivered CDS to a clinician; 4) the CDS provided therapeutic recommendations; 5) original research study; and 6) full text article was available. Pilot studies were included as long the outcome, effect on care delivery process and/or patients, was reported. Studies were included regardless of publication date or language of publication. We excluded studies using a CDS to support cancer screening, cancer risk-assessment, or cancer diagnosis as these clinical settings often differ from settings where cancer is treated, and symptoms are managed. Similarly, we excluded studies where CDS guided surgical decision making, as this differs from therapeutic decision making. We excluded studies that were conducted retrospectively or post-hoc analyses of primary data. We also excluded studies that only involved the patient and not clinician as this does not fit our a priori definition of clinical decision support but rather falls into symptom screening. Reference lists in the full-text reviewed articles were examined, and relevant articles were included for review.

2.2. Screening, abstraction, appraisal and analysis

All eligible studies were entered into an EndNote database and then de-duplicated using the Bramer Method [26]. All citations were then independently screened by two reviewers (MB, MM) by title and abstract using Covidence; [27,28] reason for exclusion was documented. Any potentially relevant citations were then included in the full text review and the same procedures were repeated. Discrepancies were reviewed, and final consensus was achieved with a third reviewer (RS), when necessary.

After full text review, variables of interest for the data synthesis were extracted from each included article by two reviewers (MB and MM). These included: 1) disease(s) or symptom(s) being treated, 2) geographic location of study, 3) the proportion of clinicians who adhered to the CDS recommendation, 4) patient disease or symptom outcomes, 5) if the CDS recommendation was based on a clinical practice guideline (CPG), 6) if the CDS was linked or integrated into the EHR. The Joanna Briggs Institute Critical Appraisal Checklist for Quasi-experimental Studies was used to determine the study quality of the included articles [29,30]. Studies received 1 point for every component met, and total score indicated study quality with 1–3 low, 4–6 moderate, and 7–9 high. Two reviewers (MB and MM) completed the checklist for each of the included studies, and any discrepancies were reviewed for consensus with a third reviewer (RS), when necessary.

All studies were described qualitatively. The CDS were described by study type, geographic location, disease or symptom studied, CDS characteristics. The outcomes of interest were described and synthesized by effect on clinician and/or patient.

3. Results

3.1. Search results

Our initial literature search retrieved 951 citations; after de-duplication, 663 studies were included for title and abstract screening. Reasons for excluding 565 are listed in Fig. 1; most studies were excluded either because they described the technical CDS development process or they were clinical practice guidelines and not CDS studies. Ninety-eight studies were included in the full-text review, two of which were identified by reference searching, and ten unique, original studies met our inclusion criteria and were included in the final review.

Fig. 1.

PRISMA flowchart for literature search.

3.2. Description of studies

Table 1 summarizes the characteristics and findings in each of the ten included studies. Five were prospective, pre-post designs comparing the effect of the CDS intervention to a prior period without the intervention. Four were single-arm interventional studies, one with multiple time points [31], two utilizing historical controls as a comparison group [32,33], and one without a comparison group [34]. Only one study was a randomized control trial (RCT) [35]. Three studies were conducted in the United States (US), and the remaining seven were conducted in Europe.

Table 1.

Summary of Included Studies and Outcome Measures.

Author, year	JBI Quality Score	Study Purpose	Disease or symptom	Study design	No. of sites/providers/patients	Process outcome(s)	Statistically significant benefit of intervention on process outcome?	Patient outcome(s)	Patient-reported	Statistically significant benefit of intervention on patient outcome?	CDS informed from guideline	EHR-integration
[38]	8	To test the impact of a CDSS on prescriber deviations from CDS recommendation for CPG-based pain management in cancer patients	Pain	Pre-post	1/NA/100	Decreased number of patients with at least 1 CPG deviation from 84% to 14% (p < 0.001)	Yes	Pain at rest by NVAS in intervention group decreased from 3.0 to 1.5 compared to discharge (p < .01) and during activitiy decreased from 7.0 to 2.5 (p < .001) in intervention group	Yes	Yes	Yes	Yes
[33]	4	To test the adherence of physician’s treatment decision compared to that provided by a CDSS in breast cancer patients	Breast cancer	Single-arm pilot study	1/13/127	Overall compliance with CDSS recommendation: 61.42% vs 85.03% (p < 0.001)	Yes	NA	No	NA	Yes	No
[34]	3	To test the adherence of physician’s treatment decision (2nd site studied) compared to that provided by a CDSS in breast cancer patients	Breast cancer	Single-arm pilot study	1/NR/NR	Overall compliance with CDSS recommendation: 55%	NA	NA	No	NA	Yes	No
[39]	7	To test the impact of a CDSS on pharmacist deviations from CDS recommendation for CPG-based pain management in cancer patients	Pain	Pre-post	2/14/88	NCCN-CPG adherent pain regimen prescribed not changed (40% vs. 46.9% p = 0.97)	No	Patient-reported attainment of analgesia at 24h: 10.5% improvement (33.3% vs 43.8%, p = .78)	Yes	No	Yes	Yes
[31]	7	To test the feasibility of symptom screening linked to CDSS for clinicians to manage symptoms	Symptoms (multiple)	Single-arm pilot study	1/14/88	Adherence of provider to CDS recommendation averaged 57% (95% Cl: 52–62%)	NA	NA	Yes	NA	Yes	No
[37]	8	To test the implementation of the Distress Assessment and Response Tool (DART)	Emotional distress	Pre-post	1/16/196	Intervention for depression increased from 7% to 33% (p < .001)	Yes	Patients with intervention reported significantly greater satisfaction with emotional support (no further results available); patients with low income reported greater satisfaction with emotional support (8.67 v 5.75, p < .001) and treatment support (9.23 v 7.68, p < .05)	Yes	Yes	Yes	Yes
[36]	8	To test the effect of routine symptom screening linked to CPG-based CDS for clinician follow-up on the patient’s symptom severity	Symptoms (multiple)	RCT	6/NR/358	NA	NA	Mean treatment impact for intervention group was 3.59 severity points (p < 0.001), 43% of the non-intervention group value. Intervention group had 3x fewer severe days (p < .001) than usual care. 10 of 11 measured symptoms were significantly lower for intervention group (p.025 to < .001).	Yes	Yes	Yes	No
[40]	9	To test the impact of a CDS for prescribers on patient-reported pain intensity levels and opioid prescribing practices in cancer patients	Pain	Pre-post	1/NR/247	Proportion of patients starting a new opioid medication was not statistically different between groups (8.8% vs. 10.5%, p = 0.69)	No	Mean pain intensity score were 3.6 and 3.3 between pre- and post-intervention groups (between group difference = 0.12, 95% Cl: −0.33 – 0.58)	Yes	No	Yes	No
[32]	8	To test the impact of a CDSS for breast oncologists when determining treatment plan for cancer patients	Breast cancer	Single-arm pilot study	2/12/70	Site 1: Adherence 96.6%, compliance 64.28%; Site 2: Adherence 79%, Compliance 88%. (From Escher: increased compliance rate of decisions from 79% to 93%)	Yes	NA	No	NA	Yes	No
[41]	6	To test the impact of an anemia-management CDSS on clinicians’ management of cancer patients with anemia	Anemia	Pre-post	1/NR/68	Improvement in mean congruence scores to CPG between pre- and postcohort (3.00 + −1.48 compared to 8.18 + −1.38, p < 0.001)	Yes	Mean hemoglobin levels significantly increased in postcohort (0.80 + − 1.51 compared to 1.90 + − 1.61, p < .01) and patients in postcohort more likely to have Hb > =11 g/dL (OR3.64 (1.12–11.80, p = 0.03))	No	Yes	Yes	No

Of the ten included studies, three provided decision support for cancer-directed treatment, specifically breast cancer, and the other seven for supportive care or symptom management. Of the latter seven, the most common symptom was cancer-related pain (n = 3). Notably, two of these seven studies approached cancer-related symptoms broadly, assessing and providing decision support on multiple symptoms [31,36] while one study focused on patient distress [37].

The CDS interventions themselves varied. Three of the studies utilized the same CDS, OncoDoc, however, each study described a different study design or a different setting [32–34]. The seven studies where the CDS was symptom-focused utilized distinct CDS systems. Six of these interventions utilized patient-reported symptom information that fed the CDS algorithm to prompt and guide a clinician response [31,36–40]. Three of the ten included studies had CDS interventions integrated into the EHR; one of these studies disclosed the EHR vendor as Epic Systems Corporation [37–39,39]. However, all included studies reported that the CDS provided recommendations from a published clinical practice guideline.

3.3. Outcomes

3.3.0.1. Process outcomes

There was variability in the outcomes of the ten included studies (Table 1). Although all of these studies included the clinician in the intervention as an inclusion requirement, one study did not measure process outcomes or an effect on the clinician’s behavior [36]. Of nine studies that included process outcomes, five studies demonstrated an improvement in the clinician adhering to the CDS recommendation, four of which were statistically-significant (all with p < .001). Two studies did not show a significant difference, and the remaining two studies only provided an estimate of provider adherence. The effect varied and due to different process outcome measures, such as adherence to the recommendation (yes/no), provider intervention based on the CDS (yes/no), or deviation from CPG (multiple measures reported), summarizing the magnitude of benefit would not be appropriate. Table 1 provides further details about the individual process outcomes and effects.

3.3.0.2. Patient specific outcomes

Patient outcomes were measured in six of the ten included studies. One study assessed mean hemoglobin levels [41], and the remaining five studies assessed patient-reported symptoms, or patient satisfaction (pain n = 3, multiple symptoms n = 1, distress assessment satisfaction n = 1) [31,36–38,40]. Four of these six demonstrated an improvement in symptoms or in satisfaction with their care for the patients treated by CDS-informed providers (provided between-group differences with p < .05); [36–38,41] and the other two showed no difference in groups. In the studies that showed improvement or benefit to the patient, the outcome measures varied, including pain scores, measured using different instruments, patient satisfaction scores, treatment impact scores, and mean hemoglobin levels. A more in-depth summary of patient-specific benefit cannot be accurately described because of variability in outcome measures and definitions. Table 1 provides further details about the patient outcomes and effects. None of the studies reported a worsening of symptoms or adverse effect related to the CDS.

3.3.0.3. Comparison by outcome

Studies that measured both process outcomes and patient-specific outcome measures (n = 5) varied in the agreement of the outcome effects. Three of these studies reported significant improvements in both process and patient outcomes, and all three CDS focused on symptom management: anemia, emotional distress, and pain [37,38,41]. In addition, two studies, Bertsche et al and Li et al, were integrated into the EHR [37,38].

3.4. Appraisal

Table 1 presents the quality score of the studies that were included in this review using the Joanna Briggs Institute quality appraisal tool [30]. Overall, the study quality was determined to be moderate to high with a mean score of 6.8 out of 9. Most studies explicitly asked a research question, defined a comparison or control group, and stated the study period for each group. In addition, attrition was low. Studies published prior to 2009 scored lower than more recent publications.

4. Discussion

This systematic review of the literature provides a comprehensive summary of the existing studies that have been conducted utilizing CDS to guide therapeutic decision making in cancer settings. We identified ten studies that suggest a trend toward both provider- and patient-benefit. The small number of studies and variable outcome measures, however, are suggestive that this is an understudied area and the effect from CDS interventions on patient outcomes is unclear. Of the nine studies that measured process outcomes, five demonstrated an improvement; and of the six studies that measured patient-specific outcomes, four demonstrated improvement. The findings are discussed here in further detail.

First, because only ten studies met our inclusion criteria, it appears that although there has been much emphasis on utilizing CDS to guide decision making in cancer [11], these systems have not yet been developed, tested, or published. Our initial literature search identified many studies where CDS systems or tools were under development, with many technical aspects described in detail. Therefore, it may be that these studies have not yet been developed into full, testable CDS or that other barriers have developed. Conversely, it is also possible that CDS have been developed and are being utilized to guide therapeutic decision making in cancer, but they have not been studied or published. For example, a 2013 ASCO abstract reported on the development of a CancerLinQ CDS that would provide treatment decision support for breast cancer through an algorithm that made the ASCO guidelines machine readable and patient-tailored through CancerLinQ, a rapid learning system for oncology [42]. This suggests that therapeutic CDS have been developed and are potentially in use but not yet published.

However, it is also possible that CDS relevant for certain diseases and symptoms have not yet been developed at all. This may be due to two possible reasons. First, the heterogeneity of cancer and cancer symptoms may threaten the validity and reliability of a CDS even within a certain cancer setting. For example, the OncoDoc studies encountered many challenges incorporating the complexity of breast cancer treatment into its algorithm and required multiple iterative updates to the algorithm. In addition, common barriers to technology-based approaches, such as cost, usability, and integration into workflow are well-established and may contribute to the lack of full CDS development in this area [11].

Another reason for the small number of studies that were identified may be related to the definition of CDS. We defined CDS as an electronic or automated tool or system that processes current evidence in the context of patient-specific information to provide knowledge and decision support to the clinician at the point of clinical care. We only included studies that provided decision support directly to a clinician and was then anticipated to be delivered to the patient.

Our results differ from a recent systematic review of CDS systems in oncology practice [43]. This review focused on CDS systems used to diagnose, treat, and manage cancer and identified 24 studies. In contrast, our review excluded the diagnostic period, resulting different studies for inclusion for analysis. A major difference, however, relates to variability in definition of CDS. Our search strategies differed slightly, including our inclusion of the terms “expert system” and “decision aid” [11,12,25]. We did not include CDS for “clinical pathways” or “online order entry”. Finally, we defined clinical decision support as a system or tool that provides patient-specific information and a recommendation for management to the provider. We excluded studies where patient-reported symptoms were captured and informed a prompt for follow-up by a clinician if it did not describe the decision recommended to the provider.

There has recently been focus on automated patient-reported symptom screening linked to automated feedback directly to the patient. For example, Basch et al. have extensively studied a symptom tracking and reporting system that can link to automated feedback compared with nursing feedback through an email notification [44]. Although these studies are promising from a symptom self-management approach, they do not meet our definition of CDS as the nursing feedback model did not provide decision support to guide the nursing-patient interaction. Importantly, however, this approach has demonstrated a significant improvement in overall survival for patients in the intervention arm, meaning that systematic screening of patient-reported symptoms is associated with better patient outcomes [45]. An earlier study by Sikorskii et al compared two multi-modal interventions for symptom management and found that both an automated intervention and a nurse-assisted intervention significantly reduced patient-reported symptoms (p < .01) [46] Again, this intervention did not guide the decision making for the nurse. Therefore, in these and other similar studies [47] it is unclear how or if high-quality feedback, such as guideline-informed care, is being provided to the clinician. Novel approaches to this challenge, such as those included in this systematic review by Li et al. [37], Cooley et al. [31], and Mooney et al. [36] suggest a broad symptom screening approach linked to evidence-based guideline recommendation may benefit patients and also be integrated into clinical workflows.

We noted patterns between studies by the outcomes measured. Process outcomes in an implementation study of CDS serve as a surrogate outcome due to its proximity to the intervention and may be easier to directly measure [48]. Although the goal of CDS systems is improving patient outcomes, these may be challenging to measure and changes in patient outcomes may have additional confounding factors that need to be considered. In this systematic review, five of the ten included studies measured both process and patient outcomes, and three of these reported significant improvement in both measures. In these three studies, one studied a CDS system to improve management of anemia [41], another to improve management of fatigue [37], and the third to improve pain management [38]. Two of these, Li et al and Bertsche et al, were integrated into the EHR, and these same two also include patient-reported data that was integrated into the CDS algorithm. The remaining two studies by Raj et al and Christ et al, where both process and patient outcomes were measured, reported no significant improvement in either. Both studies used CDS interventions that provided pain-management decision support. Importantly, of the studies that measured patient outcomes (n = 6), four reported significant improvement. The remaining two studies reported either no change or a non-significant trend toward improvement in the intervention group; no harm or worsening of symptoms was noted.

Integration into workflows should include EHR-integration [49] however, only three of the identified studies included EHR integration as a component of the CDS. EHR-integration, although it may be logistically challenging to implement, may overcome some of the major workflow challenges that many have cited in clinical decision support [11,50]. Kilsdonk et al. conducted a systematic review of barriers to implementing CPG-based CDS systems guided by the human, organizational and technological factors framework 50]. They found that along with utilizing a user-center design process and providing a recommendation at the exact time it is needed by the provider, the system should be integrated into the EHR or computerized provider order entry (CPOE) system to offer the best chance of success in providing useful information and improving patient outcomes. Of the three EHR-integrated studies [37–39] two were recently published, suggesting that future research will continue to explore this important component of CDS development. In addition, of these three that included EHR-integration, two showed improvement in process outcomes and patient-reported outcome measures [37,38].

The geographic location of these studies is important to acknowledge. Although CDS has been highlighted as an important area of research focus by the National Academy of Medicine and other US institutions [11], as well as specifically within cancer-specific organizations, only three of ten studies were conducted in the US. This highlights an important gap which may be related to the complex healthcare system of the US and specific challenges of achieving the Meaningful Use goals of interoperability and health information standards, an important contributor to CDS success and sustainability [51,52].

Finally, all included studies reported using guidelines to inform the CDS recommendation. This is important and promising, as current recommendations include provision of evidence-based care as a necessary component of CDS [16,17].

4.1. Limitations of this study

There are some limitations to acknowledge within this systematic review. Although we conducted our literature search in multiple databases, it is possible that we missed relevant studies that may be indexed elsewhere. Similarly, grey literature was not included in the literature search, which may have limited the inclusion of pilot studies, QI initiatives, or studies with negative findings.

Another limitation is the lack of a singular definition for CDS. We defined CDS broadly in our search, including terms such as “expert system” and “decision aid”, however, we may have missed studies where CDS was the intervention and would have met our definition criteria, but the authors used other terms. Although the informatics literature clearly states the technology-based definition, the clinical arena is sometimes ambiguous and includes a broad range and fast-paced integration of CPOE, order alerts, and other automated approaches to guide clinical decision making. In addition, these initiatives may occur in clinical practice as quality improvement (QI) initiatives that inform iterative updates to EHR-workflow. They may therefore not be published in a peer-reviewed journal or the clinical team leading the QI initiative may not include publication as part of their project. This highlights the importance of rigorous science to inform implementation studies as well as dissemination as a critical component of implementing initiatives to improve care delivery. Both dissemination and implementation should be encouraged across the landscape of quality improvement and assurance projects within healthcare settings.

Finally, we excluded paper-based CDS as the purpose of this study is to understand how technology can improve care delivery. As we move toward learning healthcare systems and interoperability goals, it is imperative that strategies to improve clinical decision making, such as CDS, be developed in an electronic format. It is possible, however, that these excluded studies may have been electronically-based but did not explicitly state that they were using automation or electronic strategies.

5. Conclusion

This study highlights the available evidence related to CDS that have been used in cancer settings to guide therapeutic decision making. Few studies were identified, signifying an important gap that needs to be addressed in future research. The studies that we identified had wide variability in their study setting, design and outcome measures. Encouragingly, all studies prompted a guideline-informed recommendation to the clinician, and more recent studies incorporated patient-reported information, supporting current initiatives toward standardized assessment of PROs and guideline-based interventions [53,54]. Future research should focus on continuing to develop CDS that are usable, provide recommendations that are informed by CPGs to clinicians, are interoperable and integrated into the EHR, and ultimately impact upon and improve patient outcomes.

Summary table.

Clinical decision support may improve the delivery of high quality, individualized care. Cancer treatment decision making is complicated by the increasing knowledge that clinicians require, both patient-specific and from the published literature.

Although CDS has demonstrated improvement in care delivery in many settings, its benefit has not been well-established in cancer treatment settings, specifically where treatment and supportive care decisions are made.

This systematic review of the literature identified 10 studies where CDS was studied in clinical cancer settings to guide therapeutic decision making.

Although few studies were identified, all of the CDS were informed by guidelines, and of those that measured patient- or process-specific outcomes, four of six and five of nine respectively demonstrated benefit.

In addition, terms commonly used in CDS studies, including interventions and outcome measures, are heterogeneous and therefore challenging to interpret the benefit of CDS in cancer settings.

Appendix 1. Concept Box Framework¹

Question: “In what settings have clinical decision support systems been tested in cancer settings to guide therapeutic decision making and in what ways were they successful?”

	Concepts	Related terms
Concept A	Clinical cancer settings	Healthcare settings, cancer treatment
Concept B	Clinical decision support	Clinical decision support system (CDSS), decision support system (DSS), expert system, decision aid
Concept C	Treatment-related	Therapeutic, Supportive care treatment
Concept D	Electronic	Automated
Concept E	Clinical practice guidelines	Evidence-based guidelines, evidence-based practice, evidence-based medicine, practice guidelines
Concept F	Provider response to CDS recommendation	Decision making, Provider adherence, Agreement with recommendation
Concept G	Patient-reported symptoms	Side effects, adverse effects, patient-reported outcomes

Appendix 2. Databases and search terms utilized

Database	Search Terms	Initial publications retrieved
Pubmed	(“neoplasms”[MeSH Terms] OR (“neoplasms”[MeSH Terms] OR “neoplasms”[tiab] OR “cancer”[tiab])) AND (automated[tiab] OR (“electronics”[MeSH Terms] OR “electronics”[tiab] OR “electronic”[tiab])) AND (“decision support techniques”[MeSH Terms] OR (“decision”[All Fields] AND “support”[All Fields] AND “techniques”[All Fields]) OR “decision support techniques”[All Fields] OR (“decision”[All Fields] AND “aid”[All Fields]) OR “decision aid”[All Fields])	307
Embase	(‘clinical decision support’:ab,ti OR ‘clinical decision support’/exp/mj OR ‘decision aid’:ab,ti OR ‘decision aid’/exp/mj OR ‘clinical decision support system’/exp/mj OR ‘expert system’:ab,ti OR ‘expert system’/exp/mj) AND (‘cancer’:ab,ti OR ‘neoplasm’/exp/mj) AND (‘automate-d’:ab,ti OR ‘electronic’:ab,ti)	173
OVID Medline		253
IEEE	“clinical decision support” and “cancer”	218

^1.Toronto Univerity. “Frameworks for creating your search question.” Accessed 2018 from https://guides.library.utoronto.ca/c.php?g=436816&p=2978014. Published 2018. Accessed.