Skip to main content
International Journal for Quality in Health Care logoLink to International Journal for Quality in Health Care
. 2022 Feb 7;34(1):mzac007. doi: 10.1093/intqhc/mzac007

Using guideline-based clinical decision support in oncological multidisciplinary team meetings: A prospective, multicenter concordance study

Kees C W J Ebben 1,✉,*, Mathijs P Hendriks 2,3,4,*, Lieke Markus 5, Milan Kos 6, Ignace H J T De Hingh 7, Jorg R Oddens 8, Joost Rothbarth 9, Hans De wilt 10, Luc J A Strobbe 11, Maud Bessems 12, Carsten T Mellema 13, Sabine Siesling 14,15, Xander A A M Verbeek 16
PMCID: PMC8934031  PMID: 35137091

Abstract

Background

Multidisciplinary team meetings formulate guideline-based individual treatment plans based on patient and disease characteristics and motivate reasons for deviation. Clinical decision trees could support multidisciplinary teams to adhere more accurately to guidelines. Every clinical decision tree is tailored to a specific decision moment in a care pathway and is composed of patient and disease characteristics leading to a guideline recommendation.

Objective

This study investigated (1) the concordance between multidisciplinary team and clinical decision tree recommendations and (2) the completeness of patient and disease characteristics available during multidisciplinary team meetings to apply clinical decision trees such that it results in a guideline recommendation.

Methods

This prospective, multicenter, observational concordance study evaluated 17 selected clinical decision trees, based on the prevailing Dutch guidelines for breast, colorectal and prostate cancers. In cases with sufficient data, concordance between multidisciplinary team and clinical decision tree recommendations was classified as concordant, conditional concordant (multidisciplinary team specified a prerequisite for the recommendation) and non-concordant.

Results

Fifty-nine multidisciplinary team meetings were attended in 8 different hospitals, and 355 cases were included. For 296 cases (83.4%), all patient data were available for providing an unconditional clinical decision tree recommendation. In 59 cases (16.6%), insufficient data were available resulting in provisional clinical decision tree recommendations. From the 296 successfully generated clinical decision tree recommendations, the multidisciplinary team recommendations were concordant in 249 (84.1%) cases, conditional concordant in 24 (8.1%) cases and non-concordant in 23 (7.8%) cases of which in 7 (2.4%) cases the reason for deviation from the clinical decision tree generated guideline recommendation was not motivated.

Conclusion

The observed concordance of recommendations between multidisciplinary teams and clinical decision trees and data completeness during multidisciplinary team meetings in this study indicate a potential role for implementation of clinical decision trees to support multidisciplinary team decision-making.

Keywords: clinical decision trees, multidisciplinary team meeting, clinical practice guidelines, oncology, clinical decision support system, algorithms

Introduction

Evidence-based clinical decision-making in oncology is increasingly challenging considering the growing amount of available research knowledge, treatment options and target subpopulations characterized by molecular and genetic testing [1–3].

Multidisciplinary teams (MDTs) are the backbone of decision-making in oncology [4]. The MDT discussion serves to obtain insight regarding the patient and disease characteristics on an aggregated level, to consider the diagnostic and treatment options and to reach a multidisciplinary recommendation. MDTs base their recommendations on clinical practice guidelines. However, MDTs can also deliberately recommend an alternative treatment option if they believe this is better suited for an individual patient. Motivations for guideline deviations have to be recorded for legal ground [5, 6], and they can provide insights in alternatives.

To manage all relevant patient and disease characteristics for making multidisciplinary guideline-based recommendations, MDTs could potentially benefit from a computerized clinical decision support system (CDSS). Evidence for complex guideline-based CDSS usage during MDT meetings is limited [7]. Also, it is unknown to what extent the complexity of a decision (i.e. the number of patient characteristics that need to be taken into consideration) is related to the usability of CDSS and concordance with MDTs [8].

It has been shown that implementation of clinical practice guidelines (hereafter: ‘guidelines’) improves the quality of care [9]. However, recommendations in textual guidelines in oncology are often extensive, may be ambiguous and inconsistent [10], spread across the full text of the guideline document, and not systematically aligned with the clinical decision process in the care path. This impedes implementation of guidelines in clinical practice. Previously, Hendriks et al. described a method that remodels guideline recommendations into unambiguous, data-driven decision algorithms called clinical decision trees (CDTs). CDTs were constructed by nodes, branches and leaves, representing data-items (patient and tumor characteristics, e.g. T-stage), data-item values (e.g. ≤T2) and recommendations (e.g. chemotherapy) and are identical representations of the concerning CPGs. To date, CDTs were evaluated on validity for usage in MDTs retrospectively by Hendriks et al. [11] for breast cancer and by Keikes et al. [12] for colorectal cancer.

Implementing CDTs in daily clinical practice proves to be challenging [13–15]. First of all, because physicians may tend to feel compromised in their autonomy and to not accept guidelines in a computerized manner [16]. Secondly, the evidence that clinical decision support increases MDT performance is currently sparse, because adequate techniques to measure MDT performance are challenging [17]. Finally, optimal usage of any guideline-based CDSS requires the explicit availability of relevant patient and disease characteristics during the MDT [18]. The latter implies a motivational and a technical challenge: clinicians should record the appropriate information and the CDSS should be suitable for connection with the electronic health record. However, integration of CDSS in electronic health records is currently challenging.

We performed an observational study to explore the following research questions: (i) what is the concordance between MDT and CDT recommendations for breast cancer, colorectal cancer and prostate cancer, including reporting motivations for deviation of the CDT recommendation?, (ii) to which degree required patient and disease characteristics were available during MDT meetings to apply CDTs such that it results in a guideline recommendation? and the final research question (iii) what is the influence of CDT complexity on concordance?

Methods

Design

This study was designed as a prospective, multicenter, observational, cross-sectional concordance study. The participating medical centers were academic, teaching and general hospitals [Figure 1]. The study design was exempt from approval requirement by independent medical ethics committees.

Figure 1.

Figure 1

Participating hospitals and evaluated cancer types.

Data collection

A medical doctor with several years of (international) experience did observe, but not participate in, the MDT discussion and manually collected all available data at the time of MDT meetings (both discussed data and available reports in the electronic health records) in all participating centers. The MDT meetings were not recorded to minimize a potential Hawthorne effect. The collected data included (i) patient and disease characteristics in general (sex, age, tumor type, and tumor stage), (ii) additional data necessary for completing the relevant CDT, (iii) the individual treatment plan proposed by the MDT and (iv) the reason for deviating from the guideline (if applicable). Data were collected from August 2019 until December 2019. Case report forms are available on request.

Inclusion and exclusion criteria

Patients with suspected or pathological confirmed breast cancer (including ductal carcinoma in situ (DCIS)), colorectal cancer or prostate cancer who are discussed in an MDT meeting were eligible for inclusion, if the intended decision matched 1 of the 17 CDTs under study. The list of selected CDTs is included in Table 1. The tumor types were selected because of their high incidence and availability of guideline-based CDTs, focusing on multidisciplinary decision support.

Table 1.

CDT complexity scores and concordance of 17 CDTs under study.

graphic file with name mzac007a1.jpg

A patient was excluded when (i) the proposed decision fell outside the scope of the guideline (e.g. second relapse); (ii) the proposed decision did not match with 1 of the 17 selected CDTs under investigation (e.g. neoadjuvant therapy and patients with (loco-)regional recurrence) and (iii) the MDT preparation was insufficient and the MDT decided to postpone the decision pending further investigation results.

CDTs

The method for designing CDTs from guidelines is described elsewhere [2]. In short, CDTs are composed of nodes (data-items representing patient and disease characteristics), branches (representing the possible values of the data-items) and leaves (representing recommendations from the guideline). The CDTs are published on www.oncoguide.nl [Figure 2]. By entering patient-specific data, a single path through the CDT is generated leading to the guideline recommendation applicable for this patient. The CDTs evaluated in this study are based on the prevailing Dutch guidelines during the study period (breast cancer version 1.0, 2018 [19]; colorectal cancer version 3.0, 2014 [20] and prostate cancer version 2.1, 2016 [21]). In total, 17 CDTs were selected for evaluation in this study including primary treatment, adjuvant treatment and treatment for metastatic disease (synchronous or metachronous).

Figure 2.

Figure 2

(Continued)

Figure 2.

Figure 2

Examples of clinical decision trees in Oncoguide. (a) Hypothetical CDT for a specific population at a specific step in the care pathway. (b) All data-items (nodes) required by the CDT for this patient are available and filled in on each node, resulting in a single highlighted pathway, leading to a single leaf with CPG recommendation. (c) One data-item (white node) is missing, the CDT generates two possible leaves with CPG recommendations. (d) One data-item (white node) is missing. Since other data-items are known, the CDT generates two leaves with CPG recommendations. CDTs are composed of (I) a stem (defining the population and step in the care pathway the CDT applies to), (II) nodes (data-items representing patient and disease characteristics), (iii) branches (representing the possible values of the data-items) and (IV) leaves (representing recommendations from the CPG). By entering patient specific values, a single leaf with a recommendation applicable for this patient can be generated.

CDT = clinical decision tree; CPG = clinical practice guideline.

Data analysis and statistics

After each MDT meeting, the collected data were plotted onto the corresponding CDT in order to generate a guideline-based recommendation [Figure 2]. To evaluate our secondary objective, patients were assigned to one of two categories: (i) sufficient data were available during the MDT meeting to complete a single pathway through a CDT leading to a guideline recommendation [Figure 2b] or (ii) one or more parameters to fully complete a single pathway were missing [Figure 2c and d]. Consequently, multiple pathways remain open, resulting in more than one possible guideline recommendation.

For our primary objective (concordance), the cases assigned to category 1 (sufficient data) were further analyzed. The recommendation pairs (from MDT and CDT) were assigned to one of the four following groups, depending on the level of concordance: (i) concordant: the recommendation of the MDT was corresponding with (one of) the guideline recommendations; (ii) conditional concordant: the recommendation of the MDT was corresponding with (one of) the guideline recommendations; however, the MDT provides an explicit condition for the recommendation made (e.g. perform surgery after cT1-stage breast cancer based on mammography is confirmed by a MRI scan) and (iii) non-concordant: the recommendation of the MDT was not corresponding with (one of) the guideline recommendations. These are subdivided into (i) motivated cases—the MDT explicitly motivates why they deviate from the guideline—and (ii) not motivated cases—the MDT deviated from the guideline but did not provide a motivation.

Subgroup analyses regarding concordance were performed based on tumor type and tumor stage (represented by the TNM staging system: the tumor, node, metastasis classification of malignant tumors). If available, we categorized the MDT motivations for recommendations that deviated from the guideline: specific tumor characteristics, comorbidity, patient preference, age, study inclusion or obsolete guideline (= a guideline is alleged not to reflect the current status of evidence and therefore presumed to be outdated). These categories were based on prior interviews with several clinicians during the development of the Oncoguide tool. These reasons were categorized and consensus was achieved and implemented in Oncoguide.

Finally, we evaluated the presence of a potential correlation between the complexity of a CDT and the concordance. Complexity of a CDT is defined as a combination of the total number of nodes, the total number of leaves, the number of unique nodes and the tree depth (longest path) [22]. This theoretically results in scores that range from 2 to infinite. Higher scores are related to a more complex decision. The CDTs were then classified in quartiles based on their total complexity score. The first and fourth quartiles were compared for the percentage of concordant cases. The correlation for complexity and concordance was evaluated by a unifactorial analysis of variance.

Data analyses were performed using Microsoft Excel for descriptive statistics.

Results

Inclusion

In total, 59 MDT meetings were attended in 8 different hospitals [Figure 1]. From these meetings, 355 unique cases were included: 118 cases for breast cancer (including DCIS), 111 cases for colorectal cancer and 126 cases for prostate cancer [Table 2, Figure 3].

Table 2.

Patient and disease characteristics of included cases

Total Breast Colorectal Prostate
n 355 118 111 126
Gender
Female (%) 162 (45.6) 117 (99.2) 45 (40.5) NA
Male (%) 193 (54.4) 1 (0.8) 66 (59.5) 126 (100)
Age ± SD, years 66.8 ± 11.3 63.0 ± 12.5 66.3 ± 11.6 71.4 ± 7.4
TNM stage, n (%)a
0 23 (6.5) 23 (19.5) NA NA
I 96 (27.0) 49 (41.5) 26 (23.4) 21 (16.7)
II 88 (24.8) 38 (32.2) 19 (17.1) 31 (24.6)
III 33 (9.3) 7 (5.9) 15 (13.5) 11 (8.7)
IV 114 (32.1) 1 (0.8) 50 (45.0) 63 (50.0)
Unknown 1 (0.3) 0 (0.0) 1 (0.9) 0 (0.0)

NA: not applicable; SD: standard deviation.

a

Percentages may not equal 100% due to rounding.

Figure 3.

Figure 3

Flow diagram of inclusion and exclusion, data completeness and concordance.

MDT: Multidisciplinary Team; CPG: Clinical Practice Guideline; CDT: Clinical Decision Tree. *The MDT was unable to provide a policy proposal due to lacking data.

Availability of data as input for CDTs

For 296 cases (83.4%), all data-items to complete a single CDT pathway were available during the MDT meeting. Per tumor type this was 102 (86.4%) for breast cancer, 90 (81.1%) for colorectal cancer and 104 (82.5%) for prostate cancer [Table 3]. In 59 cases, (16.6%) one or more data-items were not available during the MDT meetings and therefore CDTs generated multiple possible recommendations [Figure 2c-d]. Of these 59 cases, a total of 41 (11.5%) cases resulted of 2 open paths in the CDT, both leading to a recommendation, 9 (2.5%) in 3 open paths and 9 (2.5%) in 4 or more open paths. The distribution regarding the number of highlighted pathways (2, 3, ≥4) for each disease, with stage subdivision, is shown in Table 3. An overview of the missing data-items is presented in Table 4.

Table 3.

Availability of data as input for CDTs.

graphic file with name mzac007a2.jpg

Table 4.

Missing data during MDT meetings per CDT

Cancer type CDT Missing data-item (patient/disease characteristic) Number of cases per CDT in study CDT complexity scoref Data-item missing frequency Percentage of missing data-items per CDT under study
Breast cancer
Primary treatment breast cancer 67 26
cN0 risk statusa 8 11.9
Risk on invasion (DCIS)b 6 9.0
Post-operative adjuvant treatment breast cancer g , h 50 13
Locoregional treatment after breast conserving therapyg 33 22
ER-status 1 3.0
HER2-status 1 3.0
Local treatment after mastectomyg,h 17 51
Regional treatment after mastectomyg,h 17 25
Adjuvant systemic therapyg 50 92
Menopausal statusc 5 10.0
Metastatic diseaseh 1 15
Colorectal cancer
Primary treatment colon cancer 20 20
cT-stage 1 5.0
Adjuvant treatment colon cancer 15 25
Contra-indication for oxaliplatin 3 20.0
Microsatellite status 3 20.0
Primary treatment rectal cancer 22 50
cT-stage 3 13.6
Extramesorectal pathological lymph nodes 2 9.1
Extramural invasion 2 9.1
Tumor diameter 1 4.5
Vascular invasion polypectomy 1 4.5
Polypectomy performed 1 4.5
Differentiation grade 1 4.5
(lymph)angio-invasion 1 4.5
Adjuvant treatment rectal cancer 5 19
Mesorectal fascia distance 5 100.0
Cutting edge 1 20.0
Metastatic disease 49 38
Number of resectable liver metastases 6 12.2
Local treatability liver metastases 4 8.2
Resectability of extrahepatic metastases 2 4.1
Prostate cancer
Primary local treatment 69 25
Chance of lymph node involvementd 8 11.6
Life expectancy 3 4.3
Number of positive biopsies 3 4.3
EAU/ESTRO risk groupe 1 1.4
PSA 1 1.4
Extensiveness disease 1 1.4
Adjuvant treatment 2 12
Cutting edge 1 50.0
Metastatic disease 55 9
Localization of metastases 4 7.3
mCRPC pre-chemotherapy h 13 9
mCRPC post-chemotherapyh 10 9

CDT: clinical decision tree; MDT: multidisciplinary team; DCIS: ductal carcinoma in situ; ER-status: estrogen receptor status; HER2-status: human epidermal growth factor receptor 2 Status; EAU: European Association of Urology; ESTRO: European Society for Radiotherapy and Oncology; PSA: prostate-specific antigen.

N.B. In single cases, >1 data-item can be missing.

NA: not applicable.

a

Aggregated score contains age, HER2 status, ER-status, grade, tumor diameter.

b

Aggregated score contains age, palpability, MRI coloring, grade, tumor diameter.

c

The patients’ age in all five cases was ≥60 years and was therefore in our analyses considered as post-menopausal.

d

Aggregated score (prediction model) contains PSA, cT, Gleason variant 1, Gleason variant 2, Positives cores.

e

Aggregated score contains cN, cT, Gleason, iPSA.

f

CDT complexity scores method are displayed in Figure 3.

g

Multiple CDTs are applicable to each unique case.

h

These CDTs were filled in completely in all applicable cases and therefore had no missing data-items.

Concordance

From the 296 generated CDT recommendations, the MDT recommendations were completely concordant, conditionally concordant and non-concordant in 249, (84.1%), 24 (8.1%) and 23 (7.8%) cases, respectively. In 7 out of 23 (30.4%) non-concordant cases, the MDT did not provide reasons for non-concordance.

Complete and conditional concordance rates for breast cancer, colorectal cancer and prostate cancer were 85.3% and 8.8%, 88.9% and 5.6%, and 78.8% and 9.6%, respectively. For non-concordance, the results were as follows: breast cancer 5.9%, colorectal cancer 5.5% and prostate cancer 11.5% [Table 5]. Subgroup analysis on the effect of tumor stage on concordance showed that 13 (9 prostate cancer cases and 4 colorectal cancer cases) out of 16 (81.3%) motivated non-concordant cases had stage IV disease. Most common MDT motivations for guideline deviation were inclusion in a clinical trial (n= 13), age/comorbidity (n= 10) and specific tumor characteristics (n= 8). In Table 6, all motivations are listed.

Table 5.

Concordance of MDT and CDT recommendations per tumor type and stage

Concordant cases, n (%) Non-concordant cases, n (%)
Total Concordant cases Conditional concordant Total Motivated Not motivated Distribution per tumor stage in research sample in percentages Distribution per tumor stage in the Netherlands in percentages
Breast cancer cases (n= 102) 96 (94.1) 87 (85.3) 9 (8.8) 6 (5,9) 2 (2.0) 4 (3.9) Incidences 2018a
TNM stage 0 17 0 0 0 17 12
I 36 2 0 3 40 41
II 28 5 2 1 35 33
III 5 2 0 0 7 9
IV 1 0 0 0 1 5
Colorectal cancer cases (n = 90) 85 (94.4) 80 (88.9) 5 (5.6) 5 (5.6) 4 (4.4) 1 (1.1) Incidences 2017a
TNM stage 0 NA NA NA NA NA NA
I 22 0 0 0 24 26
II 13 1 0 1 17 23
III 7 3 0 0 11 28
IV 38 1 4 0 48 20
Prostate cancer cases (n= 104) 92 (88.5) 82 (78.8) 10 (9.6) 12 (11.5) 10 (9.6) 2 (1.9) Incidences 2016a
TNM stage 0 NA NA NA NA NA NA
I 14 0 1 0 14 38
II 18 1 0 0 18 20
III 10 1 0 0 11 17
IV 40 8 9 2 57 25
Total (n= 296) 273 (92.2) 249 (84.1) 24 (8.1) 23 (7.8) 16 (5.4) 7 (2.4)

MDT: multidisciplinary team; CPG: clinical practice guideline; CDT: clinical decision tree; NA: not applicable.

a

The most recent complete years per tumor type were retrieved from the Netherlands Cancer Registry.

Table 6.

MDT motivations for conditional concordance and motivations for non-concordance

Breast cancer (n) Colorectal cancer (N) Prostate cancer (n)
MDT motivations for conditional concondant casesa
Uncertainty on patient/tumor characteristics (additional testing will be performed; T-category uncertain) 26 9 21
Specific tumor characteristics (very small size, aggressive biology) 2 2 3
Comorbidity 0 2 0
Patient preference 0 1 0
Other 0 3 2
MDT motivation for non-concordant casesa
Patient preference 0 0 1
Age 0 1 2
Comorbidity 2 3 2
Clinical trial inclusion 1 10 2
Other:
Specific tumor characteristics (very small size, aggressive biology) 3 4 1
Current CPG outdated 0 1 2

MDT: multidisciplinary team; CPG: clinical practice guideline.

a

Multiple motivations can be put forward per case.

CDT complexity

Complexity scores of the included CDTs are available in Tables 1 and 3. The mean concordance of the CDTs in the first quartile and fourth quartile was 89.4% and 91.1%, respectively, and did not differ statistically significantly (P= 0.8).

Discussion

Statement of principal findings

This concordance study in breast, colorectal and prostate cancers showed concordant recommendations between CDT and MDT in a large majority (92.2%) of evaluated cases. In 16.6% of cases, concordance could not be evaluated due to insufficient available patient and disease characteristics during MDT meetings. An unconditional recommendation from a CDT depends on availability of complete data. In this study, data availability per case was higher than previously reported [11, 12, 18]. The systematic application of a CDT uncovers the amount of missing data required for guideline-based decision-making and thereby may stimulate a more complete reporting of necessary data.

Focusing on the most frequently found missing data-items per CDT in this study, there are some remarkable observations: (i) composite data-items like ‘cN0-risk status’ or ‘risk on invasion (in DCIS)’ are prone to be incomplete, perhaps through their complexity and unfamiliarity, (ii) a data-item like ‘contraindication for oxaliplatin’ is important for the final selection of chemotherapy regimen in the outpatient clinic, but it can be argued this goes beyond the scope of the MDT meeting (as assessment of contraindications may be performed by the treating physician), (iii) unavailability of ‘microsatellite stability status’ in colorectal cancer could indicate that this test is not incorporated as standard diagnostic entity in all hospitals and (iv) ‘cT-stage’ in rectal cancer is a known difficult feature, requiring assessment of a dedicated radiologist. The characterization of these data-items is very diverse in terms of data source (radiology, pathology). This emphasizes the importance of involvement of all medical disciplines for effectuating complete registration to enable MDTs making guideline-based recommendations.

In patients where concordance could be evaluated, the MDT recommendation was non-concordant with the CDT recommendation in 7.8% of cases. In nearly a third of those cases, no motivation was reported for guideline deviation. In the CDTs under study, no clear trend was found regarding CDT complexity and concordance. We therefore hypothesize that the used method of CDTs, which is following the clinical processes, is useful for MDT decision support independent of the CDT complexity.

Cases with conditional concordance were provided with a recommendation, but it can be argued that data were missing for unambiguous decision-making. This might indicate either a suboptimal preparation of the MDT or acting on newly acquired insights during the MDT session.

A relatively high number of cases (9%) were excluded from analyses since the MDT was not provided with sufficient information to properly discuss a patient. The discussion and therefore also a proposal for a policy had to be postponed. For these cases, the MDT could be considered inadequately organized. Although not investigated further, our dataset revealed differences in the percentages of exclusions between hospitals due to insufficient preparation. Despite the ubiquitous availability of data in the electronic health records, difficulties in having access to complete information is a known phenomenon in MDTs [23].

Strengths and limitations

This prospective multicenter study included three types of solid cancer at various phases in the clinical pathway, representing a wide variety of MDT-based decisions with their associated specific challenges. Therefore, it is likely that the results of this study can be extrapolated to CDTs of other (oncological) diseases. Another strength was the attendance of an independent researcher who was able to track the course of the MDT discussion, rather than simply extracting the recommendation of the MDT found in the electronic health record, retrospectively.

The current study has a non-interventional design. MDTs were not provided with the CDT and recommendations during or after their discussion. A suggestion for future research is to confront MDTs with CDT recommendations and evaluate if this alters their decision. There are some interventional studies performed, mostly single center studies focusing on one type of malignancy [24–26]. However, obtaining strong evidence is difficult because double-blinded randomized clinical trials are difficult to perform in decision support settings, obviously. Secondly, we did not recruit a prespecified number of patients for each CDT under investigation. Patients with metastatic breast cancer (TNM stage IV) were for instance underrepresented, and patients with stage IV colorectal cancer and prostate cancer were overrepresented in our study. This might have lowered the perceived guideline adherence. Since this population has a large diversity of disease manifestation, one might expect a more individualized treatment strategy. Another potential limitation is the Hawthorne effect [27]. Being observed could influence the clinicians and this could result in recommendations that agreed to the guideline more strictly. To minimize the Hawthorne effect, the MDT sessions were not recorded. Lastly, because the data collection was performed by a single medical doctor, observer bias may have occurred.

Interpretation within the context of the wider literature

This multicenter study has investigated if innovative methods can support the decision making process in a multidisciplinary setting. Middleton et al. describe in their review the importance of standardized available data and development of knowledge bases for CDS, which are prominently taken into account in our study [15]. Other studies showed that a multitude of requirements must be met for successful implementation of clinical decision support [13, 14]. This study has focused on several of these requirements (e.g. (i) clinicians attitude toward scientific evidence in guidelines, (ii) organizational ethos of transparency and accountability, (iii) understanding of human interaction and workflow implications of CDS and (iv) proprietary implementations with limited interoperability and sharing) and therefore contributes in the further acceptance by clinical community of the health information technology.

Implications for policy, practice and research

The next step toward a successful data-driven healthcare system, especially in multidisciplinary settings, is the implementation and integration of CDSSs into existing clinical processes [28, 29]. This requires (i) the introduction of standardized, structured high-quality reporting by MDTs, including motivation for deviations from guidelines, (ii) integration of CDTs in electronic health records in such a way that it supports clinical workflow and (iii) feedback reporting of real-world treatment recommendations in MDTs to guideline working groups. If these conditions are met, MDTs can be supported real-time for preparing and conducting their MDT meetings for individual patients. On a population level, it can be investigated if MDT decisions deviating from the guideline are attributed to situations where evidence for best practice is low, new evidence outdates the prevailing guideline or unwanted practice variation occurs.

However, the latest guidelines such as the 2020 version of the Dutch breast cancer guidelines stress in each recommendation the value of shared decision-making. Moreover, recommendations are formulated as ‘to consider’, rather than in an imperative way [19]. CDTs can support shared decision-making, since they identify all theoretical possible treatment options. The transparent nature of CDTs enables clinicians and patients to deliberate and judge which treatment option is most suitable.

Conclusion

Increasing knowledge of a myriad of tumor characteristics, internet access and appreciation of patient preferences leads to progressive individualization of choices regarding diagnostics and therapy. This evolution should be recognized, not as a threat, but rather as a continuing challenge for the MDT members and the CDT pathways to provide treatment choices instead of single options.

Acknowledgements

Agnes Jager PhD, medical oncologist, Erasmus MC, Rotterdam; Saskia van der Meer MD PhD, urologist, Jeroen Bosch Hospital, ’s-Hertogenbosch; Martijn van Oijen PhD, clinical epidemiologist, Amsterdam UMC, Amsterdam; Prof. Theo M. de Reijke MD PhD, urologist, Amsterdam UMC, Amsterdam; Ton Roeleveld MD MSc, urologist, Northwest Clinics, Alkmaar; Maurice van der Sangen MD PhD, radiation therapist, Catharina Hospital, Eindhoven; Rik Somford MD PhD, urologist, Canisius Wilhelmina Hospital, Nijmegen; Prof. Cornelis Verhoef MD PhD, oncological surgeon, Erasmus MC, Rotterdam; Jurrian van der Werf PhD PDEng, clinical informatician, Netherlands Comprehensive Cancer Organization, Utrecht all in the Netherlands.

Contributor Information

Kees C W J Ebben, Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands.

Mathijs P Hendriks, Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands; Department of Medical Oncology, Northwest Clinics, Wilhelminalaan 12, Alkmaar 1815JD, The Netherlands; Department of Health Technology and Services Research, Technical Medical Center, University of Twente, Hallenweg 5, Enschede 7522NH, Overijssel, The Netherlands.

Lieke Markus, Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands.

Milan Kos, Department of Medical Oncology, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, Amsterdam 1105AZ, Noord-Holland, The Netherlands.

Ignace H J T De Hingh, Department of Surgical Oncology, Catharina Hospital, Michelangelolaan 2, Eindhoven 5623EJ, The Netherlands.

Jorg R Oddens, Department of Urology, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, Amsterdam 1105AZ, Noord-Holland, The Netherlands.

Joost Rothbarth, Department of Surgical Oncology and Gastrointestinal Surgery, Erasmus MC Cancer Institute, Doctor Molewaterplein 40, Rotterdam 3015GD, The Netherlands.

Hans De wilt, Department of Surgical Oncology, Radboud University Medical Center, Geert Grooteplein Zuid 10, Nijmegen 6525GA, The Netherlands.

Luc J A Strobbe, Department of Surgical Oncology, Canisius Wilhelmina Hospital, Weg door Jonkerbos 100, Nijmegen 6532SZ, The Netherlands.

Maud Bessems, Department of Surgical Oncology, Jeroen Bosch Hospital, Henri Dunantstraat 1, ‘s-Hertogenbosch 5223 GZ, The Netherlands.

Carsten T Mellema, Department of Urology, Spaarne Hospital, Boerhavelaan 22, Haarlem 2035RC, The Netherlands.

Sabine Siesling, Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands; Department of Health Technology and Services Research, Technical Medical Center, University of Twente, Hallenweg 5, Enschede 7522NH, Overijssel, The Netherlands.

Xander A A M Verbeek, Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands.

Funding

This work was not financially supported by any organization.

Contributorship

All authors contributed to this work organizationally, intellectually and in the writing process. The views expressed in the submitted text are the views of the authors.

Ethics and other permissions

The study design was submitted to two independent medical ethics committees and deemed exempt from approval.

Data availability statement

The data underlying this article will be shared on reasonable request to the corresponding author.

Presented elsewhere

Preliminary results of this study were presented as poster at the 2021 HealthRI conference.

Abbreviations

CDT Clinical decision tree

CDSS Clinical decision support systems

GUIDELINE Clinical practice guideline

DCIS Ductal carcinoma in situ

MDT Multidisciplinary team

TNM  Tumor, node, metastasis classification of malignant tumors

References

  • 1. Elsevier . Cancer research current trends & future directions. 2016.
  • 2. Hendriks  MP, Verbeek  XAAM, van Vegchel  T  et al.  Transformation of the national breast cancer guideline into data-driven clinical decision trees. JCO Clin Cancer Inf  2019;3:1–14.doi: 10.1200/CCI.18.00150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Willems  SM, Abeln  S, Feenstra  KA  et al.  The potential use of big data in oncology. Oral Oncol  2019;98:8–12.doi: 10.1016/j.oraloncology.2019.09.003. [DOI] [PubMed] [Google Scholar]
  • 4. Pillay  B, Wootten  AC, Crowe  H  et al.  The impact of multidisciplinary team meetings on patient assessment, management and outcomes in oncology settings: a systematic review of the literature. Cancer Treat Rev  2016;42:56–72.doi: 10.1016/j.ctrv.2015.11.007. [DOI] [PubMed] [Google Scholar]
  • 5. Metcalfe  D, Pitkeathley  C, Herring  J. ‘Advice, not orders’? The evolving legal status of clinical guidelines. J Med Ethics Medethics  2020;47:e78. [DOI] [PubMed] [Google Scholar]
  • 6. Rutgers  EJTH, Nortier  JWR, Tuut  MK  et al.  CBO-richtlijn “behandeling van het mammacarcinoom”. Ned Tijdschr Geneeskd  2002;146:2144–51. [PubMed] [Google Scholar]
  • 7. Patkar  V, Acosta  D, Davidson  T  et al.  Using computerised decision support to improve compliance of cancer multidisciplinary meetings with evidence-based guidance. BMJ Open  2012;2:e000439.doi: 10.1136/bmjopen-2011-000439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Miller  KE, Singh  H, Arnold  R  et al. (eds). Clinical decision-making in complex healthcare delivery systems. In Clinical Engineering Handbook. 2nd edn. Elsevier, 2019, 858–64. [Google Scholar]
  • 9. Institute of Medicine . Clinical Practice Guidelines We Can Trust. Washington D.C: The National Academies Press, 2011. [Google Scholar]
  • 10. Martinez Garcia  L, Sanabria  AJ, Garcia Alvarez  E  et al.  The validity of recommendations from clinical guidelines: a survival analysis. Cmaj  2014;186:1211–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Hendriks  MP, Verbeek  XAAM, van Manen  JG  et al.  Clinical decision trees support systematic evaluation of multidisciplinary team recommendations. Breast Cancer Res Treat  2020;1:3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Keikes  L, Kos  M, Verbeek  XAAM  et al.  Conversion of a colorectal cancer guideline into clinical decision trees with assessment of validity. Int J Qual Heal Care  2021;33:mzab051. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Liberati  EG, Ruggiero  F, Galuppo  L  et al.  What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci  2017;12:113.doi: 10.1186/s13012-017-0644-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Greenes  RA, Bates  DW, Kawamoto  K  et al.  Clinical decision support models and frameworks: seeking to address research issues underlying implementation successes and failures. J Biomed Inform  2018;78:134–43. [DOI] [PubMed] [Google Scholar]
  • 15. Middleton  B, Sittig  DF, Wright  A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb Med Inform  2016;Suppl 1:S103–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Medlock  S, Wyatt  JC, Patel  VL  et al.  Modeling information flows in clinical decision support: key insights for enhancing system effectiveness. J Am Med Inf Assoc  2016;23:1001–6. [DOI] [PubMed] [Google Scholar]
  • 17. Patkar  V, Acosta  D, Davidson  T  et al.  Cancer multidisciplinary team meetings: evidence, challenges, and the role of clinical decision support technology. Int J Breast Cancer  2011;2011:831605.doi: 10.4061/2011/831605. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Ebben  KCWJ, Sieswerda  MS, Luiten  EJT  et al.  Impact on quality of documentation and workload of the introduction of a national information standard for tumor board reporting. JCO Clin Cancer Inf  2019;4:246–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Breast cancer guideline [Internet] . 2018. https://richtlijnendatabase.nl/richtlijn/borstkanker/algemeen.html (7 January 2021, date last accessed).
  • 20. Colorectal cancer guideline [Internet] . 2014. https://richtlijnendatabase.nl/richtlijn/colorectaal_carcinoom_crc/startpagina_-_crc.html (7 January 2021, date last accessed).
  • 21. Prostate cancer guideline [Internet] . 2014. https://richtlijnendatabase.nl/richtlijn/prostaatcarcinoom/prostaatcarcinoom_-_korte_beschrijving.html (7 January 2021, date last accessed).
  • 22. Rokach  L, Maimon  O. Decision trees. In: Maimon O and Rokach L (ed). Data Mining and Knowledge Discovery Handbook. Boston, MA: Springer-Verlag, 2006, 165–92. [Google Scholar]
  • 23. Rosell  L, Alexandersson  N, Hagberg  O  et al.  Benefits, barriers and opinions on multidisciplinary team meetings: a survey in Swedish cancer care. BMC Health Serv Res  2018;18:249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Bouaud  J, Blaszka-Jaulerry  B, Zelek  L  et al.  Health information technology: use it well, or don’t! Findings from the use of a decision support system for breast cancer management. AMIA Annu Symp Proc  2014;eCollection:315–24. [PMC free article] [PubMed] [Google Scholar]
  • 25. Séroussi  B, Bouaud  J, Gligorov  J  et al.  Supporting multidisciplinary staff meetings for guideline-based breast cancer management: a study with OncoDoc2. AMIA Annu Symp Proc  2007;656–60. [PMC free article] [PubMed] [Google Scholar]
  • 26. Séroussi  B, Soulet  A, Spano  JP  et al.  Which patients may benefit from the use of a decision support system to improve compliance of physician decisions with clinical practice guidelines: a case study with breast cancer involving data mining. Stud Health Technol Inform  2013;192:534–8. [PubMed] [Google Scholar]
  • 27. Mccambridge  J, Witton  J, Elbourne  DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol  2014;67:267–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Cohan  A, Soldaini  L, Yates  A  et al.  On clinical decision support, in ACM BCB 2014-5th ACM conference on bioinformatics. In: Bernar ES and La Lande TJ (ed). Computational Biology, and Health Informatics. Newport Beach California, 2014. [Google Scholar]
  • 29. Berner  ES, La Lande  TJ. Overview of clinical decision support systems. New York: Springer-Verlag, 2016,3–22. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data underlying this article will be shared on reasonable request to the corresponding author.


Articles from International Journal for Quality in Health Care are provided here courtesy of Oxford University Press

RESOURCES