Skip to main content
NCBI home page
AMIA Annual Symposium Proceedings logoLink to AMIA Annual Symposium Proceedings
. 2018 Apr 16;2017:1527–1536.

Reconciliation of multiple guidelines for decision support: a case study on the multidisciplinary management of breast cancer within the DESIREE project*

Brigitte Séroussi 1,2,3, Gilles Guézennec 1, Jean-Baptiste Lamy 1, Naiara Muro 1,4,5, Nekane Larburu 4,5, Booma Devi Sekar 6, Coralie Prebet 7, Jacques Bouaud 1,8
PMCID: PMC5977621  PMID: 29854222

Abstract

Breast cancer is the most common cancer among women. DESIREE is a European project which aims at developing web-based services for the management of primary breast cancer by multidisciplinary breast units (BUs). We describe the guideline-based decision support system (GL-DSS) of the project. Various breast cancer clinical practice guidelines (CPGs) have been selected to be concurrently applied to provide state-of-the-art patient-specific recommendations. The aim is to reconcile CPG recommendations with the objective of complementarity to enlarge the number of clinical situations covered by the GL-DSS. Input and output data exchange with the GL-DSS is performed using FHIR. We used a knowledge model of the domain as an ontology on which relies the reasoning process performed by rules that encode the selected CPGs. Semantic web tools were used, notably the Euler/EYE inference engine, to implement the GL-DSS. “Rainbow boxes” are a synthetic tabular display used to visualize the inferred recommendations.

Introduction

Breast cancer is the most common cancer among women in France with about 50,000 new cases per year. After having doubled between 1985 and 2005, the incidence rate of breast cancer seems now to be stabilized. However, if it is decreasing for women aged 50-79 that benefit from the nationally organized screening, it has increased of more than 60% for the women aged 30-39 and 40-491. With 12,000 deaths per year (figures consolidated in 2012), the mortality rate of breast cancer in France is decreasing. Breast cancer is one of the cancers with the best survival rate at five and 10 years (87%, resp. 76%). This may be explained by the evolution of therapeutics and the development of endocrine and targeted agents as well as to the reduction of menopausal hormone therapy prescription. However, breast cancer comprises a complex and heterogeneous group of diseases at the clinical, morphological, and molecular levels. For some of the subtypes, especially the triple-negative and HER2+ breast cancers, margins for improvement are both possible and necessary2.

Clinical practice guidelines (CPGs) are elaborated to provide best evidence-based recommendations for the management of patients with specific conditions. Studies showed that implementing oncology CPGs does improve clinical outcomes in both overall and recurrence-free survivals3. This is especially true in the case of breast cancer4. However, despite the publication of CPGs and the provision of state-of-the-art recommendations, cancer management remains subject to variable practices, and poor levels of compliance with oncology CPGs are observed5, 6. For instance, Wöckel et al. reported a 51.9% guideline adherence rate for the complete treatment received by primary breast cancer patients6.

In the last decade, organizational measures such as multidisciplinary tumor boards (MTBs) have been introduced to promote quality in care delivery to cancer patients7. The aim is to gather the various cancer specialists (surgeons, medical oncologists, radiologists, pathologists, radiotherapists, etc.) to promote the collective discussion of cancer patient clinical cases and the decision of the best care plan for the patients. MTBs are expected to improve CPG implementation and to help capture cases for clinical trials. If studies on cancer care generally associate multidisciplinary tumor boards with improvements of guideline compliance rates8–10, daily practice in MTBs is hampered by the complexity of the disease and the vast amount of patient and disease data available. Implementing CPGs is difficult since a high case load is actually discussed in MTBs and individual cases usually only receive a very limited amount of time for review. Thus, the impact of multidisciplinary tumor boards has been questioned11, suggesting that decision support tools could improve MTB efficiency.

Indeed, CPGs are usually developed by health professional societies and national health agencies as textual documents in a narrative format. The sole dissemination of narrative guidelines poorly helps CPG implementation and has almost no impact on physician decisions12. On the contrary, embedding CPGs in clinical decision support systems (CDSSs) has shown some benefit on the improvement of physician decision compliance with CPGs13, 14. Different prototypes of CDSSs supporting the management of breast cancer patients have been developed. However to the authors’ knowledge, only few systems have been routinely used to support decisions made for actual breast cancer patients in real life breast MTBs or breast units (BUs), and only some of them evidenced they were improving compliance of BU decisions with CPGs. Among them, we can mention MATE15 which uses the PROforma language, OncoCure16 based on Asbruencoded protocols of pharmacological therapies for breast cancer, and OncoDoc17, 18 that offers to navigate through a knowledge base structured as a decision tree to obtain patient-specific recommendations. Different formalisms have been proposed to translate the narrative recommendations into computer-interpretable guidelines19. More recently, research works using web semantic approaches have been conducted, e.g. Abidi et al. proposed to use ontologies in a rule-based reasoning process to manage breast cancer patients20.

However, all previous propositions only rely on the modeling and implementation of a single CPG applied to a unique pathology, in this case breast cancer. Other research works have developed solutions to handle the concurrent application of different CPGs in order to manage patients with comorbidities that need different treatment regimens. Wilk et al.21 have proposed a framework based on first order logic to represent CPGs and to mitigate possible adverse interactions (drug-drug or drug-disease) between the recommendations provided by the different CPGs. Galopin et al.22 have implemented an ontological reasoning process to allow for the flexibility necessary to deal with patients suffering from both hypertension and type 2 diabetes. In these cases, CPGs are reconcile on the basis of competition.

DESIREE is a European-funded project† which aims at developing a web-based software ecosystem for the personalized, collaborative, and multidisciplinary management of primary breast cancer by BUs. DESIREE would offer guideline-based, case-based, and experience-based23 decision support. The system is expected to be used by the clinical partners of the consortium on actual breast cancer patients and in real life BUs. In this perspective, we have translated the different contemporary breast cancer CPGs that clinical partners are implementing in their BUs in a computer-interpretable format. The aim is then to reconcile breast cancer CPGs on the basis of complementarity and to concurrently apply all of them. The idea is to avoid the CDSS silence observed when no recommendation is issued for a clinical case not covered by single CPGs, and to extend the clinical coverage of the CDSS by considering additional breast cancer CPGs.

In this paper, we present the guideline-based decision support system (GL-DSS) of DESIREE and describe how we used a domain ontology as the conceptual and terminological structure on which relies the reasoning process performed from the rules that model the selected breast cancer CPGs. We also propose an original visualization of the output of the concurrent application of CPGs as “rainbow boxes”24.

Material and methods

General design of the GL-DSS and basic workflow

The aim of the GL-DSS of DESIREE is to provide state-of-the-art guideline-based recommendations structured as action plans issued from multiple breast cancer CPGs for any clinical case discussed by BUs. Informed BU participants may then choose to comply of not with one of the options provided. Since patient clinical cases discussed during BUs may be more or less complete, and because clinical data and reasoning process may be described at various levels of abstraction, we chose to use semantic web tools allowing for subsumption. A data model and a knowledge model dedicated to breast cancer management were consistently developed. The data model relies on a standard entityrelationship model.The Breast Cancer Knowledge Model (BCKM) is a specification of all concepts that have been identified for the management of breast cancer patients, structured as an OWL ontology. It enables the multi-level description of patient cases and recommended action plans. The BCKM is a central static resource shared by all data-oriented modules and components of DESIREE, so that they can be semantically interoperable.

To elaborate the knowledge base of the GL-DSS, the first step of the method was to select the different breast cancer CPGs to be used. Once validated by the clinical partners of the DESIREE consortium, each of the considered CPGs has been first structured as a set of human-readable decision rules, then encoded into a formal model-driven rule language (NRL). Each formalized CPG was then processed in order to check its consistency regarding the BCKM and to generate an internal computer-interpretable representation of the rules in the N325 notation‡

In the DESIREE project, a patient data repository, part of the DESIREE Information Management System (DESIMS), constitutes a local breast-cancer-specific Electronic Medical Record (EMR) to store all patient cases according to the common data model. In order to be interoperable beyond the DESIREE system, we decided to align both the knowledge and the data models with standard reference terminologies to address semantic interoperability, and we chose the FHIR messaging standard for communicating with external components to warranty syntactic interoperability26.

Finally, we expected the GL-DSS to be used in real life BUs for actual patients. Thus, we needed to demonstrate the good performance of the system and chose to use a fast and powerful rule based inference engine (Euler/EYE27). The GL-DSS module is implemented as a servlet providing web services through the use of a REST API. Figure 1 illustrates the different internal components of the GL-DSS, the external resources, and data flows. At run time, the GL-DSS queries a FHIR server, linked to DESIMS, for patient data. Then, it converts the received bundle of FHIR resources into an internal N3 representation. On the basis of N3 rules and of the N3 version of the BCKM, the inference engine produces new facts, including the set of recommended action plans, which are returned to the user.

Figure 1:

Figure 1:

Open in a new tab

Guideline-Based Decision Support System - General design and workflow.

Clinical practice guideline collection and structuration

Various CPGs have been published for the management of breast cancer both at the international and national levels. Our goal was to identify up-to-date CPGs that would be implemented within the GL-DSS module of DESIREE. The selection criteria were: primary breast cancer; no restriction on the management step (including diagnosis, therapy, and follow-up); published in English, or in the languages of the project’s clinical partners (Spanish and French); most recent version when several versions existed. We conducted a search through different on-line resources: (i) biomedical literature, using PubMed, (ii) international and national web repositories of CPGs including governmental health care agencies (e.g. NGC, G-I-N, NCI…)§, (iii) websites of professional societies and associations, health care institutions (e.g. NCCN, ESMO, ASCO, SEOM…), and (iv) custom local CPGs used as references by clinical partners’ BUs.

Eight CPGs were finally validated by the clinical partners of DESIREE and chosen for implementation within the project: two NCCN guidelines Version 2.2016 “Breast Cancer” and “Genetic/Familial High-Risk Assessment: Breast and Ovarian” [English]; the ESMO guidelines published in Annals of Oncology published in 2015 [English]; the SEOM clinical guidelines for early-stage breast cancer published in 2015 [English]; three CPGs from the Paris public hospitals (AP-HP), guidelines for “breast cancer management” (2016), for “fertility preservation” (2016), and for the “management of cancer and pregnancy” (2014) [French]; the Onkologikoa’s breast cancer guidelines 2016 [Spanish].

Each CPGs has been structured by a medical oncologist specialized in the management of breast cancer as humanreadable IF-THEN decision rules. The rules built have been double-checked by a public health physician. Then, they have been normalized to comply with the domain concepts defined by the breast cancer knowledge model.

The Breast Cancer Knowledge Model

The purpose of the knowledge model is to provide a common reference and terminology for all modules and components with the guaranty that clear, unique, and accredited concepts and definitions are being used. All the relevant concepts used for a patient case description and for the reasoning process are stored in a single place as an ontology in the OWL format. The choice to define a specific terminology focusing on the breast cancer domain was mainly made in order to be able to easily add new concepts. Nevertheless, the model remains tightly coupled with some existing established authoritative terminologies such as the NCI thesaurus, LOINC, and SNOMED CT, to further warranty semantic interoperability with potential hospital information systems (HISs) or EMRs. Alignment is maintained through links whenever it is possible by means of systematic annotations (see Figure 2). All the relevant clinical procedures, possible examinations, clinical findings, and observations are present in the model.

Figure 2:

Figure 2:

Open in a new tab

Example of a reference concept and its link with standard terminologies.

The ontology also accounts for the breast cancer data model which includes the main concepts such as Patient, Side, and Lesion which are the main entities characterized by series of parameters or attributes, and allows for the description of a breast cancer patient case. Consequently, from the decision support point of view, the BCKM ontology is divided into two main hierarchies. The first one contains all the concepts specific to the DESIREE environment, namely the concepts used to define the entity parameters, and the second one contains all reference concepts defined in the breast cancer domain. The reference concepts are mainly used as potential values for the former DESIREE concepts.

Decision rules formats

We followed the approach promoted by the HL7 CDS group on the notion of “knowledge artifact”28 to represent rule sets. Rules are first encoded in a formal language, human readable, and possibly editable by some trained clinicians. We used NRL29 which is a model-driven language allowing to write logical expressions matching model components and rules, independently of any implemented inference engine. In our approach, the BCKM ontology acts as the reference model; an XSD model is automatically derived from the OWL model to feed the NRL parser that validates rule specifications. NRL has already been used to manipulate clinical information models30 and is similar to the GELLO language. NRL rule sets are then transformed into an XML representation, depending on the model, but independent from the source format (NRL) and the target computer-interpretable format. This intermediate representation is close to the HL7 Clinical Decision Support Knowledge Artifact Specification28 and is used to be shared by other modules (e.g., the experience-based DSS23). Another transformation yields an N3 representation25 of the NRL rules which is the target computer-interpretable format.

Data representations

Patient-related data is stored outside the GL-DSS, within the DESIMS component. Input and output data exchange with the GL-DSS is performed using FHIR. No specific FHIR profile was defined and only standard FHIR resources were used, mainly Patient, Observation, Condition, BodySite, and Specimen. In order to preserve the data model through the FHIR transport, the main assumption has been that the observation resource was used to convey parametervalue pairs of the model. DESIREE codes defined in the DESIREE namespace of the knowledge model are used to particularize these observations and make them describe the same concepts as those used by the GL-DSS. Once parsed and decoded from the FHIR messages, patient data are converted to N3 notation25 and the data model is instantiated as a set of triples. This N3 representation is matched with the N3 executable rules by the inference engine at run time.

Execution engine

To combine rule-based reasoning and ontological reasoning, we adopted Euler, more specifically the EYE implementation by De Roo et al31 This provides the system with description logic (DL) reasoning facility along with some classical logic powerful features. Euler is notably using Notation3 (N3), it is directly interoperable with the knowledge model since it belongs to the same family of semantic web tools able to deal with graphs of triples. Moreover, it appeared to be among the fastest reasoners with a full OWL-DL27, 32.

In the GL-DSS module, the engine is fed with a set of N3 rules and a set of N3 facts produced from the patient data according to the knowledge model concepts and properties. It takes the full knowledge model as additional input and using a generic query it then saturates the knowledge base until no more new triple can be produced generating new inferred facts. The content of the triple store is then explored in order to build new recommendations that have been produced with the basic concepts and building primitives from the knowledge model.

Visualization of recommendations

In order to display the multiple recommendations that may be produced by the different CPGs, we adapted the rainbow boxes visualization technique designed by Lamy et al.24 for the comparison of drug properties. It performs overlapping set visualization, a visualization problem that considers elements and the sets made out of these elements. The objective of the visualization is to clearly display which elements a given set includes, and which sets a given element belongs to, to facilitate the discovery of new knowledge such as similarities between elements or sets.

Rainbow boxes represents the elements in columns and each set is displayed as a rectangular labeled box that covers the columns associated with the elements belonging to the set. The boxes are stacked vertically as pieces in a Tetris game, and ordered by size (largest ones at the bottom). “Holes” can occur in a box when the elements belonging to a given set are not in consecutive columns. However, columns are ordered using a heuristic algorithm that minimizes the number of holes.

The presentation of multiple recommendations can be seen as an overlapping set visualization problem where each recommendation is a set of therapeutic units. Thus, in this context, a set is a recommendation and an element is a therapeutic unit. Therapies are grouped in six categories (pre-operative chemotherapy, pre-operative endocrine therapy, surgery, post-operative chemotherapy, radiotherapy, post-operative endocrine therapy) displayed from left to right, and each of the four types of therapies (surgery, chemotherapy, radiotherapy, endocrine therapy) were distinguished by color hues. In addition, some recommendations have a temporal dimension, e.g. chemotherapy with Adriamycin- Cyclophosphamide followed by Docetaxel. Color saturation is used to emphasize the first step of the recommendation.

A case study

As an illustration of how the GL-DSS operates, we present a case study processed with two of the breast cancer management CPGs that were selected in the DESIREE project, namely the 2016 version of NCCN CPGs “Breast cancer” and the 2016 version of AP-HP CPGs “Breast cancer management”. Both guidelines have been formalized as human readable IF-THEN rules by an oncologist specialized in breast cancer.

We consider the case of a patient, aged 67, diagnosed with an invasive bifocal breast cancer of the lower outer quadrant of the left breast. The first lesion is 35 mm, the second 12 mm, and the distance between the two is 18 mm. There is no clinical axillary lymph node (cN0). The two lesions have the same pathological profile: Estrogen receptors (ER)=95%, Progesterone receptors (PR)=40%, HER2-, SBR 3, and KI67=25%. The patient has no contra-indication to chemotherapy. Patient data is transmitted using a FHIR message to the GL-DSS and translated into the N3 format. Some preliminary inference rules are triggered to enrich the patient profile and assess the postmenopausal status, the TNM staging as T3 N0 M0, the stage as IIa, and the positive hormonal receptor status (HR). Figure 3 displays an excerpt of NCCN and AP-HP rules triggered on this patient case. NCCN guidelines recommend two management strategies, either surgery (lumpectomy and sentinel node excision or mastectomy and sentinel node excision) or systemic therapy: endocrine therapy with aromatase inhibitors or chemotherapy with the proposition of two different protocols (AC followed by Paclitaxel and TC). In the same way, AP-HP guidelines propose both surgery and systemic therapy (see Figure 4). However, both surgeries and systemic therapies are different: AP-HP CPGs recommend a surgery by mastectomy with axillary dissection, there is no recommendation of endocrine therapy, and if some of the chemotherapies are those recommended by NCCN CPGs, there are two additional protocols.

Figure 3:

Figure 3:

Open in a new tab

Excerpt of NCCN and AP-HP rules triggered by the patient data.

Figure 4.

Figure 4.

Open in a new tab

Recommendations inferred by NCCN and AP-HP guidelines on the patient clinical case.

The rainbow boxes visualization of the case study is displayed in Figure 5. The box label indicates the guideline(s) leading to the recommendation. When both CPGs produce the same recommendation, there are two labels, and thus the box is larger. Consequently, it is easy to determine at a glance the most consensual recommendations. Similarly, the use of color makes it easy to see that most recommendations involve chemotherapy. It is also noticeable that all these chemotherapies include Cyclophosphamide.

Figure 5.

Figure 5.

Open in a new tab

Rainbow boxes visualizing the eight recommendations obtained for the case study. Therapies are presented in columns and there is one recommendation per rectangular box (not one per line). One box includes a hole in the Docetaxel column. Colors identify the four categories of therapies (green, yellow, red, blue), the temporal dimension (the vivid part of a box corresponds to the first step, while the dimmed part corresponds to the other steps if any), and the guidelines (black and white labels).

Discussion

We have developed the GL-DSS module of the DESIREE project applied to the management of breast cancer patients. The aim was to process the reconciliation of contemporary CPGs developed on a given pathology on the basis of complementarity to extend the coverage of patient profiles for which the GL-DSS provides recommendations. Although BU clinicians would have the opportunity to tick the CPGs they want to select to feed the GL-DSS, they may chose to work with all of them to reduce the silence of the system or see when different CPGs are consistent.

The choice of developing the knowledge model as an ontology presents two major advantages on top of providing a unique vocabulary to all the DESIREE project components: on the one side, the model resolves multilingual issues through label language tags, which is strongly required in an international project and, on the second side, the inference engine benefits from the ontological reasoning capabilities of the ontological model in addition to those provided by the rule engine. For instance, the use of the subsumption relationship leads in some cases to decrease the number of rules that have to be written and permits to match parameter values expressed at various levels of abstraction. In the same way, the issued recommendations may have different levels of specialization depending on the granularity of the information contained in CPGs for a particular patient case.

Different approaches have been developed to represent and execute CPGs19. However, in the case of breast cancer management, the choice of IF-THEN rules is appropriate to represent CPG contents since patient clinical data are easily formalized and recommended actions are given as care plans described at a high level of abstraction (to be refined and implemented by care providers outside the BU. In the same way, the temporal nature of CPG recommendations is easily represented by the ordered sequence of the therapeutic steps that compose the recommended care plan. We could have chosen the HL7 Clinical Decision Support Knowledge Artifact Specification to define Condition-Action rules but we considered the standard was not stable enough to be used in an international project stressed on the production of outputs and outcome measures. In addition, a end-user formal language would still have been required, and we preferred to use the NRL language which accepts any declared information model like our BCKM, instead of using existing dedicated formalisms (like GELLO or Arden Syntax). IF-THEN rules have been manually built which sets the issue of the scalability of the approach. If natural language processing (NLP) methods to semi-automatically extract IF and THEN parts of rules have been proposed33, the result is not yet satisfactory and solutions for the development of structured CPG contents should rely on the initial authoring of CPGs in these formats.

As for the execution engine, using tools from the semantic web domain as a base should have led to some kind of solution based on SWRL rules associated to a classical OWL reasoning engine to produce inferences. However, because of the lack of expressiveness of the latter, the impossibility to deal with non monotonicity and negation within the open world assumption and the degraded performances obtained on a real scale with these techniques, we preferred to adopt an alternative solution with Euler/Eye which does not have these limitations. When running the GL-DSS, the different CPGs are operated, which may lead to intra and inter-CPG conflicting recommendations. Only intra-CPG conflicts are resolved in the way proposed by34. The resolution of inter-CPG conflicts are left to BU participants.

Currently, only two CPGs have been implemented, and work is still in progress with the other CPGs. It’s only when all CPG rule bases would be developed that we will be able to assess the performance of the semantic reasoner Euler/EYE and validate the choices we made. In addition, further evaluation is needed to assess the use of rainbow boxes to visualize the set of concurrent recommendations provided by the different CPGs.

Acknowledgments

The authors would like to thank all the partners of the project, especially the clinicians from Onkologikoa, Ander Urruticoechea Ribate, Jon Belloso Linacisoro, and Gerardo Cajaraville Ordoñana, as well as the participating members of Bilbomatica, among which Amaia Ugarriza Pinedo, Ander Elzo Zarraga, and Roberto Álvarez Sánchez.

Footnotes

*

The DESIREE project has received funding from the European Union’s Horizon 2020 research and innovation programmme under grant agreement No 690238.

‡

N3 is a compact and human readable syntax for RDF serialization but it has several features that go beyond a serialization for RDF models, such as support for RDF-based rules. It currently has the status of a W3C submission whose specifications dated back to 2011 can be found at http://www.w3.org/TeamSubmission/2011/SUBM-n3-20110328/

§

NGC = The National Guideline Clearinghouse (http://www.guideline.gov), G-I-N = Guideline International Network (http://www.g-i-n.net), NCI = National Cancer Institute (http://www.cancer.gov), NCCN = National Comprehensive Cancer Network (http://www.nccn.org), ESMO = European Society for Medical Oncology (http://www.esmo.org), ASCO = American Society of Clinical Oncology (http://www.asco.org), SEOM = Spanish Society of Medical Oncology (http://www.seom.org).

References

  • 1.Institut national du cancer, France. Collection “les données” 2016 edition. 2017. Apr, [Accessed July 6, 2017]. http://www.e-cancer.fr/ressources/cancers_en_france/#page=65.
  • 2.Elsamany S, Abdullah S. Triple-negative breast cancer: future prospects in diagnosis and management. Med Oncol. 2014;31(2):834. doi: 10.1007/s12032-013-0834-y. [DOI] [PubMed] [Google Scholar]
  • 3.Hebert-Croteau N, Brisson J, Latreille J, Rivard M, Abdelaziz N, Martin G. Compliance with consensus recommendations for systemic therapy is associated with improved survival of women with node-negative breast cancer. J Clin Oncol. 2004;22(18):3685–93. doi: 10.1200/JCO.2004.07.018. [DOI] [PubMed] [Google Scholar]
  • 4.Schwentner L, Wolters R, Koretz K, Wischnewsky MB, Kreienberg R, Rottscholl R, et al. Triple-negative breast cancer: the impact of guideline-adherent adjuvant treatment on survival-a retrospective multi-centre cohort study. Breast Cancer Res Treat. 2012;132(3):1073–80. doi: 10.1007/s10549-011-1935-y. [DOI] [PubMed] [Google Scholar]
  • 5.Lebeau M, Mathoulin-Pelissier S, Bellera C, Tunon-de Lara C, Daban A, Lipinski F, et al. Breast cancer care compared with clinical guidelines: an observational study in France. BMC Public Health. 2011;11:45. doi: 10.1186/1471-2458-11-45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Wockel A, Kurzeder C, Geyer V, Novasphenny I, Wolters R, Wischnewsky M, et al. Effects of guideline adherence in primary breast cancer-a 5-year multi-center cohort study of 3976 patients. Breast. 2010;19(2):120–7. doi: 10.1016/j.breast.2009.12.006. [DOI] [PubMed] [Google Scholar]
  • 7.Patkar V, Acosta D, Davidson T, Jones A, Fox J, Keshtgar M. Cancer multidisciplinary team meetings: evidence, challenges, and the role of clinical decision support technology. Int J Breast Cancer. 2011;2011:831605. doi: 10.4061/2011/831605. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Kesson EM, Allardice GM, George WD, Burns HJ, Morrison DS. Effects of multidisciplinary team working on breast cancer survival: retrospective, comparative, interventional cohort study of 13 722 women. BMJ. 2012;344:e2718. doi: 10.1136/bmj.e2718. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.van Hoeve J, de Munck L, Otter R, de Vries J, Siesling S. Quality improvement by implementing an integrated oncological care pathway for breast cancer patients. Breast. 2014;23(4):364–70. doi: 10.1016/j.breast.2014.01.008. [DOI] [PubMed] [Google Scholar]
  • 10.Brar S S, Hong N L WFC. Multidisciplinary cancer care: Does it improve outcomes? J Surg Oncol. 2014;110(5):494–9. doi: 10.1002/jso.23700. [DOI] [PubMed] [Google Scholar]
  • 11.Johnson CE, Slavova-Azmanova N, Saunders C. Development of a peer review framework for cancer multidisciplinary meetings. Intern Med J. 2017 doi: 10.1111/imj.13374. [Epub ahead of print] [DOI] [PubMed] [Google Scholar]
  • 12.Giguère A, Légaré F, Grimshaw J, Turcotte S, Fiander M, Grudniewicz A, et al. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;10:CD004398. doi: 10.1002/14651858.CD004398.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18(3):327–34. doi: 10.1136/amiajnl-2011-000094. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Roshanov PS, Misra S, Gerstein HC, Garg AX, Sebaldt RJ, Mackay JA, et al. Computerized clinical decision support systems for chronic disease management: a decision-maker-researcher partnership systematic review. Implement Sci. 2011;6:92. doi: 10.1186/1748-5908-6-92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Patkar V, Acosta D, Davidson T, Jones A, Fox J, Keshtgar M. Using computerised decision support to improve compliance of cancer multidisciplinary meetings with evidence-based guidance. BMJ Open. 2012;2(3):e000439. doi: 10.1136/bmjopen-2011-000439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Eccher C, Seyfang A, Ferro A. Implementation and evaluation of an Asbru-based decision support system for adjuvant treatment in breast cancer. Comput Methods Programs Biomed. 2014;117(2):308–21. doi: 10.1016/j.cmpb.2014.06.021. [DOI] [PubMed] [Google Scholar]
  • 17.Séroussi B, Bouaud J, Antoine ÉC. OncoDoc, a successful experiment of computer-supported guideline development and implementation in the treatment of breast cancer. Artif Intell Med. 2001;22(1):43–64. doi: 10.1016/s0933-3657(00)00099-3. [DOI] [PubMed] [Google Scholar]
  • 18.Séroussi B, Laouénan C, Gligorov J, Uzan S, Mentré F, Bouaud J. Which breast cancer decisions remain noncompliant with guidelines despite the use of computerized decision support? Br J Cancer. 2013;109(5):1147–56. doi: 10.1038/bjc.2013.453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Peleg M. Computer-interpretable clinical guidelines: a methodological review. J Biomed Inform. 2013;46(4):744–63. doi: 10.1016/j.jbi.2013.06.009. [DOI] [PubMed] [Google Scholar]
  • 20.Abidi SR, Abidi SS, Hussain S, Shepherd M. Ontology-based modeling of clinical practice guidelines: a clinical decision support system for breast cancer follow-up interventions at primary care settings. Stud Health Technol Inform. 2007;129(Pt 2):845–9. [PubMed] [Google Scholar]
  • 21.Wilk S, Michalowski W, Michalowski M, Farion K, Hing MM, Mohapatra S. Mitigation of adverse interactions in pairs of clinical practice guidelines using constraint logic programming. J Biomed Inform. 2013;46(2):341–53. doi: 10.1016/j.jbi.2013.01.002. [DOI] [PubMed] [Google Scholar]
  • 22.Galopin A, Bouaud J, Pereira S, Seroussi B. An ontology-based clinical decision support system for the management of patients with multiple chronic disorders. Stud Health Technol Inform. 2015;216:275–9. [PubMed] [Google Scholar]
  • 23.Larburu N, Muro N, Macia I, Sanchez E, Wang H, Winder J, et al. Augmenting guideline-based cdss with experts’ knowledge; In: 10th International Conference on Health Informatics; February 21-23; Porto, Portugal. 2017. [Google Scholar]
  • 24.Lamy JB, Berthelot H, Favre M. Rainbow boxes: a technique for visualizing overlapping sets and an application to the comparison of drugs properties; In: 20th International Conference Information Visualisation; Lisboa, Portugal. 2016. pp. 253–60. [Google Scholar]
  • 25.Berners-Lee T, Connolly D, Kagal L, Scharf Y, Hendler J. N3Logic: A logical framework for the World Wide Web. Theory and Practice of Logic Programming. 2008;8(3) [Google Scholar]
  • 26. HL7.org. FHIR STU3 Candidate (v1.8.0-10521) 2016. Dec, [Accessed July 6, 2017]. http://hl7.org/fhir/STU3/
  • 27.Verborgh R, De Roo J. Drawing conclusions from linked data on the web: The EYE reasoner. IEEE Software. 2015;32(3):23–7. [Google Scholar]
  • 28. HL7.org. HL7 standard: Clinical decision support knowledge artifact specification, release 1 DSTU release 1.3. 2015. Jul, [Accessed July 6, 2017]. http://www.hl7.org/implement/standards/product_brief.cfm?product_id=337.
  • 29.Natural rule language (nrl) specification 1.4.0. [Accessed July 6, 2017]. http://nrl.sourceforge.net/spec/
  • 30.Boscá D, Maldonado JA, Moner D, Robles M. Automatic generation of computable implementation guides from clinical information models. J Biomed Inform. 2015;55(2):143–52. doi: 10.1016/j.jbi.2015.04.002. [DOI] [PubMed] [Google Scholar]
  • 31.Eiter T, Ianni G, Polleres A, Schindlauer R, Tompits H. Reasoning with rules and ontologies. Reasoning Web. 2006 [Google Scholar]
  • 32.Tiberghien T, Aloulou H, Moktari M, Biswas J. Semantic reasoning in context-aware assistive environments to support ageing with dementia. ISWC. 2012:212–27. [Google Scholar]
  • 33.Kaiser K, Miksch S. Formalizing ‘living guidelines’ using LASSIE: A multi-step information extraction method. In: Bellazzi R, Abu-Hanna A, Hunter J, editors. AIME 2007, (vol4594) of Lecture Notes in Computer Science. Amsterdam, The Netherlands: Springer; 2007. pp. 401–10. [Google Scholar]
  • 34.Zamborlini V, Hoekstra R, Da Silveira M, Pruski C, ten Teije A, van Harmelen F. Inferring recommendation interactions in clinical guidelines. Semantic Web. 2016;7(4):421–46. [Google Scholar]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association

RESOURCES