Use of mind maps and iterative decision trees to develop a guideline-based clinical decision support system for routine surgical practice: case study in thyroid nodules

Abstract

Objective

The study sought to develop a clinical decision support system (CDSS) for the treatment of thyroid nodules, using a mind map and iterative decision tree (IDT) approach to the integration of clinical practice guidelines (CPGs).

Materials and Methods

Thyroid nodule CPGs of the American Thyroid Association and Korean Thyroid Association were analyzed by endocrine surgeons (domain experts) and computer scientists. Clinical knowledge from the CPGs was expressed using mind maps. The mind maps were analyzed and converted into IDTs. The final IDT was implemented as a set of candidate rules (3700) for a knowledge-based CDSS. The system was evaluated via a retrospective review of the medical records of 483 patients who had undergone thyroidectomy between January and December 2015 at a single tertiary center (Seoul National University Hospital Bundang, Korea).

Results

Concordance between CDSS recommendations and treatment in routine clinical practice was 78.9%. In the 21.1% discordant cases, deviation from the CDSS treatment recommendation was mainly attributable to (1) refusal of the patient to undergo total thyroidectomy and (2) conversion from lobectomy to total thyroidectomy following an unexpected histological finding during intraoperative frozen biopsy lymph node analysis.

Conclusions

The present study demonstrated that a knowledge-based CDSS is feasible in the treatment of thyroid nodules. A high-quality knowledge-based CDSS was developed, and medical domain and computer scientists collaborated effectively in an integrated development environment. The mind map and IDT approach represents a pioneering method of integrating knowledge from CPGs.

BACKGROUND AND SIGNIFICANCE

A clinical decision support system (CDSS) is a health information technology that supports clinical decision making by physicians and other healthcare professionals.¹ Two types of CDSS are available: knowledge based and non–knowledge based. A knowledge-based CDSS involves correlations and rules for the analysis of accumulated data, and is mainly constructed using the logic of IF-THEN.² These rules are clearly labeled and classified as transparent algorithms. At the time of writing, most hospital-based CDSS are knowledge based, as transparent algorithms are preferred in the medical domain. By contrast, non–knowledge-based CDSS are non–rules based. In this system, artificial intelligence facilitates decision making by learning patterns identified within patient histories or clinical information.³ A non–knowledge-based CDSS is classified as an opaque algorithm, as it is difficult to confirm the process and reasoning applied in reaching the respective conclusion. A typical non–knowledge-based CDSS is the IBM platform Watson. This was first marketed in 2013 and has since become popular in Korea. However, no Watson for Oncology (WFO) results are yet available, and abstracts are limited to those presented at meetings of the American Society of Clinical Oncology (ASCO). According to the Gil Hospital of South Korea abstract from the 2017 ASCO meeting, the concordance rate for WFO and physicians varies according to the disease, and concordance for a given disorder varies depending on the disease-stage.⁴ Thus, the role of the knowledge-based CDSS is emphasized reflexively. Moreover, knowledge-based CDSS are considered white-box systems, whereas non–knowledge-based CDSS are known as black-box systems.

In theory, knowledge-based CDSS can be applied to diagnosis, treatment, prevention, and the evaluation of prognosis.⁵ In general, knowledge-based CDSS reduces errors in the prescribing of medication, which represents a key treatment strategy in most hospitals.⁶ However, knowledge-based CDSS has also been applied to facilitate clinical diagnosis,⁷ chronic disease management,⁸ and preventive care.⁹ Compared with the introduction of CDSS for healthcare process measures, few CDSS have been developed and introduced for diagnosis and treatment.⁵

In the field of surgery, one potential area of use for knowledge-based CDSS is the treatment of thyroid nodules, as treatment for this common disorder is planned on the basis of diverse text results. Although previous authors have attempted to introduce knowledge based CDSS for thyroid nodules, no study to date has analyzed patient data, and to our knowledge, no previous study has developed a knowledge-based CDSS¹⁰ for a thyroid disease with concrete recommendation outcomes. The aim of the present study was to develop a knowledge-based CDSS for the treatment of thyroid nodules. To achieve this, a formal acquisition method was developed to enable medical experts and computer scientists to transform knowledge resources—clinical practice guidelines (CPGs) for thyroid nodules and domain expert heuristics into an executable knowledge model that was understandable for all stakeholders. This involved the use of a pioneering mind map and iterative decision tree (IDT) approach. To determine concordance with treatment prescribed in the real-world setting, the knowledge-based CDSS was then applied to retrospective clinical data from a single tertiary center.

To model clinical knowledge from CPGs in a manner that was executable and understandable for both medical experts and computer scientists, a mind map and IDT approach was used. Previous authors have reported the use of mind maps in medical education.^11–13 However, in a pioneering approach, we first used mind maps to model clinical knowledge and then converted these mind maps into modified traditional DTs. These modified DTs are termed IDTs. The IDT approach has several potential advantages over the traditional DT approach. These include a relaxing of the formalism of branch selection based on multiattributes and the encapsulating of the complex semantics of medical treatment cyclic workflows. The overall objective of the use of mind maps and IDTs was to transform tacit knowledge (from CPGs and domain expert heuristics) into executable knowledge using a form of knowledge modeling formalism that was acceptable to all stakeholders. Figure 1 depicts the overall flow of knowledge elicitation from tacit knowledge to explicit and finally executable knowledge. The key approach for each form of knowledge elicitation—such as brainstorming, inspection, validation, decision tables, and algorithms are stated at the top of the respective pool, while the key inputs, key objectives, key outputs, and roles of the stakeholders are listed within the pool.

Figure 1.

Knowledge elicitation overview—tacit knowledge, explicit knowledge, and executable knowledge. CNB: core needle biopsy; Comp. Scientist: computer scientist; CPG: clinical practice guideline; S:(DL or I): surgery diagnostic lobectomy or isthmectomy; FNA: fine needle aspiration; S:(DL or I): surgery diagnostic lobectomy or isthmectomy; IDT: iterative decision tree; S:(L): surgery lobectomy; S:(TT): surgery total thyroidectomy.

MATERIALS AND METHODS

Study design

The study was performed by endocrine surgeons from Seoul National University Bundang Hospital (SNUBH) and computer scientists from Kyung Hee University and Sejong University. The study was approved by the institutional review boards of SNUBH (B-1801/447-103).

Clinical knowledge models were established by analyzing CPGs and healthcare system workflows for patients with thyroid nodules. Clinical knowledge was interpreted into a set of rules and plugged as a knowledge base for clinical decision support.

To evaluate the knowledge-based CDSS, anonymized clinical data from patients who had undergone surgery for thyroid nodules at SNUBH between January and December 2015 were accessed. Data on the following factors were extracted from the respective medical records: gender, age, underlying thyroid disease, pregnancy status, family history, preoperative blood test results (thyroid-stimulating hormone [TSH]), preoperative imaging results (thyroid sonography, thyroid computed tomography), preoperative histopathology results (fine needle aspiration [FNA] and core needle biopsy), thyroid surgery approach, and outcome. The medical data of each patient were then entered into the knowledge-based CDSS. The knowledge-based CDSS recommendations were compared with those that were applied in the real-world clinical setting. Concordance was expressed as a percentage.

Review of CPGs and surgical principles

The 2015 CPGs of the American Thyroid Association and the 2016 CPGs of the Korean Thyroid Association were inspected by 2 endocrine surgeons (domain experts) in collaboration with computer scientists.^14,15 These CPGs specify that thyroid nodules should be subjected to FNA and that the results should be confirmed. Further management is dependent on the FNA result and comprises the following 5 categories:

1. If the result indicates a malignancy then risk assessment is conducted. Total thyroidectomy should be performed for cases showing evidence of extrathyroidal extension, lateral lymph node metastasis, distant metastasis, multifocality, a history of radiation exposure, or a positive family history of thyroid malignancy. If none of these risk factors are present, TSH, nodule size, and nodal status of the other lobe should be evaluated to determine whether the patient should undergo lobectomy or total thyroidectomy.

2. If the result indicates a suspected follicular neoplasm, a suspected Hurthle cell neoplasm, or any other form of suspected malignancy, the opposite lobe should be inspected to determine whether the patient should undergo lobectomy or total thyroidectomy.

3. If the result indicates an atypia of undetermined significance or a follicular lesion of undetermined significance, core needle biopsy should be performed. Alternatively, the patient may undergo immediate surgery if preferred.

4. If the result indicates a benign nodule, measurement of the TSH level and nodule size, and an inspection of the opposite lobe, should be performed to determine the necessary scope of surgery.

5. All women with a confirmed or suspected malignancy should undergo a pregnancy confirmation. If the patient is pregnant, core needle biopsy should be performed. If a pregnant woman requires surgery, the timing of the operation should be determined according to the gestation period. In women presenting in the third trimester, surgery should be postponed until after delivery.

Clinical knowledge modeling for thyroid nodule

A 2-step process was used to analyze the CPGs and convert them into the final clinical knowledge models: (1) representation of the thyroid nodule treatment plans in the form of a semiformal model (ie, mind map) and (2) conversion of the mind map into formal clinical knowledge model (ie, IDT). The IDT (explicit knowledge) was then converted into rules (executable knowledge). The overall elicitation process is represented as a BPMN (Business Process Model and Notation)¹⁶ standard diagram in Figure 2.

Figure 2.

Knowledge elicitation process using BPMN (Business Process Model and Notation). ATA: American Thyroid Association; CPG: clinical practice guideline; IDT: iterative decision tree; KB: knowledge base.

Step 1: Representation of thyroid nodule treatment plans as mind map

Domain experts presented treatment plans for thyroid nodules in the form of mind map. The formal definition of a mind map is a visual, nonlinear representation of ideas and relationships (ie, a network of connected and related concepts).¹¹ The mind map is an easily understandable, semiformal modeling approach for tacit knowledge representation, and comprises only 4 concepts: central topic, main topic, topic, and subtopic.^11,17 Details and examples of each concept and their usage are provided in Table 1.

Table 1.

Mind map concepts and best practices for knowledge elicitation

Mind Map Concept	Description	Remarks
CT (rectangle with curved corners) color: (any) optional	Context and Semantics: CT is the starting point of the mind map. Only 1 CT is allowed in the mind map. Best use practices (for knowledge elicitation): Start with the key domain concept that drives toward the objective (clinical). Assign a concept that is central and a starting point for all other domain concepts.	Example: For clinical knowledge to introduce an intervention in a thyroid cancer treatment plan, “Treatment of Thyroid Cancer” becomes the CT of the mind map.
MT (rectangle with curved corners) color: (any) optional	Context and Semantics: MT is the second level concept in the mind map. One or more MT(s) are associated with the CT. Best use practices (for knowledge elicitation): MTs include the key concepts, which have a direct association to the CT. Assign concepts that are key preliminaries for the final objectives.	Example: Domain expert decisions for any treatment plan for thyroid nodule depend on tumor malignancy status, which is derived from FNA results. Therefore, “FNA result” is a key candidate in deriving the overall treatment plan and should be included as an MT.
TC (rectangle with curved corners) color: (any) optional	Context and Semantics: TC expands the MT(s) to more detailed information. MT may have 1 or more TCs. TC may include other TCs if required. Best use practice (for knowledge elicitation): Introduce detailed key concepts for each MT or TC (if a further level is required). Typically, the key outcomes of MTs become the CTs for the corresponding MTs. TCs cover concepts that are near to the achievement of the key objective.	Example: The key observation set of FNA results (MT) are the next level of domain concepts that domain experts use for further decision making. For example, the FNA observation “Malignant” is a key candidate TC where the domain expert narrows down the options of suggesting appropriate action.
ST (ST is represented as plain text) SubTopic	Context and Semantics: STs are considered the end points of the mind map CT. One or more STs can be associated to the TC. ST may have other associated STs. One ST may be shared with more than 1 TC or ST. Some TCs play a role in further refinements of STs (ie, they may not be associated with any TC or MT). Best use practices (for knowledge elicitation): Use STs to formulate a conclusion that supports the key objective. Use supporting TCs if the ST conclusion requires further refinement.	Example: Surgery is the key ST for the thyroid nodule treatment plan, and is decided upon inspection of the patient FNA result status, which was covered as a TC. Moreover, domain experts further categorize the “Surgery” into “Surgery: Lobectomy” or “Sugery: Total-Thyrodectomy” via further analysis of patient symptoms. This information is provided as supporting TCs.

CT: central topic; FNA: fine needle aspiration; MT: main topic; ST: subtopic; TC: topic.

Other approaches to the modeling of initial tacit knowledge include concept maps, conceptual diagrams, and visual metaphors. The mind map was selected due to its alignment to the key objectives of the present study (see Figure 1). Table 2 compares the mind map with the most similar approach (ie, the concept map).

Table 2.

Key benefits of mind maps over concept maps in terms of clinical knowledge elicitation

Key Features	Mind Map	Concept Map	Key Benefits of Mind Map Over Concept Map
Formalism	Involves much less formalism. For example: no need for named-relationship.	Involves more formalism. For example: named-relationship.	At the initial stage of knowledge modeling, it is difficult for physicians to learn constrained models and mold their experiences into a more formal representation.
Semantic relationship	Straightforward structure with free multidirectional knowledge representation focused around a central concept. For example: A single central concept is extended into multiple layers in all directions, with more general to specific knowledge elicitation.	Hierarchical structure with a top-down single directional knowledge representation approach. For example: It contains a top root concept, and subsequent concepts are derived in a children and grandchildren fashion.	For tacit knowledge with distributed fragmented artifacts in mind and other sources, it becomes difficult for domain experts to associate concepts so tightly in a hierarchical manner.
Simplicity	Less complex structure and easier refinements in further iterations of knowledge elicitation phase. For example: Every concept has a single parent and is easily traceable due to a lack of concern for semantically-named relationships.	Complex structure with emphasis on capturing most of the semantics of the knowledge. For example: Concepts may have multiple parents with proper-named relationships.	For domain experts with limited time for research, prefer less complex models with minimal formalism, and a process that allows a refinement in multiple iterations.
Domain expert training	No formal training is required. For example: A lower formalism burden, less constrained relationships, and freestyle multidimensional expansion of concepts frees domain experts from requiring formal training sessions.	Requires intensive training. For example: Identifying concepts and their proper placement with proper relationship, and understanding key relationship names requires hands on practice under supervision.	The most important goal for a domain expert is to provide optimal patient management, and most of their working time is spent with patients. They adapt technology and toolsets that genuinely save time, facilitate decision making, and require less time to learn.

The domain experts inspected the recommendations of the American Thyroid Association and Korean Thyroid Association CPGs^14,15 concerning treatment for thyroid nodule. The first step was to identify all key concepts of relevance to patient symptoms, clinical observations, clinical findings, and treatment recommendations. In modeling the treatment plan for thyroid nodules, the candidate key concepts included FNA result—malignant (test result), distant metastasis—clinical stage M1 (clinical findings), voice change (patient symptoms), and surgery (treatment recommendation). After identifying the key candidate concepts, the second step was to align and connect the concepts appropriately. Most of the relationships were tangible and easily manageable. For example, the concepts malignant and benign derive from the concept FNA result. However, for some concepts, the relationships were ambiguous, or more than one possibility for establishing the connection was available. In such cases, the domain experts brainstormed or consulted additional sources of evidence to clarify the relationship. The review process was then performed. Here, the computer scientists investigated the mind map and refined it as required. The detailed processes conducted by each stakeholder (domain expert and computer scientist) are shown in Figure 2 (Tacit Knowledge Pool). The final mind map for the treatment of thyroid nodules was generated following 4-5 refinement iterations (Figure 3). As an outcome, it was shared as a key input for IDT (explicit knowledge) modeling.

Figure 3.

Mind map for the planning of thyroid nodule treatment.

Step 2: Converting mind map into a formal clinical knowledge model—IDT

To refine mind map knowledge and facilitate its implementation, an IDT was introduced. This is an extension of the traditional DT model. A DT is a visual tool that can be used in a decision-making process, and which presents all possible options and their corresponding consequences.¹⁸ The DT also assists domain experts in terms of facilitating decision-making processes in the field of health and medicine, and has been used as a baseline tool for most healthcare decision-making approaches, such as PROACTIVE (problem, reframe, objectives, alternatives, consequences and chances, trade-offs, integrate, value, explore and evaluate).¹⁸ The decision to introduce IDT rather than DT was based on the key modeling objectives indicated in Figure 1. The key benefits of IDT in comparison with DT are shown in Table 3.

Table 3.

Key benefits of IDT compared with traditional DT

Key Features	IDT	DT	Key benefits of IDT over DT
Rich and flexible formalism	IDT has the capacity to cover-up guideline semantics with rich and flexible formalism. Example: In principle, IDT covers up DT formalism and provides flexibility and extension, such as support for complex logical expression at nodes and branches to enable multivalued domain concept evaluation.	DT has the capacity to cover-up guideline semantics but with strict formalism. Example: Individual nodes can evaluate only 1 concept at a time.	IDT allows domain experts and computer scientists to model clinical knowledge in a more flexible way.
Easily understandable to stakeholders	IDT is compact and easily understandable to all stakeholders. Example: The rich and flexible formalism of IDT renders it both compact and understandable.	DT is easily understandable but the model is not compact. Example: For detailed guidelines, the DT may expand with no compact visual snapshot.	IDT allows the compact capture of detailed guideline semantics.
Support manageable model	IDT is easily manageable in the context of expanding and repeating knowledge artifacts. Example: Handling repeated knowledge patterns is easily manageable using the iterative cyclic branch structure.	DT is not feasible for extensive knowledge volumes. Example: The top-down noncyclic approach renders DT difficult to manage in the context of expanding knowledge.	The CPGs include patterns of knowledge. IDT allows the identification of patterns within knowledge artifacts, as well as reuse to avoid duplicating via the incorporation of a cyclic branch structure.

CPG: clinical process guideline; DT: decision tree; IDT: iterative decision tree.

The IDT encompasses all of the fundamental principles of the traditional DT, while (1) introducing flexibility in terms of certain formal constraints and (2) extending its capabilities in terms of expressing the semantics of clinical knowledge concepts. However, in contrast to traditional DT, IDT provides a cyclic top-down structure, whereby the leaf nodes may connect to any intermediate node. A leaf node can therefore assume the role of a decision node. Moreover, in contrast to the traditional DT,^19,20 the branch division in IDT may involve multiple attributes, and relaxes the formalism of division by supporting domain expert intuitions or experience, or some other external form of evidence. Detailed information concerning IDT formalism and artifacts is provided in Table 4.

Table 4.

IDT concepts with examples and formal notations

IDT Artifact	Description	Remarks
Condition Node (Rectangle)	Semantic: Decision Node Decision branch Making: single/multiple attributes Color (Optional): Yellow When to use: Attribute value set is limited Alternate for Composite Condition Node	Examples:   DN1: TSH {Normal, Elevated, Subnormal} DB1: Normal DB2: Elevated OR Subnormal DN2: Pregnancy {Y, N} AND Sex {M, F} DB1: Y AND F DB2: N AND F
		Note: value set of TSH and other concepts are known as domain-values.
	Formal BNF:    $\mathit{condition Node}\colon\colon =\mathrm{ }〈\mathit{domain Term}〉\mathrm{ }|\mathrm{ }〈\mathit{domain Term Binary}〉$  $\mathit{domain Term Binary}\colon\colon =$  $\mathrm{ }〈\mathit{domain Term}〉〈\mathit{com}.\mathit{Opr}〉〈\mathit{domain Value}〉\left\{〈\mathit{log}.\mathit{Opr}〉〈\mathit{domian Value}〉\right\}|$  $\mathrm{ }〈\mathit{domian Term}〉〈\mathit{log}.\mathit{Opr}〉〈\mathit{domain Term}〉\left\{〈\mathit{log}.\mathit{Opr}〉〈\mathit{domain Term}〉\right\}$	$\mathit{domain Term}\colon\colon =\mathrm{ }\mathit{domain}\mathrm{ }\mathit{concept}\mathrm{ }|\mathrm{ }\mathit{activity}$  $\mathit{com}.\mathit{Opr}\colon\colon =\mathrm{ }>\left|<\mathrm{ }\right|=\left|\mathrm{ }\le \right|\ge \left|\ne \right|!=|:$  $\mathit{log}.\mathit{Opr}\colon\colon =\mathit{OR}\mathrm{ }|\mathrm{ }\mathit{AND}$
	Formal BNF (Decision Branch):  $\mathit{decision Branch}\colon\colon =\mathrm{ }〈\mathit{binary Branch}〉\mathrm{ }|\mathrm{ }〈\mathit{composit Branch}〉$  $\mathit{binary Branch}\colon\colon =\mathrm{ }〈\mathit{Yes}|\mathit{No}〉\mathrm{ }|\mathrm{ }〈\mathit{True}|\mathit{False}〉$	$\mathit{composite Branch}\colon\colon =\mathrm{ }〈\mathit{com}.\mathit{Opr}〉〈\mathit{domainValue}〉\left\{〈\mathit{log}.\mathit{Opr}〉〈\mathit{com}.\mathit{Opr}〉〈\mathit{domainValue}〉\right\}\mathrm{ }|$  $\mathrm{ }〈\mathit{domainTerm}〉〈\mathit{com}.\mathit{Opr}〉〈\mathit{domainValue}〉\left\{\begin{array}{c}〈\mathit{log}.\mathit{Opr}〉〈\mathit{domain Term}〉\\ 〈\mathit{com}.\mathit{Opr}〉〈\mathit{domain Value}〉\end{array}\right\}$  $\mathrm{ }|\mathrm{ }〈\mathit{domain Term Binary}〉$
Activity Node (Rectangle with curved corners)	Semantic: Decision Node / Leaf Node Decision branch Making: single/multiple attributes Color (Optional): Brown When to use: Attribute represent outcome of some complex medical process Attribute represent recommendation for some complex medical process	Examples:   DN1: FNA {Yes, No} DB1: Yes DB2: No LN1: EA or RFA
	Formal BNF:  $\mathit{activityNode}\colon\colon =\mathrm{ }〈\mathit{condition Node}〉\mathrm{ }|\mathrm{ }〈\mathit{leaf Node}〉$	$\mathit{leaf Node}\colon\colon =\mathrm{ }\mathit{any}\mathrm{ }\mathit{recommendation}\mathrm{ }|\mathrm{ }〈\mathit{domain Term}〉$
Composite Condition Node (Diamond)	Semantic: Decision Node Decision branch Making: single/multiple attributes Color (Optional): Light Green When to use: Attribute represent outcome of some complex medical process Attribute represent recommendation for some complex medical process Alternate for Condition Node	Examples:   DN1: FNA Result {Malignant, susp. Malig., FN, susp. FN, HCN, susp. HCN, AUS, FLUS, Benign, Non. Diag} DB1: Malignant DB2: AUS or FLUS CB3: Non. Diag DN2: Size {[0-$\mathrm{\infty }$]} DB1: [0-1) -> Size < 1 DB2: [1-4) -> Size ≥ 1 and Size < 4 DB3: [4-$\mathrm{\infty }$] -> Size ≥ 4
	Formal BNF:  $\mathit{composite Condition Node}\colon\colon =\mathrm{ }〈\mathit{condition Node}〉$
Recommendation Node (Rectangle)	Semantic: Leaf Node Decision branch Making: single/multiple attributes Color (Optional): Light Blue When to use: Attribute represent final consequences as recommendation. Leaf node may play role as decision node.	Examples:   LN1: Surgery: L LN2: Surgery: (DL) OR Surgery (I)
	Formal BNF:  $\mathit{RecommendationNode}\colon\colon =\mathrm{ }〈\mathit{leafNode}〉$

AUS: atypia of undetermined significance; BNF: Backus–Naur form; DB: decision branch; DL: diagnostic lobectomy; DN: decision node; EA: ethanol ablation; F: female; FLUS: follicular lesion of undetermined significance; FN: follicular neoplasm; FNA: fine needle aspiration; HCN: Hurthle cell neoplasm; I: isthmectomy; IDT: iterative decision tree; L: lobectomy; LN: leaf node; M: male; N: no; non. diag: nondaignostic; RFA: radiofrequent ablation in symptomatic case; susp. FN: suspicious for follicular node; susp. HCN: suspicious for Hurthle cell neoplasm; susp. Malig: suspicious for malignancy; TSH: thyroid-stimulating hormone; Y: yes.

In the present study, the various IDT artifacts were used to transform the mind map for the thyroid nodule treatment plan (from Figure 3) into an IDT model of the thyroid nodule treatment plan. This process of transformation was iterative in nature. Details of the process are shown in Figure 2 (Explicit Knowledge Pool). After several iterations, the mind map of the thyroid nodule treatment plan was transformed into the final IDT knowledge model (Figure 4). However, in best practice, the following tasks should be performed to transform the mind map into an IDT in a linear manner:

Figure 4.

Iterative decision tree (IDT) as a final clinical knowledge model for the planning of nodule treatment.

• Identify concepts in the mind map that are of relevance to decision making.

• Separate decision-making concepts in the mind map and align them with the IDT artifacts—decision node, leaf node.

• For decision nodes and leaf nodes, assign appropriate specialization artifacts—such as activity, condition, or recommendation.

• Use connections from the mind map to enlarge the IDT model with decision branches.

• Make necessary refinements by investigating each decision branch.

• Approve the IDT model for use in clinical decision support in the healthcare workflow.

Step 3: Conversion of IDT into executable knowledge—rules

The IDT was represented in UML (unified modeling language) notation and converted into XMI (XML Meta Interchange)²¹ format using the Enterprise Architect software.²² To generate the set of candidate rules, the IDT-XMI model was provided to algorithms (1 and 2). The IDT-to-rule transformation algorithms are depicted in Figure 5a. The rules were generated in 2 passes:

Figure 5.

Iterative decision tree (IDT) to rule generation. (a) IDT to rules transformation algorithm (b) Flattening example of IDT.

Pass-I: Consumed the IDT as an XMI model, parsed the IDT-XMI model into an internal object model, and flattened the model into DT structure (as shown in algorithm 1). In flattening, the cycles were removed and the model was aligned into a top-down structure, as shown in Figure 5b.

Pass-II: Consumed the flattened IDT object model and generated rules using the recursive procedure (as shown in algorithm 2). The recursive procedure acts on the IDT model in a top-bottom, left-right manner, whereby rules are generated for each decision branch (also termed edges) and returned as a set of candidate rules. In the case of thyroid nodules, the IDT model was represented by a set of 3700 rules. Test-case-based validation was used to verify rule consistency. The rules were then plugged into the knowledge base for inferencing in cases from the real-life setting.

RESULTS

During the study period, a total of 483 patients received treatment for thyroid nodules at SNUBH (Figure 6).

Figure 6.

Comparison of surgical management in the clinical setting and the respective knowledge-based clinical decision support system (CDSS) recommendation via a retrospective review of patient data. CNB, core needle biopsy; EAR, ethanol or radio frequency ablation; I, isthmectomy; L, lobectomy; N, number; SNUBH, Seoul National University Bundang Hospital; TT, total thyroidectomy.

A total of 239 patients underwent total thyroidectomy. For these 239 patients, CDSS recommended total thyroidectomy in 198, lobectomy in 35, core needle biopsy in 2, and ethanol or radio frequency ablation in 4. The accuracy for total thyroidectomy was 82.8%.

A total of 242 patients underwent lobectomy. For these 242 patients, CDSS recommended total thyroidectomy in 57, lobectomy in 182, and ethanol or radio frequency ablation in 3. The overall accuracy for lobectomy was 75.2%.

The remaining two patients underwent isthmectomy. In these two patients, CDSS recommended total thyroidectomy in 1, and isthmectomy in 1. The overall accuracy for isthmectomy was 50%.

Overall concordance between SNUBH management and the CDSS recommendation was 78.9%.

DISCUSSION

Previous research has shown that around 80% of diagnostic errors are attributable to a failure to perform necessary orders or to obtain accurate past records; inaccurate tests; or an inaccurate review of documents.²³ These errors may also occur during the decision-making phase of drug prescription and surgical treatment. CDSS has been introduced in an attempt to reduce such errors, and, in theory, CDSS can be introduced for all aspects of diagnosis, treatment, prescribing, and prevention. The non–knowledge-based CDSS WFO from IBM has attracted much attention. However, its internal process is unclear, rendering verification problematic. According to results presented at the ASCO meeting in 2017, WFO performance varies according to country, disease, and disease stage.^4,24,25 This emphasizes the importance of transparent, knowledge-based CDSS. To date, knowledge-based CDSS has been used primarily for drug prescriptions, with a few cases being applied to diagnosis and treatment.⁵ Although CDSS for the treatment of thyroid nodule has been reported,¹⁰ no patient-based review data have been published. In the present study, an IDT model was used to create an easily implementable knowledge-based CDSS for thyroid nodule treatment. The applicability of the knowledge-based CDSS to thyroid nodule treatment was verified, and the CDSS recommendations were compared with treatment prescribed in the real-world setting.

The results suggest that the choice of surgical intervention is largely (78.9%) based on the preoperative medical history, and on the results of the TSH blood test, thyroid imaging, and histopathological examination. A total of 21.1% patients received treatment that was discordant with the CDSS recommendation. To investigate this discrepancy, the 2 groups accounting for the majority of discordant patients were examined: (1) 35 patients for whom lobectomy was recommended by CDSS but who underwent total thyroidectomy and (2) 57 patients for whom total thyroidectomy was recommended by CDSS but who underwent lobectomy. This analysis showed that in these patients, conversion was attributable to either an unexpected histological finding during intraoperative frozen biopsy lymph node analysis, or a refusal on the part of the patient to undergo total thyroidectomy. In routine clinical practice, the detection of metastatic tissue in lymph nodes during intraoperative frozen section analysis is an indication for conversion from lobectomy to total thyroidectomy.²⁶ The percentage of conversions from lobectomy to total thyroidectomy in the present cohort is accepted in the clinical environment. To reduce this rate, CDSS must be improved. Therefore, future CDSS should include results in the operative field. A reasonable assumption is that future CDSS will have a superior prediction rate. Nonetheless, the present CDSS for thyroid nodule can provide patients with important information on treatment modality before surgery. In patients who opted for lobectomy, total thyroidectomy remained an option during subsequent outpatient follow-up.

The construction of a CDSS for diagnosis and treatment necessitates a substantial investment of time and resources, as it is difficult to change the logic once configured. Existing CDSS are highly reliant on computer scientists for knowledge maintenance. This is due to a lack of (1) any integration of the medical diagnosis process, (2) an evidence-based support function, (3) standardized vocabulary and knowledge expression use, (4) interoperability, and (5) integration with other systems. Previously developed knowledge-based CDSS have therefore tended to focus on relatively simple diseases. In addition, previously configured CDSS face the challenge of integrating an ever-expanding knowledge base. Through the implementation of mind maps and IDTs, the gap of sharing and modeling clinical knowledge between medical experts and computer scientists is virtually eliminated. In the present study, this enabled the successful deployment of CDSS with an easily maintainable clinical knowledge base for thyroid nodule treatment. Basing CDSS on the creation of mind maps and IDTs thus represents a promising precedent for future research.

To acquire knowledge from CPGs and integrate this into CDSS, team members from diverse domains who offer differing capabilities and expertise must agree on common models. These must use language and notations that reflect knowledge from CPGs and which can be readily understood by all stakeholders. Therefore, to generate a final knowledge base for the present CDSS, 2 models were applied during the knowledge acquisition process: a mind map and an IDT. The mind map and the IDTs have the properties required to capture the intent and inherent knowledge of CPGs, and provide common modeling notations for domain experts and computer scientists. The mind map has been applied to various knowledge sharing domains, particularly in the field of medicine.^11–13 The main advantage of the mind map is its semiformal representation of concepts.^11,17 This enables domain experts to identify key clinical concepts and associate them with simple but meaningful relationships that conform to the minimal formal constraints of concept modeling. With simple but comprehensive modeling support, Mollberg et al²⁷ demonstrated that mind maps facilitate insights into the individual patient and their tumor characteristics, thus broadening therapeutic options in clinical practice. Ultimately, this personalized approach maximizes efficacy and reduces risk to the patient. However, this free form also generates limitations in terms of the clear linking of concepts.¹² The absence of clear links between concepts leads toward implicit knowledge representations. In this regard, domain experts must explain the implicit relationships between concepts, which yield directly executable knowledge for the purposes of clinical decision support. Therefore, an alternative knowledge model, which is executable or at least easy to convert, is required. Furthermore, the model must reflect the explicit knowledge of experts from diverse domains (ie, medicine and computer science). To deal with this challenge, IDT was used for thyroid knowledge modeling. As with the traditional DT, IDTs express the knowledge model more explicitly than mind maps. Using the extended formalism of a cyclic top-down structure, IDT provides flexibility in terms of representing and maintaining an ever-expanding knowledge domain. Furthermore, IDT is reflected in standard UML, in which it is easily transformed into executable knowledge. In the present study, the IDT was converted to rules using a formal recursive algorithm. However, the IDT is easily readable and traceable, and a domain expert or computer scientist may convert it into an intermediate format, such as a decision table.

The advantages of introducing CDSS for thyroid nodules are 2-fold. First, it engenders trust in the recommended treatment approach, since the patient is seen on a preoperative basis. Second, it enhances the comprehensive documentation of necessary information before surgery. CDSS is an effective tool for reducing errors among physicians with high case loads and can provide guidance to physicians who are unfamiliar with practices in the given field. The present knowledge-based CDSS for thyroid nodules may therefore be helpful to physicians in primary care medical institutions with less experience in the surgical management of thyroid disease, or in countries with less thyroid expertise.

The present study had several limitations. First, the knowledge-based CDSS was validated using retrospective data from the medical records of 483 patients. However, to implement a knowledge-based CDSS in the real-world setting, a large-scale preliminary study is warranted. Second, the IDT (from Figure 2) reflects a thyroid nodule treatment plan that was derived from CPGs in general. However, due to a lack of suitable cases, some of the decision paths in the developed IDT could not be assessed using the retrospective SNUBH clinical data. Decision paths that were not encompassed by the 483 patients were therefore validated using test case–based validation techniques. Third, since the 78.9% concordance rate is inadequate, further research is warranted to increase the accuracy of the present knowledge-based CDSS. Knowledge-based CDSS is an area in which future development is likely.

CONCLUSION

In the present study, a knowledge-based CDSS was developed for thyroid nodule treatment. A key aspect of CDSS is knowledge acquisition, and this was achieved using 2 different models: mind maps and IDTs. The knowledge-based CDSS was evaluated using retrospective data from cases treated at SNUBH. The results indicate that CDSS may assist domain experts in the planning of treatment for thyroid nodules. The use of mind maps and IDTs facilitated both knowledge acquisition from thyroid nodule CPGs, and collaboration between stakeholders from the domains of endocrine surgery and computer science.

To increase the accuracy of final recommendation outcomes, we now plan to investigate the 21.1% discordant cases. Machine learning approaches will be applied to create a hybrid recommendation model for the planning of thyroid nodule treatment.

FUNDING

This research was supported by the Ministry of Science and ICT, Korea, under the Information Technology Research Center support program (IITP-2017-01629) supervised by the Institute for Information and Communications Technology Promotion.

AUTHOR CONTRIBUTORS

H.W.Y. and M.H. designed the study and prepared the manuscript, and are joint first authors. M.A. and T.A. acquired and analyzed the data and reviewed the manuscript. J.Y.C., H.H., and S.L. contributed to the interpretation of data. J.Y.C. and S.L. are joint corresponding authors.

Conflict of interest statement. None declared.