CORR Synthesis: Can Decision Tree Learning Advance Orthopaedic Surgery Research?

In the Beginning

Decision tree learning is a promising machine learning alternative that could enhance research interpretability. Researchers have acknowledged a major problem in clinical analytics; statistical results are often poorly presented and not expressed in terms that are useful for decision-making [32]. Producing accurate outcome predictions in research is essential, but just as crucial is the ability of the study to deliver practical insights for decision-makers [6, 35]. Decision trees have this capability and allow users to organize data into simple diagrams that have predictive accuracy comparable to that of regression analyses [6].

Decision trees were developed as an alternative to multivariable regression analysis to manage the interactive effects that variables have on each other in statistical analyses. Belson proposed the initial concept in 1959 [5], and it was later advanced by Morgan et al. [25, 26] in 1963, when they introduced their automatic interaction detection and theta automatic interaction detection models. Kass [21] further enhanced these models into chi-square automatic interaction detection (CHAID), which uses the chi-square test to split variables based on their respective statistical significance with the outcome. Decision trees’ greatest enhancements occurred when two independent research teams developed their unique models several years later. Breiman [6] first described his binary classification and regression tree (CART) in 1984, and Quinlan [34] published his multiway splitting tree ID3 in 1986. Both models separated data into subgroups in ways that made predictive modeling more effective, and over time, these versions were enhanced considerably. Notably, Quinlan's ID3 advanced into versions C4.5 and C5.0, and Breiman's CART advanced into random forest [7, 44]. Random forest builds multiple trees through resampling training data and combines the trees to produce greater prediction accuracy [7]. However, random forest discards the decision tree diagram, blinding the researcher to how the results were produced [7]. This blinding transition is known as the black box method. Black box methods have algorithms so complex that interpreting them is infeasible. Decision trees are a “clear box” method because the predictive structures are displayed.

The Argument

The habitual domain theory suggests people develop a collection of ideas, perceptions, and actions through experience and knowledge [22]. Over time, this collection will stabilize and influence decisions and behavior, unless new information is presented or purposeful effort is exerted to change it [22]. If there is a barrier to understanding new information, such as complex statistical information, the new information will be lost, and decision-makers will continue to act in their habitual domain, potentially leading to biased decisions [22]. Decision trees could remedy this potential problem with their simple and interpretable predictive algorithms.

Decision trees are a machine-learning method that perform classifications with categorical variables and regression with numerical variables such as logistic and linear regression [6, 7]. However, trees have been described as simpler to interpret and more practical to use in a clinical setting than regression analyses [6, 35]. This is attributed to the trees' ability to create simple clinical decision rules with their algorithmic diagram, which works similar to a flowchart [6]. A clinician can follow the flowchart to see which combinations of variables have the greatest accuracy at predicting an outcome. Another major advantage to trees compared with regression is that they are nonparametric, which allows trees to handle highly skewed datasets, outliers, and missing values, eliminating the need for scaling or data transformations [6]. Because of these advantages, if adopted, trees may enhance the interpretation and predictive accuracy of orthopaedic surgery research. As such, this review aimed to describe, identify, and highlight published research that has used decision tree learning in orthopaedic surgery research and to discuss the advantages and disadvantages of using this approach.

Essential Elements

This review sought to identify original prospective and retrospective studies in the most published and cited orthopaedic surgery journals that used decision tree learning as a statistical analysis. The decision tree analyses I searched for included CHAID, ID3, C4.5, C5.0, and CART. An advanced PubMed search was performed on the top 20 orthopaedic medicine and surgery journals ranked according to Google Scholar's h5-index from the earliest date to 2023 (Table 1). Terms queried in a title and abstract search included “classification and regression tree”, “chi-square automatic interaction detection tree”, “ID3”, “C4.5”, “C5.0”, “CART”, and “CHAID”. The search yielded 159 articles. After removing 10 duplicate articles, 149 articles remained, and their abstracts were inspected for decision tree analysis results. If abstracts reported the results of decision tree analyses, articles were further evaluated for verification. After inspection, 130 studies were eliminated, resulting in the inclusion of 19 studies in this review.

Table 1.

Journals queried

The American Journal of Sports Medicine

The Journal of Arthroplasty

The Journal of Bone and Joint Surgery, American Volume

The Journal of Bone and Joint Surgery, British Volume

Knee Surgery, Sports Traumatology, Arthroscopy

The Bone & Joint Journal

Clinical Orthopaedics and Related Research

Spine

Journal of Shoulder and Elbow Surgery

Arthroscopy: The Journal of Arthroscopic Related Surgery

European Spine Journal

The Spine Journal

BMC Musculoskeletal Disorders

Orthopaedic Journal of Sports Medicine

International Orthopaedics

Injury

The Journal of the American Academy of Orthopaedic Surgeons

Journal of Orthopaedic Research

Journal of Orthopaedic Surgery and Research

Current Reviews in Musculoskeletal Medicine

Acta Orthopaedica

On the Science

Predictive analytics is an analytical approach that discovers patterns in a dataset that can predict an outcome or enhance insight into a decision [36]. Machine learning is a method of predictive analytics that takes a proportion of a dataset, then analytically finds patterns in that proportion that can best predict an outcome [6, 34, 35]. These identified patterns are then tested on the remaining proportion of the dataset, or an independent external dataset, to determine how accurately that pattern predicts the outcome [6, 34, 35]. Decision tree learning is simply performing this pattern recognition by analytically splitting the dataset into multiple subgroups (nodes) [6, 34, 35] (Fig. 1). This splitting process continues sequentially until the last subgroup can completely predict the outcome or predetermined stopping rules are met [6]. When the process is complete, the sequence of splitting multiple subgroups forms a diagram shaped like an inverted tree. The result is a path of variables that traces to the outcome with their respective likelihood of predicting the outcome.

Fig. 1.

A fictional dataset was analyzed with classification and regression tree analysis to predict nonunion in high-energy tibia fractures. AO/OTA = AO Foundation/Orthopaedic Trauma Association; 41C1 = tibia proximal end simple articular, simple metaphyseal fracture; 41C2 = tibia proximal end simple articular, wedge or multifragmentary metaphyseal fracture; 41C3 = tibia proximal end fragmentary or multifragmentary metaphyseal fracture; 43C1 = tibia distal end simple articular, simple metaphyseal fracture; 43C2 = tibia distal end simple articular, multifragmentary metaphyseal fracture; 43C3 = tibia distal end multifragmentary articular and multifragmentary metaphyseal fracture.

For example, Figure 1 shows how a fictional dataset was analyzed with classification and regression tree analysis to predict nonunion in high-energy tibia fractures. Nonunion occurred in 12% (612 of 5000) of fictional patients with high-energy tibia fractures (Node 0). Nonunion occurred in 31% (320 of 1005) of fictional patients with an open injury high-energy tibia fracture (Node 1). Nonunion occurred in 77% (94 of 122) of fictional patients with high-energy tibia fractures that were open injuries with AO/OTA classifications 41C1 and 41C2 (Node 4). Nonunion occurred in 46% (69 of 149) of fictional patients with high-energy tibia fractures that were open injuries with AO/OTA classifications 41C3, 43C1, 43C2, or 43C3, and a postoperative surgical site infection (Node 6). The fictional model's accuracy was 90.1%.

What We (Think) We Know

Complications in Spine Surgery

Of the orthopaedic surgery specialties, spine has used decision trees the most. Trees were typically used to predict patient outcomes, complications, and prognosis after spine surgery. CHAID correctly classified 90.1% of the successful outcomes in 40 patients who underwent decompressive laminarthrectomy [39]. CHAID also predicted clinical outcomes with 67.7% accuracy, 65.1% sensitivity, and 69.8% specificity in 331 patients who underwent minimally invasive lumbar spinal stenosis [41]. CART predicted postoperative complications among 279,135 patients who underwent varying spine procedures [37]. However, in this study, logistic regression and least absolute shrinkage and selection operator logistic regression with interactions yielded better predictive performance than CART, with the area under the curve ranging between 0.66 and 0.72 in varying cohorts with different approaches. These authors all demonstrated that trees have predictive power in various sample sizes and procedures, but the predictive accuracy may depend on the researchers’ supervision. This includes sample size selection, variable selection, using cross-validation and pruning, and setting specific stopping rules.

Proximal Junctional Failure

C5.0 was effectively used in spinal deformity research to predict proximal junctional failure with 75% to 90% accuracy. C5.0 predicted proximal junctional failure with or without the bone mineral density score in 145 patients with adult spinal deformities [45]. In 10 variations of trees, the accuracy ranged between 82.6% and 90.9% [45]. Additionally, C5.0 predicted proximal junctional kyphosis and proximal junctional failure in 510 patients with adult spinal deformities with 86.3% accuracy and an area under the curve of 0.89 [38]. Both studies suggested the tree models can support preoperative decision-making and risk stratification and advise prophylactic strategies for patients with adult spinal deformities.

Tracheostomy

CART predicted tracheostomy after traumatic cervical spinal cord injuries, with classification accuracies ranging between 87% and 94%. Tracheostomy was predicted by CART in 345 patients with acute traumatic cervical spinal cord injuries, with an area under the curve of 0.90, 73.7% sensitivity, and 89.7% specificity [16]. A similar study used CART to predict the need for tracheostomy in 340 patients with traumatic cervical spinal cord injuries, with 94.1% accuracy, 78.0% sensitivity, and 96.3% specificity [40]. When used to identify risk factors for tracheostomy after traumatic cervical spinal cord injury in 105 patients, CART produced 87.8% accuracy [23]. All three studies highlight that CART can be precise in research investigating the same injury in different patient populations.

Quality-of-life Outcomes in Spine Surgery

CART also predicted quality-of-life outcomes such as preoperative pain scores, catastrophic surgery costs, and independent living after spine surgery. CART identified important factors associated with low preoperative Scoliosis Research Society pain scores in 2585 adolescent patients with idiopathic scoliosis [18]. A regression tree predicted the rate of catastrophic cost accruement in 210 patients with adult spinal deformities, with adjusted r² values of 56% for 90-day and 35.6% for 2-year periods [2]. In comparison, random C-forest regression produced adjusted r² values of 57.4% for 90-day and 28.8% for 2-year direct cost prediction [2]. CART also predicted which patients with cervical spinal cord injuries would have difficulty obtaining an independent living, with 78.6% accuracy, an area under the curve of 0.81, 80.7% sensitivity, and 75.1% specificity [15]. These studies emphasize that CART can identify patients at risk for quality-of-life and postoperative complications.

Clinical Decision Making in Spine

Aside from predicting postoperative complications, trees have been used diagnostically and operationally. CART distinguished acute lumbar spondylolysis from nonspecific low back pain in 223 patients, with an area under the curve of 0.79, 92% sensitivity, and 92% specificity [4]. A single classification tree identified the most influential intraoperative and postoperative blood transfusion variables in 1029 patients with adult spinal deformities, with an area under the curve of 0.79 [11]. However, a random forest model with the same variables produced an area under the curve of 0.85 [11]. These results highlight that trees can be used to inform or improve the quality of decision-making for surgeons and their clinical teams.

ACL Reconstruction Retear

There is limited use of trees outside spine surgery; however, a developing trend is applying the algorithms to knee biomechanical research, such as predicting second ACL injuries and accelerated osteoarthritis. CART identified risk factors for a second ACL injury in 117 female soccer players, with 89% accuracy, 100% sensitivity, and 76% specificity [12]. A similar study used CART to predict secondary ACL injuries in 163 patients and created two high-risk profiles, with 66.7% sensitivity and 72.0% specificity [31]. These studies demonstrate that trees can organize complex interactions of biomechanical variables into interpretable diagrams.

Accelerated Knee Osteoarthritis

Similar to the ACL biomechanical studies, CART has identified risk factors for developing accelerated knee osteoarthritis in 162 patients, with 44% sensitivity and 94% specificity [10]. In a similar study, CART classified adults who would experience accelerated radiographic knee osteoarthritis within 4 years among 162 patients and three separate models [33]. Adding magnetic resonance–based features to the model improved specificity from 82% to 90% but lowered sensitivity from 70% to 62% [33]. Regarding patient-reported outcome scores after TKA, CART determined nonlinear relationships with simulated joint dynamics in 96 patients [42]. These studies revealed complex biomechanical and physiology metrics that can be combined in analyses despite often being incompatible with traditional statistics.

Acetabular Fractures

Researchers in the trauma specialty may find the most value in decision trees because trees allow researchers to take advantage of uncontrollable and complex confounders that are often challenging for randomized controlled trials. However, only one study in this query used decision trees to predict clinical outcomes in trauma [14]. CART identified important predictors of modified clinical grading scores of 34 patients with T-shaped acetabular fractures [14].

Knowledge Gaps and Unsupported Practices

The misuse and misinterpretation of statistical analyses in medical research have been widely discussed for several decades [1, 13, 19]. This has been attributed to a lack of statistical education, journals without statistical refereeing, and studies with poor methodologic quality [1]. Recently, these problems have been found in artificial intelligence, machine learning, and predictive analytics studies, and will likely continue as these methods are used more often [3, 8, 29]. Although decision trees can offer innovative advantages, they are only as effective as the researcher’s diligence to appropriately perform and validate the models. Decision trees are sensitive to sample size alterations, can overfit to training data, and are less accurate and less stable than other black box analyses such as random forest [6, 7]. Small samples and overfitting may exaggerate results [6, 7]. As with other predictive analytics models, it is recommended that trees be validated with internal or external datasets, compared with other predictive models, and transparently reported with instability and calibration measurements so others can judge their accuracy and usefulness [8, 9, 17, 27].

Barriers and How to Overcome Them

Previous publications have presented artificial intelligence and machine learning with ambitious applications such as image reading, patient diagnoses, and replacing the clinician. These ideas may disinterest clinicians and create reservations about and resistance to their adoption [43]. Recent systematic reviews have reported weaknesses in artificial intelligence studies that will likely become more common over time [3, 29]. If clinicians are hesitant about adopting these analytic approaches, a conservative yet innovative option would simply involve adopting decision tree learning for case control and cohort studies that use a predictive analytics framework. Orthopaedic surgery articles commonly use predictive analytics in case control or cohort studies, but to achieve this, they primarily use linear and logistic regression [30]. Decision trees can simply complement the regression analyses as a predictive diagram, be used to identify confounding interactions for regression analysis, or may completely replace regression analysis, if appropriate. In some cases, clinicians may prefer accuracy over interpretability and select more appropriate analyses. Ultimately, innovation diffusion will rely on what is the best analytical approach to support clinicians’ decision-making clearly, accurately, and reliably.

5-year Forecast

I speculate that machine learning will increasingly be used in orthopaedic surgery research and may eventually replace regression analysis [20, 24, 27, 28]. Decision tree learning will likely become more common, and increased clinician familiarity may be necessary. Journals are increasingly recommending that articles be written with readability, interpretability, and the cognitive ease of the reader in mind while enhancing statistical accuracy. As such, trees may be increasingly adopted once researchers recognize their practicality in clinician decision-making compared with regression analyses. The studies I have shared have demonstrated that trees can produce accurate predictive results, and that decision tree learning can be used effectively in orthopaedic surgery research. As such, I believe it is a valuable alternative for researchers with specific aims of providing interpretable research for decision-making.