Tree-Based Analysis: A Practical Approach to Create Clinical Decision Making Tools

Abstract

Tree-based methods have become one of the most flexible, intuitive, and powerful data analytic tools for exploring complex data structures. Tree-based methods provide a natural framework for creating patient subgroups for risk classification. In this paper, we review methodological and practical aspects of tree-based methods, with a focus on diagnostic classification (binary outcome) and prognostication (censored survival outcome). Creating an ensemble of trees improves prediction accuracy and addresses instability in a single tree. Ensemble methods are described that rely on resampling from the original data. Finally, we present methods to identify a representative tree from the ensemble, that can be used for clinical decision-making. The methods are illustrated using data on ischemic heart disease classification, and data from the SPRINT trial on adverse events in patients with high blood pressure.

1. Introduction

“Data!data!data!” he cried impatiently. “I can’t make bricks without clay.” ———- Arthur Conan Doyle, in The Adventure of the Copper Beeches.

Tree-based methods are considered to be one of the best and most commonly used statistical learning methods for analyzing complex data. Tree-based methods produce predictive tools with high accuracy, stability and ease of interpretation. The applications of these methods are far reaching. The best documented, and arguably most popular uses of tree-based methods are in biomedical research where classification is a central issue. For example, a clinician may be very interested in the following question: Is this patient with chest pain suffering a heart attack, or does he simply have a musculoskeletal pain [1]? To answer this question, information on the patient must be collected, and a good diagnostic test utilizing such information must be in place. Tree-based analyses provide one solution for constructing such diagnostic tests [1,2].

Originally tree-based methods were developed for classification and subsequently regression [3]. Later, interest in tree-based methods for survival (i.e. continuous time-to-event) data arose from the need of clinical researchers to define interpretable rules for prognostication, as well as identifying patient subgroups for clinical trial design [4–10]. Tree-based methods offer a natural way of “binning” patients, allowing researchers to compare risk estimates across the “bins” [4]. Furthermore, they are adept at capturing non-linear and heterogeneous effects, and can automatically detect interactions (i.e. without the need to having the interactions specified a priori). From these respects, tree-based methods offer potential improvement over traditional regression models.

More recently, single trees have been expanded to forests or ensemble of trees with the goal of achieving improved prediction accuracy and addressing instability that often plagues a single tree [11,12]. Representative tree from an ensemble offers a useful tool for gaining insights into clinical decision-making, while still harnessing the improved prediction accuracy of the ensemble [13]. Trees are appealing to clinicians because they serve as a very human representation to data. In contrast, statisticians find them appealing because they serve as a good base learner for ensemble methods.

The current paper provides a review of tree-based methods, with a focus on the methodological aspects, implementation and interpretation of results from such analyses. The target audience is clinical researchers and practitioners familiar with basic regression framework (e.g. linear regression, logistic regression, proportional hazards regression), but little or no prior experience with tree-based analyses. We present two heart disease examples, along with programming code and data sets, to illustrate the power of tree-based methods for diagnostic classification and prognostication.

2. Tree-based Methods

The literature on tree-based methods dates from work in the social sciences. In statistics, Breiman et al. had a seminal influence both in bringing the work to the attention of statisticians and in proposing new algorithms for constructing trees [3]. At around the same time decision tree induction was beginning to be used in the field of machine learning, and in engineering.

The terminology of trees is graphic: a tree T has a root which is the top node, and observations are passed down the tree, with decisions being made at each node (also called daughters) until a terminal node or leaf is reached. Each non-terminal node (also called internal node) contains a question on which a split is based. The terminal nodes of a tree T are collectively denoted by $\tilde{T}$, and the number of terminal nodes is denoted by $\left|\tilde{T}\right|$. Each terminal node contains the class label (for a classification problem) or an average response (for a least-squares regression problem). Nodes in a tree can also be chance, or probability, nodes. The branch T_t that stems from node t includes t itself and all its daughters. A subtree of T is a tree with root a node of T; it is a rooted subtree if its root is the root of T.

In the classification and regression tree (CART) paradigm, the covariate space is partitioned recursively in a binary fashion. The partitioning is intended to increase within-node homogeneity, where homogeneity is determined by the dependent variable in the problem. There are three basic elements for constructing a tree under the CART paradigm. These are: (1) tree growing, (2) finding the “right-sized” tree, and (3) testing. The first element is aimed at addressing the question how and why a parent node is split into daughter nodes. CART uses binary splits, phrased in terms of the covariates, that partition the predictor space. Each split depends upon the value of a single covariate. For ordered (continuous or categorical) covariates, X_j, only splits resulting from questions of the form “Is X_j ≤ c?” for c ∈ domain(X_j) are considered, thereby allowing at most n−1 splits for a sample of size n. For example, for patients who came to emergency departments with acute chest pain, systolic blood pressure was evaluated to predict the need for intensive care [1]. The recursive partitioning analysis assessed every possible split of the form “is systolic blood pressure less than or equal to 100 mm Hg, 105 mm Hg, 110 mm Hg etc.” (corresponding to each unique value of systolic blood pressure in the dataset except the maximum value). For nominal covariates no constraints on possible subdivisions are imposed. Thus, for a nominal covariate with M categories, there are 2^M−1−1 splits to examine. For example, in evaluating race/ethnicity (with categories whites, african americans, hispanics, others) as a potential splitter, there are a total of 7 splits to consider, namely (whites vs. the rest), (african americans vs. the rest), (hispanics vs. the rest), (asians vs. the rest), (whites and african americans vs. hispanics and others), (whites and hispanics versus african americans and others), and (whites and others vs. african americans and hispanics).

The natural question that comes next is, how do we select one or several preferred splits from the pool of allowable splits? Before selecting the best split, one must define the goodness of split. The objective of splitting is to make the two daughter nodes as homogeneous as possible. Therefore, the goodness of a split must weigh the homogeneities in the two daughter nodes. Extent of node homogeneity is measured quantitatively using an impurity function [3]. Potential splits are evaluated for each of the covariates, and the covariate and split value resulting in the greatest reduction in impurity is chosen.

Corresponding to a split s at node t into left and right daughter nodes t_L and t_R, the reduction in impurity is given by

$$\Delta I\left(s,t\right)=i\left(t\right)-P\left(t_L\right)i\left(t_L\right)-P\left(t_R\right)i\left(t_R\right),$$

where i(t) is the impurity in node t, and P (t_L) and P (t_R) are the probabilities that a subject falls in nodes t_L and t_R, respectively. For classification problems, i(t) is measured in terms of entropy or Gini impurity [3]. In most practical situations though, there is very little difference between the two criteria. For regression problems, i(t) is typically the mean residual sum of squares. The probabilities P(t_L) and P(t_R) are estimated via corresponding sample proportions. The splitting rule that maximizes ΔI(s, t) over the set S of all possible splits is chosen as the best splitter for node t. In the context of the chest pain study, systolic blood pressure (covariate chosen) of less than 110 mm Hg (value/threshold chosen for the specific covariate) was found to be the best splitter for a node in the tree based on goodness of split [1].

A useful feature of CART is that of growing a large tree and then pruning it back to find the “right-sized tree”. The point is to find a subtree of the “large” tree that is most “predictive” of the outcome and least vulnerable to the noise in the data. Let c(t) be the misclassification at a node t. Now define C(T) to be the misclassification for the entire tree $T:C\left(T\right)=\sum _{t\in \tilde{T}}P\left(t\right)c\left(t\right)$. Note that C(T) is a measure of the quality of the tree T. The purpose of pruning is to select the best subtree of an initially overgrown (or saturated) tree, such that C(T) is minimized. In this context, an important concept is tree cost-complexity [3]. It is defined as

$$C_{\alpha }\left(T\right)=C\left(T\right)+\alpha \left|\tilde{T}\right|,$$

where α (≥ 0) is a penalty parameter for the complexity of the tree. The total number of terminal nodes, $\left|\tilde{T}\right|$, is used as a measure of tree complexity. Note that the total number of nodes in a tree T (i.e. its size) is twice the number of its terminal nodes minus 1. Thus, tree complexity is really another term for the size of the tree. The difference between C_α (T) and C(T) as a measure of tree quality resides in that C_α (T) penalizes a large tree.

For any tree, there are many subtrees, and therefore many ways to prune. The challenge is how to prune, i.e. which subtrees to cut first. Breiman et al. [3] showed that (1) for any value of the penalty parameter α, there is a unique smallest subtree of T that minimizes the cost-complexity, and (2) if α₁ > α₂, the optimal subtree corresponding to α₁ is a subtree of the optimal subtree corresponding to α₂. The use of tree cost-complexity therefore allows one to construct a sequence of nested optimal subtrees from any given tree T. This is done by recursively pruning the branch(es) with the weakest link; that is, the node t with the smallest value of α such that C_α(t) ≤ C_α(T_t).

Having obtained a nested sequence of pruned optimal subtrees, one is left with the problem of selecting a best tree from this sequence. Using a test sample or cross-validation is recommended to obtain honest estimates of C(T). The subtree with the smallest misclassification cost is chosen as the final tree [3,14]. In the context of the chest pain study, the final tree had four terminal nodes that identified patient subgroups in terms of very low, low, moderate, and high risk of complications [1].

Lastly, it is worth mentioning that while CART is the most commonly used tree method in biomedical applications, other tree growing algorithms have been proposed that offer improvements on various aspects of CART. These include CHAID (Chi-Squared Automated Interaction Detection), QUEST (Quick, Unbiased and Effcient Statistical Tree), and GUIDE (Generalized, Unbiased Interaction Detection and Estimation) [15–18]. In contrast to CART, CHAID builds non-binary trees that tend to be “wider”, i.e. yielding many terminal nodes connected to a single branch, which can be conveniently summarized in a simple two-way table with multiple categories for each variable or dimension of the table. This makes the CHAID method particularly popular in market research applications. Also, CART can be biased towards selecting covariates with many possible splits (i.e. covariates which have many unique values in the observed data) or missing values. The QUEST and GUIDE algorithms, in contrast, provide unbiased alternatives for split variable selection. More recently, tree growing algorithms have been proposed based on a inference framework, which incorporates tree growing and stopping rule based on the concept of statistical significance and implemented via formal hypothesis tests [19].

2.1. Tree Ensemble

The mechanism of selecting a best split in CART and the recursive partitioning of data leads to smaller and smaller data sets. This can lead to instability in the tree structure, whereby small changes in the data and/or algorithm inputs can have dramatic effects on the nature of the solution (variables and splits selected). Another major shortcoming of CART is its modest prediction performance. Growing an ensemble (collection) of trees is a way to fix these problems [11,12,20]. It also leads to classifiers and predictors that are drawn from a richer class of models [21]. Ensemble methods such as bagging, random forest, and boosting can yield substantial prediction improvements over a single tree, and are known to be stable [11,12,20]. Improvements can be achieved through either bias or variance reduction. It is this principle (i.e. bias versus variance reduction) that motivates how an ensemble is grown.

Bagging [11] involves random manipulation of the training data via bootstrap to reduce variance. A large number of pseudo datasets are generated by resampling the original observations with replacement, and a tree grown on each pseudo dataset. This results in an ensemble of trees. Predictions from individual trees are averaged to make the overall prediction from the ensemble, thereby reducing variance. In boosting [20], instead of random resampling, the data are iteratively reweighted to reduce bias. The algorithm alternates between fitting a tree and reweighting the data. The weights are adaptively chosen, with more weight given to observations that the tree models poorly. An ensemble of trees result, that serves to reduce the bias.

The simple mechanism whereby bagging reduces prediction error, is well understood in terms of variance reduction resulting from averaging [20]. Such variance gains can be enhanced by reducing the correlation between the quantities being averaged. Random forest, a second-generation bagging method, is based on this principle [12]. It is an ensemble of unpruned classification or regression trees, induced from bootstrap resamples of the training data, using random feature selection in the tree induction process. Correlation reduction is achieved by the random feature selection. Instead of determining the optimal split of a given node of a tree by evaluating all allowable splits on all covariates, as is done with growing a single tree, a subset of the covariates drawn at random is employed. The process (to grow the trees in the forest) is as follows: (1) Bootstrap the training data. Grow each tree on an independent bootstrap sample, (2) At each node, randomly select m covariates out of all M possible covariates. For theoretical reasons, the optimal choice of m is typically $\sqrt{M}$. Find the best split on the selected m covariates, (3) Grow the tree to maximal depth under the restriction of minimum nodesize (i.e. splitting is stopped when a node has fewer than a pre-specified “small” number of patients, typically 5 or 10). No pruning is performed. (4) Repeat for each bootstrap sample. (5) Prediction is made by aggregating (majority vote for classification or averaging for regression) the predictions of the ensemble.

Estimate of prediction error is best obtained using an external “test” dataset. In the absence of such data, an honest estimate of the prediction error is obtained as follows: (1) At each bootstrap iteration, predict the data not in the bootstrap sample (this is termed “out-of-bag” data) using the tree grown with the bootstrap sample. (2) Average the out-of-bag predictions. Calculate the error rate using the out-of-bag predictions, and call it the out-of-bag estimate of error. Given that enough trees have been grown, the “out-of-bag” estimate of error is an accurate estimate of test set prediction error rate [12]. Random forests demonstrate exceptional prediction accuracy, comparable to artificial neural networks and support vector machines [12]. Furthermore, a by-product of forest is a ranking of covariates in terms of the variable importance [12] in the forest. The ranking is done based on the idea that if a variable is not important, then rearranging the values of that variable will not affect prediction accuracy. Finally, some interesting extensions of Breiman’s random forest include weighted forests that incorporate tree-level weights to emphasize more accurate trees in prediction [22], and causal random forests for estimation and inference of heterogeneous treatment effects [23,24].

2.2. Representative Tree from Ensemble

Compared to a single tree, ensemble methods typically offer substantial gains in predictive accuracy, and are known to be stable. However, individual trees are lost in the ensemble. This is a significant loss, given that a tree is often used for clinical insights and decision making. While a tree from a single run should not be used for decision making, a representative tree from an ensemble may offer clinical insights, while still harnessing the improved prediction accuracy of the ensemble [13]. Methods have been proposed to identify the most representative tree from an ensemble [13]. Quite often, the problem may not be as bad as it seems; although hundreds of distinct trees are identified, many will differ only at a few nodes. Other trees may have different architectures, but produce similar partitions of the covariate space. By defining several similarity metrics on trees, an ensemble can be summarized by several representative trees [13]. Common choices of metrics include “similarity” in the terminal nodes (i.e. how similarly patients are binned), and in predictions.

The similarity score D(T) for a tree T is computed by averaging the individual distance metrics between tree T and all other trees in the ensemble. This is the average distance between tree T and all other trees in the ensemble. So a low score for a tree indicates its similarity to all other trees in the ensemble. The score D(T) is computed for each of the distance metrics (i.e. d₀, d₁, d₂, $d_1^{\ast }$) and the representative trees in the ensemble are chosen based on the smallest D(T) values [13].

2.3. Trees for Survival Data

For survival data, typically the outcome variable is time (continuous) to occurrence of an event of interest. The event can be death, occurrence of a disease, recurrence of a disease, etc. The time to event or survival time can be measured in days, weeks, years, etc. For example, if the event of interest is myocardial infarction, then the survival time can be the time in years until a person develops a myocardial infarction.

Observations are censored when the information about their survival time is incomplete; the most commonly encountered form is right censoring. Suppose patients are followed in a study for 20 weeks. A patient who does not experience the event of interest for the duration of the study is said to be right censored. The survival time for this person is considered to be at least as long as the duration of the study. Another example of right censoring is when a person drops out of the study before the end of the study observation time and did not experience the event. This person’s survival time is said to be censored, since we know that the event of interest did not happen while this person was under observation. Censoring is an important issue in survival analysis, representing a particular type of missing data, and needs to be appropriately accounted in order to avoid bias.

Several extensions of CART have been proposed in the above setting [5–10]. Algorithms for growing trees for survival data can be broadly classified under two general approaches. The first involves quantifying the within-node homogeneity with a statistic that measures how similar the subjects are in each node and choosing splits that minimize the within-node error. One such algorithm assumes the popular semiparamateric Cox model at each node, and constructs the within-node measure based on residuals from the Cox model [5,8]. In this approach, the entire CART engineering is adopted, including cost-complexity pruning and cross-validation for final tree selection.

An alternative approach is to summarize the dissimilarity in survival experiences between two groups induced by a split and choose splits that maximize this difference [9,10]. One such algorithm uses the two-sample log-rank statistic to measure the separation in survival times between two daughter nodes. The two-sample log-rank statistic is chosen because of its extensive use in the survival analysis setting, and also because it is an appropriate measure of dissimilarity in survival between two groups. Partitioning a node, involves finding the split s, among all variables that maximizes the two-sample log-rank statistic, and hence the between-node separation. In this context, an optimal pruning algorithm that borrows the idea of weakest link cutting from CART, has been suggested [9]. The algorithm, namely split-complexity pruning, repeatedly prunes off branches with smallest average log-rank statistics in the branch [9]. Once a sequence of optimally pruned subtrees is obtained, the next step is to select the best tree from this sequence. Since the same data are used to select the split point and variable, as well as to calculate the statistic, bootstrap resampling is used to obtain a bias-corrected version of the split complexity for the final tree selection.

Methods for constructing survival tree ensembles have also been proposed in the literature [25–30]. One approach involves substituting suitably chosen residuals for the survival endpoint. The most popular choice is null deviance residuals from a Cox proportional hazards model [29]. This allows the analyst to apply random forest methodology suitable for continuous data to the transformed outcome, thereby circumventing difficulties induced by censoring. Another approach builds a relative risk forest under a proportional hazards assumption by exploiting an equivalence to Poisson tree likelihoods and using existing random forest methodology [28]. A third approach [26] uses log-transformed survival time as the outcome in a weighted random forest regression analysis. Observations in the regression analysis are weighted by, what are referred to as, inverse probability of censoring weights. All of these aforementioned approaches recast the survival setting into one that can be treated using existing forest methodology. A more recent approach directly incorporates survival time and censoring information in the splitting criterion for growing a forest [30]. Ensemble predictions from the forest account for the censored data structure directly, and may therefore provide a more interpretable analytic framework from a clinical standpoint. The tree that is most similar to other trees in the ensemble in terms of binning patients and survival predictions, is chosen as the most representative tree from the ensemble [13].

3. Methods

3.1. Example Datasets

3.1.1. Ischemic Heart Disease Classification

Ischemic heart disease (IHD) affects millions of people and is the most common cause of death worldwide [31]. IHD is typically caused by a partial or total obstruction of a coronary artery due to atherosclerotic coronary artery disease (CAD), causing reduced myocardial blood flow and ischemia.

In patients with single vessel CAD, percutaneous coronary intervention (PCI) is the recommended reperfusion strategy. However, a large number of patients referred for PCI because of suspected CAD do not have any angiographic evidence of coronary artery obstruction (<50% stenosis) when examined [32]. For example, up to 14% of all patients with biomarker confirmed myocardial infarction (MI) do not have obstructive CAD [33,34]. MI with no obstructive coronary arteries (MINOCA) represents a heterogeneous patient group of both cardiac and non-cardiac conditions mimicking MI, such as coronary artery spasm, perimyocarditis, thrombophilia disorders, demand-induced, or type 2 MI [33,34,35]. It is not possible to distinguish between MINOCA and MI by routine clinical examinations.

In this first example, we demonstrate the use of tree-based methods to classify patients for having obstructive CAD (binary outcome: yes/no), defined as having coronary artery stenosis diameter of less than 50%. The analytic cohort comprised of 920 patients, of whom 509 (55.3%) had CAD. The following sociodemographic and clinical characteristics were included as covariates in the analyses: patient age (continuous), gender (binary), resting blood pressure (continuous: mmHg), cholesterol level (continuous: mg/dL), maximum heart rate achieved during stress test (continuous: bpm) ST depression induced by exercise relative to rest (continuous), fasting blood pressure of at least 120 mg/dl (binary: yes/no) exercise induced angina (binary: yes/no), chest pain (binary: yes/no), resting electrocardiographic results (categorical: normal, ST-T wave abnormality, probable or definite ventricular hypertrophy), slope of the peak exercise ST segment (categorical: upsloping, flat, down sloping), number of major vessels colored by fluoroscopy (ordinal: 0,1,2,3), and myocardial perfusion heart scan results (categorical: normal, fixed defect, reversible defect).

3.1.2. Adverse Heart Event Prognostication

Hypertension affects almost one third of the US population [36]. If untreated, hypertension is associated with an increased risk of MI, stroke, heart failure, renal failure and death [37]. Current guidelines on hypertension recommend a treatment goal of systolic blood pressure below 140–150 mm Hg depending on age, and below 130 mm Hg in patients with diabetes [36–39].

The Systolic Blood Pressure Intervention Trial (SPRINT) trial recently showed that targeting a blood pressure of less than 120 mm Hg compared to 140 mm Hg in non-diabetic patients further reduced the overall risk of adverse cardiovascular events [39]. The SPRINT trial focused on patients who were at high risk of cardiovascular events (i.e. patients with previous cardiovascular disease and chronic kidney disease). It is unknown whether the same benefits apply to patients with isolated hypertension.

In this second example using the SPRINT data, we illustrate the use of tree-based methods to risk stratify patients based on clinical characteristics. The analytic cohort comprised of 9361 patients enrolled between November 2010 and March 2013. The outcome of interest is time to any adverse event (i.e. myocardial infarction, acute coronary syndrome, stroke, heart failure or CVD death). Thus, for this example, the outcome is continuous and censored. Median follow-up was 3.2 years. Of the 9361 patients, 562 experienced an event during the follow-up period. The following covariates were included in the analyses: smoking status (never, former, current), race (black, Hispanic, white, other), gender, age, cholesterol (mg/dL), glucose (mg/dL), HDL (mg/dL), triglycerides (mg/dL), urine albumine, serum creatinine (mg/g), eGFR, BMI, systolic blood pressure (mmHg), diastolic blood pressure (mmHg), Framingham 10-year CVD risk (> 15%or ≤ 15%), anti-hypertensive agents (yes/no), daily aspirin (yes/no), statin (yes/no), and clinical or subclinical CVD (yes/no) defined according to the SPRINT trial protocol [39].

3.2. Statistical Analyses

All analyses were performed using the statistical program R v.3.2.3. For our first case study (heart disease classification problem: binary outcome), we used the machine learning in R (mlr) package to analyze the data [40]. We specifically called upon the recursive partitioning (rpart) and random forest (randomForest) packages to create each of the classifiers [41,42]. For the second case study (SPRINT data: censored survival outcome), we used the rpart package again to generate the single tree analysis. Survival tree ensemble and random forest analyses were performed using a fast implementation of random forests (ranger) package [41,43].

For the ensemble methods, we generated 1000 bootstrap samples from the data to grow bagged as well as Random Forest classifiers, and predictions were obtained using out-of-bag data. We used Harrell’s c-statistic [44] to assess prediction accuracy. Most experts state that a c-statistic of 0.50 is the same as a coin toss, a c-statistic in the range of (0.5–0.7] represents “poor” discrimination, (0.7–0.8] adequate, (0.8–0.9] excellent, and >0.9 represents outstanding discrimination [45,46]. For all three methods, estimates of prediction accuracy were obtained using out-of-bag data. In addition, for the single tree analyses, we also used 10-fold cross validation (which is the industry standard for CART) to obtain prediction accuracies. The resulting c-statistics are very similar to those obtained using out-of-bag data.

Finally, we compared our tree based analyses results to logistic regression for the heart disease classification study, and Cox proportional hazards regression for the SPRINT study. To make a fair comparison between the traditional methods and tree based methods, we first performed stepwise variable selection to choose covariates for the logistic regression or Cox model.

4. Results

4.1. Ischemic Heart Disease Classification

The goal of this analysis was to build a clinician friendly predictive tool for classifying patients as having obstructive CAD (yes/no), based on clinical and sociodemographic characteristics. Figure 1 shows the single best tree with a c-statistic of 0.81. At each level of the tree, we show the best splitter (covariate with cutpoint). Circles denote terminal nodes in the tree. For each terminal node, we present the class prediction for that node (CAD or no CAD), the total number of patients (denoted by n), and the crude number of patients with CAD (denoted by r) in the terminal node. Competitor splits (i.e. covariate with cutpoint that had the second largest value of the split statistic) were also generated at each step of the tree (data not shown). Knowledge of such splits enable the elucidation of alternate, competing models.

Figure 1:

Single Tree Analysis for Heart Disease Classification.

The root node was split by chest pain; patients with no chest pain were separated from those with chest pain. Interestingly, probability analysis had established almost forty years ago that the clinical history of chest pain was more reliable than stress testing to make a diagnosis of obstructive coronary disease [47]. The competitor split for chest pain at the root node was exercise induced angina. The patient subgroup with chest pain was further split by age (<56 versus ≥ 56). The competitor split at this step was ST depression induced by exercise relative to rest. Of note, the subgroup with chest pain and age<56 years had the lowest risk of obstructive CAD (16%) and formed terminal node I. The subgroup with chest pain, and age 56 years was next split by gender. Females older than 56 years with chest pain had a 20% risk of having obstructive CAD, whereas males of the same age group with chest pain had a 56.4% risk of having CAD. The subgroup with no chest pain was not split any further, and formed terminal node IV with the highest risk of CAD (79%).

Next, we built an ensemble of heart disease classifiers using 1000 bootstrap samples from the original data. The c-statistic for the bagged ensemble was 0.89 and that for the random forest was 0.88. Both provide a boost in classification accuracy compared to the single tree. Figure 2 shows the variable importance plot from the random forest. The most representative tree from the random forest is in Figure 3. This tree is most similar to all other trees in the forest in terms of patient risk prediction. Similar to the single tree analysis results, the representative tree also splits first by chest pain, followed by age. Additionally, the representative tree splits by patient resting electrocardiographic results, and ST depression levels induced by exercise relative to rest. This tree identified five terminal nodes, with the risk of CAD varying from 16% to 79% in the lowest to highest risk groups (similar to the single tree analyses).

Figure 2:

Variable Importance from Random Forest for Heart Disease Classification.

Figure 3:

Representative Tree from Random Forest for Heart Disease Classification.

Logistic regression with stepwise selection chose a final model with age, gender, chest pain, maximum heart rate achieved during stress test, exercise induced angina, ST depression induced by exercise relative to rest, fasting blood pressure and cholesterol level as significant predictors of obstructive CAD. The c-statistic for the logistic regression model was 0.89.

4.2. Adverse Heart Event Prognostication

The goal of this second analysis was to build a clinical tool to help stratify patients with hypertension into adverse heart prognosis groups, using sociodemographic and clinical variables. Figure 4 shows the single best tree based on the assumption of a Cox model at each node. The c-statistic for this tree with three terminal nodes was 0.64. At each level of the tree, we show the best splitter (covariate with cutpoint). Circles denote terminal nodes in the tree. For each terminal node, we present the total number of patients (denoted by n), the (crude) number of events (denoted by r), and the estimated 3-year survival (denoted by 3Yr) for patients in that terminal node. Once again, competitor splits were also generated at each step of the tree (data not shown).

Figure 4:

Single Tree Analysis for SPRINT data.

The single tree was very simple, with the first split based on a patient’s history of subclinical CVD. Patients without any history of subclinical CVD were further split by age. The best prognosis was for patients age < 71 years without subclinical CVD, and this group had an estimated 3-year survival of 97.2%. The worst prognosis was for patients with history of subclinical CVD, and this group had an estimated 3-year survival of 88%.

Next, we built an ensemble of survival trees using 1000 bootstrapped samples from the original data. The c-statistic for the bagged ensemble was 0.66 and that for the random forest was 0.67. The ensemble resulted in a very small boost in prediction accuracy compared to the single survival tree. Figure 5 shows the variable importance plot from the random survival forest. The most representative tree from the forest is in Figure 6. This tree was most similar to all other trees in the forest in terms of 3-year survival predictions. Based on this representative tree, the best prognosis was for patients with urine albumin < 15.8 and age < 76 years. The estimated 3-year survival for this group was 97%. In contrast, the worst prognosis was for patients with urine albumin ≥ 15.8 and history of subclinical CVD. The estimated 3-year survival for this group was 81%.

Figure 5:

Variable Importance from Random Forest for SPRINT data.

Figure 6:

Representative Tree from Random Forest for SPRINT data.

Cox proportional hazards regression with stepwise selection chose a final model with age, smoking status, systolic blood pressure, total cholesterol, HDL, urine albumin, serum creatinine, eGFR, daily aspirin use, use of anti-hypertensive agent, and history of clinical or subclinical CVD as significant predictors of survival. The c-statistic for the Cox model was 0.69.

5. Conclusions

Clinical decision-making tools are often developed using regression models that rely on restrictive assumptions (e.g. proportional hazards). Observed data may not always conform to such assumptions. Since these models are often parametric, nonlinear effects of variables must also be assessed using ad hoc approaches, such as stepwise regression, or using transformations. Furthermore, discovery of interactions is di cult, because the interactions have to be specified a priori. All these difficulties can be automatically handled by using tree-based methods.

Traditional regression models favor global effects, that is, factors that have uniform effect across the entire patient population. Tree-based methods, however, can uncover factors that may act differently in different patient subgroups. This is an important point, given that in an individual patient, the discriminating power of one factor may be significantly enhanced or overshadowed by the presence or absence of other factors. Also, tree-based methods provide a superior means for risk classification. Rather than fitting a model to the data, tree-based methods sequentially divide the patient group into two subgroups based on prognostic factor values (e.g., age < 56 years vs ≥ 56 years). The repeated partitioning creates “bins” of patients that are approximately homogeneous. This permits the use of risk estimation to compare across the “bins”. The combination of binning and the interpretability of the resulting tree structure make these methods extremely well suited for use as clinical decision-making tools.

In addition to discrimination (as assessed by the c-statistic), the evaluation of a tree-based model (for that matter, any statistical prediction model) should ideally also include its calibration. Compared with discrimination, which indicates the ability of a risk model to rank order an individual patient’s risk, calibration measures the ability of a model to predict the absolute risk that is observed. For a classification problem, the idea is to produce predicted probabilities of a patient belonging to each possible class, instead of predicting class values directly. Unfortunately, tree based methods and other non-linear machine learning methods such as support vector machines, often produce poorly calibrated probabilities, and may benefit from recalibration in certain scenarios. Although outside the scope of this paper, probability calibration trees [48] have recently been developed to address this issue.

We illustrate tree-based methods using two heart disease datasets. For the ischemic heart disease data the endpoint was binary, whereas for the SPRINT data the endpoint was censored time. We illustrate the ease with which tree-based methods can handle these various outcome types. There are many versions of free-wares implementing tree-based methods. Most of these are available as R packages through CRAN (http://cran.r-project.org/). Specific references for rpart, randomForest, and ranger are provided in the bibliography [40–43]. Finally, programming codes for our analysis are provided in an Appendix.