Gait Speed and Survival of Older Surgical Patient with Cancer: Prediction after Machine Learning

Abstract

Purpose:

Gait speed in older patients with cancer is associated with mortality risk. One approach to assess gait speed is with the ‘Timed Up and Go’ (TUG) test. We utilized machine learning algorithms to automatically predict the results of the TUG tests and its association with survival, using patient-generated responses.

Methods:

A decision tree classifier was trained based on functional status data, obtained from preoperative geriatric assessment, and TUG test performance of older patients with cancer. The functional status data were used as input features to the decision tree, and the actual TUG data was used as ground truth labels. The decision tree was constructed to assign each patient to one of three categories: “TUG <10 seconds”, “TUG ≥ 10 seconds”, and “uncertain.”

Results:

In total, 1901 patients (49% women) with a mean age of 80 years were assessed. The most commonly performed operations were urologic, colorectal, and head and neck. The machine learning algorithm identified three features (cane/walker use, ability to walk outside, and ability to perform housework), in predicting TUG results with the decision tree classifier. The overall accuracy, specificity, and sensitivity of the prediction were 78%, 90%, and 66%, respectively. Furthermore, survival rates in each predicted TUG category differed by approximately 1% from the survival rates obtained by categorizing the patients based on their actual TUG results.

Conclusions:

Machine learning algorithms can accurately predict the gait speed of older patients with cancer, based on their response to questions addressing other aspects of functional status.

BACKGROUND

The number of people aged 65 years or older is expected to rise from 43.1 million in 2012 to 83.7 million in 2050 [1]. Because the incidence of cancer increases with age, it is expected that more older patients will be diagnosed with cancer in the next decades. Among older patients with cancer, those who are frail are at the highest risk for developing adverse events, such as toxicity from chemotherapy, and complications from surgery [2]. Research shows that instead of age, a patient’s level of fitness should inform cancer treatment decision making [3]. Geriatric oncology experts utilize a geriatric assessment (GA), to assess patients’ fitness for cancer treatment. The GA is a multidimensional assessment of older patients with cancer that includes an assessment of functional status as well as other parameters such as weight loss or polypharmacy [3]. Functional status assessment of older adults with cancer in the GA relies on both subjective and objective assessments. For example, patients are asked to report whether they are independent in performing basic activities of daily living, such as grooming or feeding, and in instrumental activities of daily living, such as transportation or shopping [4,5]. To obtain an objective assessment of patients’ functional status, geriatric oncology experts utilize tests such as the ‘Timed Up and Go’ test (TUG) [6]. In this test, a patient is instructed to get up from a chair without using their arms, walk ten feet, turn around, and return to the chair. The time to perform the TUG is measured in seconds [6]. In non-cancer settings, slower TUG s peed has been associated with poor performance in cognition and memory tests, [7] and to predict risk for falls and fractures [8]. In a cancer setting, TUG has been used as a test within GA [9]. It is shown that older patients with cancer with slower TUG times are at twice the risk for developing postoperative complications compared to patients with faster times [10]. TUG speed has also been associated with early death with first line chemotherapy. In a study on 348 patients with a median age of 77 years, patients with slower TUG speed were 2.55 times more likely to die within six months of starting first line chemotherapy [11]. Despite these prognostic benefits, administering TUG presents challenges. Completing the TUG requires staff time, and the error rate between different observers could be substantial. In a study on 78 participants with an average age of 84 years, TUG was assessed by 20 physical therapists. TUG was checked in 3 sessions all within one week. One physical therapist assessed TUG on session 1 and 2, and then another physical therapist assessed TUG for the third session. The interquartile difference between the assessments in the first two sessions by the same assessor, ranged from −2.1 to 3.2 seconds. The interquartile difference between assessments by the two assessors, ranged from −3.1 to 3.4 seconds. The variability increased as patients’ TUG slowed. For example, the 95% confidence interval for a TUG of 20 seconds was 14.8 to 27 seconds, while the 95% confidence interval for a patient with a TUG of 30 seconds was 22.2 to 40.5 seconds [12]. Furthermore, in another study on 235 patients with advanced chronic obstructive pulmonary disease, congestive heart failure, or chronic renal failure, three TUG assessments were performed by one assessor. The absolute minimal detectable change was as high as 4.5 seconds, corresponding to as high as a 35% difference between t h e first and second trials [13]. In addition to these personnel-associated limitations, the TUG test needs to be conducted within a clinical facility and not remotely.

We have shown that some components of GA can be acquired remotely [14]. If these components can be shown to correlate with TUG times, performing the TUG test in clinics can be potentially avoided or at least reduced substantially.

The goal of this project was to utilize a machine learning approach to develop an algorithm based on components of the GA, other than TUG, to accurately predict which patients will have slower TUG times. Due to the demonstrated relationship between TUG and overall survival, we then compared the survival rate of patients based on their predicted TUG scores against the performed TUG results. If our algorithm predicts TUG accurately, then it can serve as a screening tool for TUG assessment. In this situation, TUG can be performed only on patients that may have slower TUG times as suggested by the machine learning algorithm.

METHODS

The Memorial Sloan Kettering Cancer Center (MSKCC) Institutional Review Board authorized analysis of electronic medical records (EMR) of patients who presented at the Geriatrics Service Clinic for preoperative evaluation, and underwent surgery within a month at Memorial Hospital (MH). The data were collected between 1/2015 and 12/2016 and included patients who were 75 years or older. The patients completed a geriatric assessment through an electronic tool called the electronic Rapid Fitness Assessment or (eRFA) [14]. The geriatric assessment examines functional status of the patients through assessment of basic activities of daily living (bADL), and instrumental activities of daily living (iADL) [4,5].

The patients were asked to indicate whether they were unable to perform a task, needed some help to perform a task, or could perform a task without any help, for bADL and iADL activities. bADL assesses seven basic activities including bathing, dressing, grooming, feeding, walking inside the home, walking outside the home, and bladder-bowel control. iADL assesses eight functional activities including telephone use, doing laundry, shopping, preparing meals, doing housework, self-medication, handling money and finances, and transportation. In addition, the patients were asked to report whether they use a cane, walker, or both. The assessment also included questions about the presence and number of hours of home health aide assistance, level of social support [15], limitations in social activities [16], depression [17], cognitive status[18,19], and two questions related to nutritional status and poly-pharmacy.

Geriatric nurses used TUG to determine each patient’s gait speed. Geriatric nurses had an average of 10 years of working experience at MSKCC Geriatrics service, where TUG is performed in the geriatrics clinics as routine care. MSKCC Geriatricians reviewed and confirmed the TUG assessment performed by the geriatric nurses. The results of TUG were then stratified into three categories: “less than ten seconds”, “ten to nineteen seconds”, and “twenty seconds or more”, and were entered into patient’s electronic medical records. Finally, we combined the results of the eRFA with other clinical evaluations to inform perioperative management [19], and construct our database.

Data Collection

eRFA data was entered into an institutional database at the time of collection. Information from this dataset was extracted and integrated with socio-demographic characteristics (e.g. age, gender, marital status, educational status, and living condition), retrieved from the MH electronic medical record. Patient mortality was retrieved by linking the dataset to the Social Security Death Index.

Data Cleaning

In the data cleaning phase, 134 patients were excluded from our data analysis because the TUG scores were not recorded at those entries. After this initial pre-processing, there were 4,936 missing items (3.8% of the total items) in the dataset. This amount of missing data was associated with only 5 variables. The missing values were imputed with the mean, median, or mode for that variable, depending on the characteristics of the variable [20]. Specifically, discrete unordered variables were filled via mode, continuous variables by mean, and any remaining variables were filled by medians.

Machine Learning

The primary goal of this study was to build an interpretable model that allows a clinician to accurately predict a patient’s TUG score with less than five GA questions. To achieve this goal, we focused on developing a machine learning approach that limits the number of questions to a maximum of three. Given answers to those questions, the machine learning algorithm predicts the patient’s TUG score as one of the three categories (i.e., classes), including “TUG less than ten seconds”, “TUG more than ten seconds”, and “uncertain”. The “uncertain” category refers to cases where further testing will be needed.

We utilized the decision tree algorithm to produce an adaptive questionnaire. A decision tree algorithm is computationally simple and provides a human-interpretable output for prediction. In a decision tree, each internal (non-leaf) node is labelled with an input feature (i.e., a question). A node will branch to a child node based on the provided answer to the question associated with its parent node. Starting from a single node, called the “root”, the decision tree algorithm generates a tree structure where each non-leaf node is associated with a survey question, and each leaf in the tree is labelled with a class (i.e., the decision), or a probability distribution over the classes.

We utilized scikit-learn [20], a machine learning library for Python, for construction validation of the decision tree model. Constructing a decision tree began by choosing a question at each iteration that best split the remaining data samples [21]. We measured the informativeness of each question by calculating the homogeneity of the target variable within the subsets. Specifically, we calculated a score called Gini Impurity [22], that indicates how well each question can split patients with TUG scores of less or greater than ten seconds, into two homogenous subsets. The process of decision tree construction continued by selecting the most informative questions and adding them to the tree, until the model reached a specified complexity (e.g., tree depth), or it could categorize all the provided training data correctly [23,24].

In the process of building a predictive model using machine learning, it is important to provide the machine representative yet unbiased training data, to help the machine learn the concept (reasons of having different values for TUG) better. Because the original data distribution was biased to a specific class, we utilized an under-sampling method [25] to balance the classes. Under-sampling attempts to balance the number of positive and negative samples by eliminating some of the majority samples from the training dataset. For example, if the majority of our data have TUG scores of less than ten seconds, under-sampling methods only select a subset of data with TUG less ten seconds, to facilitate the learning procedure.

Correlation Analysis

We performed correlation analysis to achieve two goals: 1) to show that the selected variables for building the proposed machine learning model had the strongest correlation with TUG, and 2) to provide researchers with the list of additional variables with strong correlations with the TUG, for future studies that will focus on external validation of our model. To identify the correlation strength between features in the eRFA and a patient’s TUG score, we used the Kendall rank correlation coefficient (tau) [26]. This score provides a value that ranges from −1 to 1 where a larger absolute value corresponds to a stronger correlation, which is either positive or negative. The advantages of this approach are that it does not assume that the data are normally distributed or that the relation between variables is linear. Because the test assumes that features are ordered, we excluded non-ordinal features (e.g., marital status) from this correlation analysis.

Association between Predicted and Actual TUG with Survival

We calculated patient survival rates within the two prediction groups, TUG > ten seconds and TUG < ten seconds, and compared the survival rates to those of patients with actual TUG > ten seconds and TUG < ten seconds, measured through TUG tests administered in clinics. The survival rates were calculated at the 60-, 180- and 360-day thresholds. Patients who underwent surgery and did not have adequate follow up time for each survival time analysis were excluded from this analysis.

RESULTS

Patients

In total, 1901 patients (median age of 80) were included in this analysis. Among all the subjects, 49% were women, 55.4% were married, and 48.4% were college graduates or had advanced degrees. The most common operations or procedures performed were urologic (24.7%), colorectal (9.7%), head and neck (9.5%), gynecologic (9.3%), hepatobiliary (8.3%), interventional radiologic (6.7%), thoracic (6.4%), and breast surgery (5.3%). The remaining surgical procedures were less than 5% of the sample. Among all the patients, 48.4% had outpatient procedures, and the rest required hospitalization. Table 1 shows a breakdown of characteristics of patients in each TUG category as well as those of all the patients.

Table 1.

Patients’ sociodemographic characteristics and geriatric assessment per TUG category and overall

	TUG < 10 seconds (1170, 61.5%)	TUG 10–19 seconds (493, 25.9%)	TUG ≥ 20 seconds (238, 12.5%)	Missing TUG (134, 6.6%)	P value
Age (mean/SD)	79.45 (4.1)	81.51 (5.0)	82.62 (5.4)	80.47 (4.8)	<0.001
Gender	509 (43.5%)	275 (55.8%)	147 (61.8%)	68 (50.7%)	<0.001
-Female
-Male	661(56.5%)	218 (44.2%)	91(38.2%)	66 (49.3%)
Married	696 (59.6%)	244 (49.5%)	92 (38.7%)	70 (54.7%)	<0.001
Living with family/partner	826 (70.7%)	304 (61.8%)	135 (57%)	77 (59.7%)	<0.001
College graduate or higher	608 (52.4%)	198 (40.3%)	96 (40.7%)	64 (48.6%)	<0.001
KPS	90.12 (11.3)	78.53 (16.2)	62.65 (16.5)	82.05 (17.19)	<0.001
bADL score	13.28 (1.2)	11.54 (2.7)	8.44 (3.8)	11.92 (3.3)	<0.001
iADL score	14.99 (2.2)	12.39 (3.8)	8.6 (4.5)	12.98 (4.4)	<0.001
Social support score	16.52 (3.9)	16.39 (3.7)	16.66 (3.4)	15.03(4.4)	0.66
Social activity limitation score	7.57 (2.3)	9.08 (2.6)	10.59 (2.8)	8.70 (2.9)	<0.001
Depression score	0.75 (0.9)	1.21 (1.2)	1..85 (1.2)	1.12 (1.0)	<0.001
Mini-Cog score	4.1(1.2)	3.63 (1.4)	3.01 (1.7)	3.45 (1.7)	<0.001
Type of surgery	544 (48%)	225 (50.3%)	97 (47.3%)	58 (47.2%)	0.656
-Same day
-Requiring hospitalization	590 (52%)	222 (49.7%)	109 (52.7%)	65 (52.8%)

Abbreviations: TUG = Timed Up and Go; SD = standard deviation; KPS = Karnofsky performance status; bADL = basic activities of daily living; iADL = instrumental activities of daily living

Decision Tree Algorithm

Figure 1 shows the learned decision tree algorithm for TUG score prediction. As the figure shows, it is possible to reach a leaf node by asking a patient, at most, three survey questions. The leaf nodes are associated with the three desired class labels, TUG > ten seconds, TUG < ten seconds, and uncertain. This figure also shows TUG score prediction accuracy. On average, the accuracy, specificity, and sensitivity of the developed decision tree were 78.04%, 90.28%, and 65.80%, respectively. In another word, of 1170 patients with actual TUG < 10 seconds, our model was able to predict TUG correctly in 913 patients (78%), incorrectly in 92 patients (7.9%), and was indeterminate in 165 patients (14.1%). On the other hand, out of 731 patients with actual TUG > 10 seconds, our model predicted TUG accurately in 473 patients (64.7%), incorrectly in 142 patients (19.4%), and was indeterminate in 116 patients (15.9%).

Figure 1.

The Timed Up and Go test (TUG) decision tree. Each leaf represents a classification of either TUG > ten seconds, TUG < ten seconds, or indeterminate. The accuracy of a given prediction, and samples percentage, from the original dataset which fall into each leaf are included as well. The model suggests that the get up and go frailty indicator is highly related to the fact that whether a patient uses an assistive device to walk or not. If a patient uses a cane or walker, his TUG score is > ten seconds with an accuracy of 84.3%. On the other hand, if the patient does not use a cane or walker to walk, it depends on his or her ability to walk outside the house. Then, if s/he is able to walk outside without assistance, with 86% accuracy, s/he has TUG less than ten seconds. However, if the patient’s walking outside function is impaired, and cannot do daily housework, the TUG is > ten seconds, with accuracy more than 77%.

In general, our model could confidently predict a patient’s condition 85% of the time. Our experiments show that the proposed under-sampling strategy improves the model accuracy from 65.2% to 80.8%.

Comparison of Survival between Actual TUG and Algorithmic TUG

A t-test showed that there is a significant difference between the survival rates of patients classified as TUG<10 and those classified as either TUG > ten seconds, with a p-value <0.1. Similarly, a chi-squared test showed that the probability that the variation in survival rates of patients between predicted and actual TUG was due only to chance is above the 99% threshold.

To further assess the relationship between these TUG frailty predictions and patient outcomes, the survival rates of patients in various TUG classes were calculated. The 360- day survival rate included 1136 patients, the 180-day survival included 1619 patients, and the 60-day survival included 1882 patients. In Table 2, the 60-, 180- and 360-day survival rates of patients in different prediction groups are shown against the survival rates of the patients in the respective groups. The survival rates for actual and predicted TUG groups were comparable with about 1% variation, and patients with higher TUG scores had a lower survival rate, as expected.

Table 2.

The survival rates of patients based on which predicted and actual Timed Up and Go test (TUG).

	60 day survival	180 day survival	360 day survival
Actual TUG <10 seconds	92.9%	88.7%	79.8%
Algorithmic TUG <10 seconds	93.3%	88.8%	80.4%
Actual TUG > 10 seconds	93.2%	85.7%	72.8%
Algorithmic TUG >10 seconds	91.7%	86.0%	73.7%

Correlation

Analysis

In addition to using machine learning to streamline the process of assessing patient TUG scores, we aimed to identify significant correlations between TUG scores and various demographic attributes available in a pre-op survey. This analysis helped us validate our decision tree model, since the strongest correlations are used in our model. This also gives us some intuition on which other variables are the candidates to add to the model, to get more accurate results. The ten strongest positive and negative correlations are shown in Figure 2. TUG scores were negatively correlated with all grooming and personal care functions in the dataset, in addition to use of movement assistive devices, home care, and social/activity interference. In particular, difficulty in performing the TUG test was associated with difficulty getting around both inside and outside, the ability to do housework, and the ability to prepare meals.

Figure 2.

The Kendall Rank Correlation Coefficients between Time Up and Go test (TUG) and ten positive and negative correlations.

DISCUSSION

The study’s goal was to predict patient TUG results through machine learning algorithms. We were able to develop a very short algorithm to accurately predict different categories of TUG.

It is critical to determine patient functional activity, especially in older patients with cancer, to inform treatment decision making. Patients who are more active or fit are able to tolerate the cancer treatment more than patients who are inactive or frail [27]. In older patients with cancer, the process of assessing fitness is commonly through a geriatric assessment (GA) [3], which emphasizes functional status. Functional assessment can be done either via patient-reported instruments such as bADL and iADL, or via objective assessments such as TUG. It has been shown that TUG correlates with overall mortality in older patients with cancer receiving chemotherapy [11]. Slower TUG was associated with increased probability of surgical complications and one-year mortality in older surgical patients with cancer [28]. TUG can also predict future falls in community-dwelling older adults [29]. In our center, we use preoperative TUG to guide the need for requesting physical therapy evaluation in the postoperative period. Furthermore, because patients with slower TUG usually have other aging-related impairments and could be at risk for falls, as time permitted, we discussed home safety with the patients and families in the preoperative period. If following this conversation, more concerns arose, the outpatient geriatrics team communicated this to the surgical team, physical therapy team, and the case managers.

However, performing TUG is not without challenges. First, while the test takes about ten to twenty seconds per patient, performing such a test in fast-paced oncology clinics as routine care may not be feasible. As a result, there is a need to develop a short screening questionnaire or algorithm that can predict patient’s TUG with high accuracy, sensitivity, and specificity. Our algorithm can be used to screen patients for the need to perform TUG prior to being seen by a clinician, limiting administration to only 15% of patients, because our algorithm was able to predict TUG in 85% of population. Our algorithm, consisting of only three questions - querying the patient’s use of assistive device, ability to walk outside of home without any limitations, and ability to perform housekeeping tasks without any help - can predict TUG scores with 78% accuracy. To the best of our knowledge, our study is the first to utilize a machine learning algorithm to develop a quick procedure for patient TUG performance prediction.

One of the limitations of this study is that it was conducted in a single institution as a retrospective study in a cohort of older patients with cancer at the time of preoperative evaluation. In addition, the proposed model could increase its reliability and certainty if the machine learning procedure had access to a larger surgical population data. Although appropriate methods were used to account for missing data, less missing data would be preferred. Future studies should be conducted on external validation of the model.

Different studies used different cutoff scores [30–32] for the TUG. However, a descriptive meta- analysis suggests that a TUG of nine to ten seconds be used as a cutoff for slow walking [33]. In our study, nurses categorized TUG scores into three groups. As a result, the exact TUG score was unknown. Future studies should focus on whether machines can predict the exact TUG. Analysis of the GA dataset showed that the data suffered from an imbalanced class distribution, where 62% of the patients had a TUG score of less than ten seconds, and less than 48% of the patients had a TUG score of greater than ten seconds. This imbalanced class distribution biases the algorithm towards predicting one of the classes. To address this problem, we utilized the previously discussed under-sampling method to balance the classes.

Despite this limitation, our study was performed on one of the largest datasets on older patients with cancer at the time of preoperative evaluation. All patients completed geriatric assessments as routine care, which eliminated healthy selection bias. Instead of reporting odds ratios, we have used a practical method to develop a step-by-step algorithm, which can be used in any fast-paced oncology clinic with limited resources.

Our motivation for choosing a decision tree as the machine learning algorithm is as follows:

• It is easy to understand and interpret which enables medical researchers to easily analyze and validate its results.

• A decision tree can handle both numerical and categorical questions included in the medical surveys [23].

• Decision trees are robust to outliers and noisy data [24].

• The structure of a decision tree is well suited to the requirement of reducing the number of questions.

These advantages make data preparation easier and more flexible in case of missing values. Based on the structure of the decision tree, more informative questions are closer to the root of the tree and less informative questions are closer to leaves of the tree.

CONCLUSIONS

This study showed that a simple decision tree was able to predict patient gait speed with high accuracy. The proposed decision tree can be used to screen patients who need further functional assessment or intervention. Future prospective studies are needed to provide external validation of our model, and assess its impact on health care processes, and outcomes of older patients with cancer.